Mercurial > repos > galaxyp > openms_qualitycontrol

annotate get_tests.py @ 5:ea02cc72f0b9 draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
author galaxyp
date Fri, 14 Jun 2024 21:46:38 +0000
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
1 #!/usr/bin/env python
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
2
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
3 import argparse
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
4 import os.path
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
5 import re
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
6 import shlex
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
7 import sys
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
8 import tempfile
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
9 from typing import (
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
10 Dict,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
11 List,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
12 Optional,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
13 TextIO,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
14 Tuple,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
15 )
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
16
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
17 from ctdconverter.common.utils import (
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
18 ParameterHardcoder,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
19 parse_hardcoded_parameters,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
20 parse_input_ctds,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
21 )
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
22 from ctdconverter.galaxy.converter import convert_models
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
23 from CTDopts.CTDopts import (
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
24 CTDModel,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
25 ModelTypeError,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
26 Parameters,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
27 )
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
28
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
29 SKIP_LIST = [
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
30 r"_prepare\"",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
31 r"_convert",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
32 r"WRITEINI",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
33 r"WRITECTD",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
34 r"INVALIDVALUE",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
35 r"\.ini\.json",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
36 r"OpenSwathMzMLFileCacher .*-convert_back", # - OpenSwathMzMLFileCacher with -convert_back argument https://github.com/OpenMS/OpenMS/issues/4399
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
37 r"MaRaClusterAdapter.*-consensus_out", # - MaRaCluster with -consensus_out (parameter blacklister: https://github.com/OpenMS/OpenMS/issues/4456)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
38 r"FileMerger_1_input1.dta2d.*FileMerger_1_input2.dta ", # - FileMerger with mixed dta dta2d input (ftype can not be specified in the test, dta can not be sniffed)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
39 r'^(TOPP_OpenSwathAnalyzer_test_3|TOPP_OpenSwathAnalyzer_test_4)$', # no suppert for cached mzML
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
40 r'TOPP_SiriusAdapter_[0-9]+$', # Do not test SiriusAdapter https://github.com/OpenMS/OpenMS/issues/7000 .. will be removed anyway
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
41 r'TOPP_AssayGeneratorMetabo_(7|8|9|10|11|12|13|14|15|16|17|18)$' # Skip AssayGeneratorMetabo tests using Sirius https://github.com/OpenMS/OpenMS/issues/7150 (will be replaced by two tools)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
42 ]
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
43
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
44
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
45 def get_failing_tests(cmake: List[str]) -> List[str]:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
46 failing_tests = []
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
47 re_fail = re.compile(r"set_tests_properties\(\"([^\"]+)\" PROPERTIES WILL_FAIL 1\)")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
48
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
49 for cmake in args.cmake:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
50 with open(cmake) as cmake_fh:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
51 for line in cmake_fh:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
52 match = re_fail.search(line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
53 if match:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
54 failing_tests.append(match.group(1))
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
55 return failing_tests
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
56
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
57
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
58 def fix_tmp_files(line: str, diff_pairs: Dict[str, str]) -> str:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
59 """
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
60 OpenMS tests output to tmp files and compare with FuzzyDiff to the expected file.
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
61 problem: the extension of the tmp files is unusable for test generation.
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
62 unfortunately the extensions used in the DIFF lines are not always usable for the CLI
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
63 (e.g. for prepare_test_data, e.g. CLI expects csv but test file is txt)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
64 this function replaces the tmp file by the expected file.
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
65 """
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
66 cmd = shlex.split(line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
67 for i, e in enumerate(cmd):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
68 if e in diff_pairs:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
69 dst = os.path.join("test-data", diff_pairs[e])
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
70 if os.path.exists(dst):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
71 os.unlink(dst)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
72 sys.stderr.write(f"symlink {e} {dst}\n")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
73 os.symlink(e, dst)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
74 cmd[i] = diff_pairs[e]
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
75 return shlex.join(cmd)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
76
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
77
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
78 def get_ini(line: str, tool_id: str) -> Tuple[str, str]:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
79 """
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
80 if there is an ini file then we use this to generate the test
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
81 otherwise the ctd file is used
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
82 other command line parameters are inserted later into this xml
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
83 """
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
84 cmd = shlex.split(line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
85 ini = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
86 for i, e in enumerate(cmd):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
87 if e == "-ini":
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
88 ini = cmd[i + 1]
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
89 cmd = cmd[:i] + cmd[i + 2:]
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
90 if ini:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
91 return os.path.join("test-data", ini), shlex.join(cmd)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
92 else:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
93 return os.path.join("ctd", f"{tool_id}.ctd"), line
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
94
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
95
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
96 def unique_files(line: str):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
97 """
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
98 some tests use the same file twice which does not work in planemo tests
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
99 hence we create symlinks for each file used twice
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
100 """
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
101 cmd = shlex.split(line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
102 # print(f"{cmd}")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
103 files = {}
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
104 # determine the list of indexes where each file argument (anything appearing in test-data/) appears
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
105 for idx, e in enumerate(cmd):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
106 p = os.path.join("test-data", e)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
107 if not os.path.exists(p) and not os.path.islink(p):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
108 continue
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
109 try:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
110 files[e].append(idx)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
111 except KeyError:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
112 files[e] = [idx]
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
113 # print(f"{files=}")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
114 for f in files:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
115 if len(files[f]) < 2:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
116 continue
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
117 for i, idx in enumerate(files[f]):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
118 f_parts = f.split(".")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
119 f_parts[0] = f"{f_parts[0]}_{i}"
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
120 new_f = ".".join(f_parts)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
121 # if os.path.exists(os.path.join("test-data", new_f)):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
122 # os.unlink(os.path.join("test-data", new_f))
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
123 sys.stderr.write(
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
124 f'\tsymlink {os.path.join("test-data", new_f)} {f}\n'
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
125 )
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
126 try:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
127 os.symlink(f, os.path.join("test-data", new_f))
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
128 except FileExistsError:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
129 pass
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
130 cmd[idx] = new_f
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
131 return shlex.join(cmd)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
132
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
133
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
134 def fill_ctd_clargs(ini: str, line: str, ctd_tmp: TextIO) -> None:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
135 cmd = shlex.split(line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
136
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
137 # load CTDModel
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
138 ini_model = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
139 try:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
140 ini_model = CTDModel(from_file=ini)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
141 except ModelTypeError:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
142 pass
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
143 try:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
144 ini_model = Parameters(from_file=ini)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
145 except ModelTypeError:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
146 pass
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
147 assert ini_model is not None, "Could not parse %s, seems to be no CTD/PARAMS" % (
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
148 args.ini_file
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
149 )
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
150
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
151 # get a dictionary of the ctd arguments where the values of the parameters
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
152 # given on the command line are overwritten
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
153 ini_values = ini_model.parse_cl_args(cl_args=cmd, ignore_required=True)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
154 ini_model.write_ctd(ctd_tmp, ini_values)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
155
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
156
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
157 def process_test_line(
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
158 id: str,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
159 line: str,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
160 failing_tests: List[str],
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
161 skip_list: List[str],
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
162 diff_pairs: Dict[str, str],
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
163 ) -> Optional[str]:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
164
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
165 re_test_id = re.compile(r"add_test\(\"([^\"]+)\" ([^ ]+) (.*)")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
166 re_id_out_test = re.compile(r"_out_?[0-9]?")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
167
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
168 # TODO auto extract from set(OLD_OSW_PARAM ... lin
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
169 line = line.replace(
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
170 "${OLD_OSW_PARAM}",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
171 " -test -mz_extraction_window 0.05 -mz_extraction_window_unit Th -ms1_isotopes 0 -Scoring:TransitionGroupPicker:compute_peak_quality -Scoring:Scores:use_ms1_mi false -Scoring:Scores:use_mi_score false",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
172 )
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
173
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
174 line = line.replace("${TOPP_BIN_PATH}/", "")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
175 line = line.replace("${DATA_DIR_TOPP}/", "")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
176 line = line.replace("THIRDPARTY/", "")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
177 line = line.replace("${DATA_DIR_SHARE}/", "")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
178 # IDRipper PATH gets empty causing problems. TODO But overall the option needs to be handled differently
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
179 line = line.replace("${TMP_RIP_PATH}/", "")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
180 # some input files are originally in a subdir (degenerated cases/), but not in test-data
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
181 line = line.replace("degenerate_cases/", "")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
182 # determine the test and tool ids and remove the 1) add_test("TESTID" 2) trailing )
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
183 match = re_test_id.match(line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
184 if not match:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
185 sys.exit(f"Ill formated test line {line}\n")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
186 test_id = match.group(1)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
187 tool_id = match.group(2)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
188
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
189 line = f"{match.group(2)} {match.group(3)}"
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
190
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
191 if test_id in failing_tests:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
192 sys.stderr.write(f" skip failing {test_id} {line}\n")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
193 return
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
194
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
195 if id != tool_id:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
196 sys.stderr.write(f" skip {test_id} ({id} != {tool_id}) {line}\n")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
197 return
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
198
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
199 if re_id_out_test.search(test_id):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
200 sys.stderr.write(f" skip {test_id} {line}\n")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
201 return
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
202
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
203 for skip in skip_list:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
204 if re.search(skip, line):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
205 return
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
206 if re.search(skip, test_id):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
207 return
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
208
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
209 line = fix_tmp_files(line, diff_pairs)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
210 # print(f"fix {line=}")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
211 line = unique_files(line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
212 # print(f"unq {line=}")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
213 ini, line = get_ini(line, tool_id)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
214
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
215 from dataclasses import dataclass, field
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
216
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
217 @dataclass
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
218 class CTDConverterArgs:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
219 input_files: list
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
220 output_destination: str
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
221 default_executable_path: Optional[str] = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
222 hardcoded_parameters: Optional[str] = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
223 parameter_hardcoder: Optional[ParameterHardcoder] = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
224 xsd_location: Optional[str] = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
225 formats_file: Optional[str] = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
226 add_to_command_line: str = ""
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
227 required_tools_file: Optional[str] = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
228 skip_tools_file: Optional[str] = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
229 macros_files: Optional[List[str]] = field(default_factory=list)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
230 test_macros_files: Optional[List[str]] = field(default_factory=list)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
231 test_macros_prefix: Optional[List[str]] = field(default_factory=list)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
232 test_test: bool = False
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
233 test_only: bool = False
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
234 test_unsniffable: Optional[List[str]] = field(default_factory=list)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
235 test_condition: Optional[List[str]] = ("compare=sim_size", "delta_frac=0.05")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
236 tool_version: str = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
237 tool_profile: str = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
238 bump_file: str = None
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
239
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
240 # create an ini/ctd file where the values are equal to the arguments from the command line
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
241 # and transform it to xml
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
242 test = [f"<!-- {test_id} -->\n"]
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
243 with tempfile.NamedTemporaryFile(
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
244 mode="w+", delete_on_close=False
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
245 ) as ctd_tmp, tempfile.NamedTemporaryFile(
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
246 mode="w+", delete_on_close=False
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
247 ) as xml_tmp:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
248 fill_ctd_clargs(ini, line, ctd_tmp)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
249 ctd_tmp.close()
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
250 xml_tmp.close()
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
251 parsed_ctd = parse_input_ctds(None, [ctd_tmp.name], xml_tmp.name, "xml")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
252 ctd_args = CTDConverterArgs(
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
253 input_files=[ctd_tmp.name],
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
254 output_destination=xml_tmp.name,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
255 macros_files=["macros.xml"],
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
256 skip_tools_file="aux/tools_blacklist.txt",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
257 formats_file="aux/filetypes.txt",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
258 # tool_conf_destination = "tool.conf",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
259 hardcoded_parameters="aux/hardcoded_params.json",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
260 tool_version="3.1",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
261 test_only=True,
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
262 test_unsniffable=[
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
263 "csv",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
264 "tsv",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
265 "txt",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
266 "dta",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
267 "dta2d",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
268 "edta",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
269 "mrm",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
270 "splib",
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
271 ],
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
272 test_condition=["compare=sim_size", "delta_frac=0.7"],
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
273 )
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
274 ctd_args.parameter_hardcoder = parse_hardcoded_parameters(
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
275 ctd_args.hardcoded_parameters
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
276 )
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
277 convert_models(ctd_args, parsed_ctd)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
278 xml_tmp = open(xml_tmp.name, "r")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
279 for l in xml_tmp:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
280 test.append(l)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
281
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
282 return "".join(test)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
283
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
284
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
285 parser = argparse.ArgumentParser(description="Create Galaxy tests for a OpenMS tools")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
286 parser.add_argument("--id", dest="id", help="tool id")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
287 parser.add_argument("--cmake", nargs="+", help="OpenMS test CMake files")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
288 args = parser.parse_args()
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
289 sys.stderr.write(f"generate tests for {args.id}\n")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
290
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
291 re_comment = re.compile("#.*")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
292 re_empty_prefix = re.compile(r"^\s*")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
293 re_empty_suffix = re.compile(r"\s*$")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
294 re_add_test = re.compile(r"add_test\(\"(TOPP|UTILS)_.*/" + args.id)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
295 re_diff = re.compile(r"\$\{DIFF\}.* -in1 ([^ ]+) -in2 ([^ ]+)")
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
296 failing_tests = get_failing_tests(args.cmake)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
297 tests = []
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
298
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
299 # process the given CMake files and compile lists of
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
300 # - test lines .. essentially add_test(...)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
301 # - and pairs of files that are diffed
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
302 jline = ""
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
303 test_lines = []
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
304 diff_pairs = {}
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
305 for cmake in args.cmake:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
306 with open(cmake) as cmake_fh:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
307 for line in cmake_fh:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
308 # remove comments, empty prefixes and suffixes
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
309 line = re_comment.sub("", line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
310 line = re_empty_prefix.sub("", line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
311 line = re_empty_suffix.sub("", line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
312 # skip empty lines
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
313 if line == "":
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
314 continue
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
315
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
316 # join test statements that are split over multiple lines
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
317 if line.endswith(")"):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
318 jline += " " + line[:-1]
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
319 else:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
320 jline = line
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
321 continue
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
322 line, jline = jline.strip(), ""
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
323 match = re_diff.search(line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
324 if match:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
325 in1 = match.group(1).split("/")[-1]
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
326 in2 = match.group(2).split("/")[-1]
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
327 if in1 != in2:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
328 diff_pairs[in1] = in2
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
329 elif re_add_test.match(line):
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
330 test_lines.append(line)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
331
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
332 for line in test_lines:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
333 test = process_test_line(args.id, line, failing_tests, SKIP_LIST, diff_pairs)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
334 if test:
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
335 tests.append(test)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
336
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
337 tests = "\n".join(tests)
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
338 print(
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
339 f"""
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
340 <xml name="autotest_{args.id}">
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
341 {tests}
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
342 </xml>
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
343 """
ea02cc72f0b9 planemo upload for repository https://github.com/galaxyproteomics/tools-galaxyp/tree/master/tools/openms commit 5c080b1e2b99f1c88f4557e9fec8c45c9d23b906
galaxyp
parents:
diff changeset
344 )