Mercurial > repos > peterjc > effectivet3

annotate tools/effectiveT3/effectiveT3.py @ 9:512530020360 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
v0.0.18 Internal changes to command line handling
author peterjc
date Tue, 16 May 2017 09:17:17 -0400
parents 60a9b3f760cc
children a46d7861c32c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
1 #!/usr/bin/env python
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
2 """Wrapper for EffectiveT3 v1.0.1 for use in Galaxy.
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
4 This script takes exactly five command line arguments:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
5 * model name (e.g. TTSS_STD-1.0.1.jar)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
6 * threshold (selective or sensitive)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
7 * an input protein FASTA filename
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
8 * output tabular filename
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
9
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
10 It then calls the standalone Effective T3 v1.0.1 program (not the
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
11 webservice), and reformats the semi-colon separated output into
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
12 tab separated output for use in Galaxy.
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
13 """
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
14 import os
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
15 import subprocess
9
512530020360 v0.0.18 Internal changes to command line handling
peterjc
parents: 8
diff changeset
16 import sys
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
17
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
18 # The Galaxy auto-install via tool_dependencies.xml will set this environment variable
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
19 effective_t3_dir = os.environ.get("EFFECTIVET3", "/opt/EffectiveT3/")
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
20 effective_t3_jar = os.path.join(effective_t3_dir, "TTSS_GUI-1.0.1.jar")
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
21
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
22 if "-v" in sys.argv or "--version" in sys.argv:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
23 # TODO - Get version of the JAR file dynamically?
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
24 print("Wrapper v0.0.17, TTSS_GUI-1.0.1.jar")
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
25 sys.exit(0)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
26
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
27 if len(sys.argv) != 5:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
28 sys.exit("Require four arguments: model, threshold, input protein FASTA file & output tabular file")
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
29
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
30 model, threshold, fasta_file, tabular_file = sys.argv[1:]
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
31
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
32 if not os.path.isfile(fasta_file):
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
33 sys.exit("Input FASTA file not found: %s" % fasta_file)
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
34
9
512530020360 v0.0.18 Internal changes to command line handling
peterjc
parents: 8
diff changeset
35 if threshold not in ["selective", "sensitive"] and not threshold.startswith("cutoff="):
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
36 sys.exit("Threshold should be selective, sensitive, or cutoff=..., not %r" % threshold)
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
37
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
38
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
39 def clean_tabular(raw_handle, out_handle):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
40 """Clean up Effective T3 output to make it tabular."""
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
41 count = 0
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
42 positive = 0
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
43 errors = 0
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
44 for line in raw_handle:
9
512530020360 v0.0.18 Internal changes to command line handling
peterjc
parents: 8
diff changeset
45 if not line or line.startswith("#") or line.startswith("Id; Description; Score;"):
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
46 continue
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
47 assert line.count(";") >= 3, repr(line)
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
48 # Normally there will just be three semi-colons, however the
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
49 # original FASTA file's ID or description might have had
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
50 # semi-colons in it as well, hence the following hackery:
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
51 try:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
52 id_descr, score, effective = line.rstrip("\r\n").rsplit(";", 2)
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
53 # Cope when there was no FASTA description
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
54 if "; " not in id_descr and id_descr.endswith(";"):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
55 id = id_descr[:-1]
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
56 descr = ""
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
57 else:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
58 id, descr = id_descr.split("; ", 1)
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
59 except ValueError:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
60 sys.exit("Problem parsing line:\n%s\n" % line)
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
61 parts = [s.strip() for s in [id, descr, score, effective]]
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
62 out_handle.write("\t".join(parts) + "\n")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
63 count += 1
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
64 if float(score) < 0:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
65 errors += 1
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
66 if effective.lower() == "true":
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
67 positive += 1
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
68 return count, positive, errors
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
69
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
70
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
71 def run(cmd):
9
512530020360 v0.0.18 Internal changes to command line handling
peterjc
parents: 8
diff changeset
72 """Run the command line string via subprocess."""
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
73 # Avoid using shell=True when we call subprocess to ensure if the Python
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
74 # script is killed, so too is the child process.
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
75 try:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
76 child = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
77 except Exception, err:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
78 sys.exit("Error invoking command:\n%s\n\n%s\n" % (" ".join(cmd), err))
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
79 # Use .communicate as can get deadlocks with .wait(),
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
80 stdout, stderr = child.communicate()
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
81 return_code = child.returncode
7
5f85301d50bf v0.0.16, adding new model TTSS-STD-2.0.2.jar
peterjc
parents: 6
diff changeset
82 if return_code or stderr.startswith("Exception in thread"):
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
83 cmd_str = " ".join(cmd) # doesn't quote spaces etc
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
84 if stderr and stdout:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
85 sys.exit("Return code %i from command:\n%s\n\n%s\n\n%s" % (return_code, cmd_str, stdout, stderr))
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
86 else:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
87 sys.exit("Return code %i from command:\n%s\n%s" % (return_code, cmd_str, stderr))
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
88
7
5f85301d50bf v0.0.16, adding new model TTSS-STD-2.0.2.jar
peterjc
parents: 6
diff changeset
89
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
90 if not os.path.isdir(effective_t3_dir):
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
91 sys.exit("Effective T3 folder not found: %r" % effective_t3_dir)
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
92
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
93 if not os.path.isfile(effective_t3_jar):
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
94 sys.exit("Effective T3 JAR file not found: %r" % effective_t3_jar)
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
95
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
96 if not os.path.isdir(os.path.join(effective_t3_dir, "module")):
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
97 sys.exit("Effective T3 module folder not found: %r" % os.path.join(effective_t3_dir, "module"))
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
98
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
99 effective_t3_model = os.path.join(effective_t3_dir, "module", model)
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
100 if not os.path.isfile(effective_t3_model):
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
101 sys.stderr.write("Contents of %r is %s\n"
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
102 % (os.path.join(effective_t3_dir, "module"),
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
103 ", ".join(repr(p) for p in os.listdir(os.path.join(effective_t3_dir, "module")))))
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
104 sys.stderr.write("Main JAR was found: %r\n" % effective_t3_jar)
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
105 sys.exit("Effective T3 model JAR file not found: %r" % effective_t3_model)
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
106
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
107 # We will have write access whereever the output should be,
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
108 temp_file = os.path.abspath(tabular_file + ".tmp")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
109
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
110 # Use absolute paths since will change current directory...
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
111 tabular_file = os.path.abspath(tabular_file)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
112 fasta_file = os.path.abspath(fasta_file)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
113
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
114 cmd = ["java", "-jar", effective_t3_jar,
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
115 "-f", fasta_file,
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
116 "-m", model,
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
117 "-t", threshold,
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
118 "-o", temp_file,
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
119 "-q"]
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
120
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
121 try:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
122 # Must run from directory above the module subfolder:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
123 os.chdir(effective_t3_dir)
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
124 except Exception:
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
125 sys.exit("Could not change to Effective T3 folder: %s" % effective_t3_dir)
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
126
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
127 run(cmd)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
128
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
129 if not os.path.isfile(temp_file):
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
130 sys.exit("ERROR - No output file from Effective T3")
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
131
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
132 out_handle = open(tabular_file, "w")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
133 out_handle.write("#ID\tDescription\tScore\tEffective\n")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
134 data_handle = open(temp_file)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
135 count, positive, errors = clean_tabular(data_handle, out_handle)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
136 data_handle.close()
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
137 out_handle.close()
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
138
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
139 os.remove(temp_file)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
140
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
141 if errors:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
142 print("%i sequences, %i positive, %i errors"
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
143 % (count, positive, errors))
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
144 else:
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
145 print("%i/%i sequences positive" % (positive, count))
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
146
8
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
147 if count and count == errors:
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
148 # Galaxy will still allow them to see the output file
60a9b3f760cc v0.0.17 Used cached URL, python style updates
peterjc
parents: 7
diff changeset
149 sys.exit("All your sequences gave an error code")