Mercurial > repos > peterjc > effectivet3

annotate tools/effectiveT3/effectiveT3.py @ 5:1ea715da1879 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Uploaded v0.0.13, embed citation, relax test for floating point differences
author peterjc
date Tue, 25 Nov 2014 08:28:24 -0500
parents b0b927299aee
children 0f6eb4a75000
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
1 #!/usr/bin/env python
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
2 """Wrapper for EffectiveT3 v1.0.1 for use in Galaxy.
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
4 This script takes exactly five command line arguments:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
5 * model name (e.g. TTSS_STD-1.0.1.jar)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
6 * threshold (selective or sensitive)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
7 * an input protein FASTA filename
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
8 * output tabular filename
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
9
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
10 It then calls the standalone Effective T3 v1.0.1 program (not the
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
11 webservice), and reformats the semi-colon separated output into
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
12 tab separated output for use in Galaxy.
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
13 """
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
14 import sys
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
15 import os
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
16 import subprocess
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
17
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
18 # The Galaxy auto-install via tool_dependencies.xml will set this environment variable
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
19 effectiveT3_dir = os.environ.get("EFFECTIVET3", "/opt/EffectiveT3/")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
20 effectiveT3_jar = os.path.join(effectiveT3_dir, "TTSS_GUI-1.0.1.jar")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
21
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
22 if "-v" in sys.argv or "--version" in sys.argv:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
23 # TODO - Get version of the JAR file dynamically?
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
24 print("Wrapper v0.0.13, TTSS_GUI-1.0.1.jar")
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
25 sys.exit(0)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
26
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
27 def stop_err(msg, error_level=1):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
28 """Print error message to stdout and quit with given error level."""
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
29 sys.stderr.write("%s\n" % msg)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
30 sys.exit(error_level)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
31
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
32 if len(sys.argv) != 5:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
33 stop_err("Require four arguments: model, threshold, input protein FASTA file & output tabular file")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
34
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
35 model, threshold, fasta_file, tabular_file = sys.argv[1:]
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
36
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
37 if not os.path.isfile(fasta_file):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
38 stop_err("Input FASTA file not found: %s" % fasta_file)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
39
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
40 if threshold not in ["selective", "sensitive"] \
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
41 and not threshold.startswith("cutoff="):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
42 stop_err("Threshold should be selective, sensitive, or cutoff=..., not %r" % threshold)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
43
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
44 def clean_tabular(raw_handle, out_handle):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
45 """Clean up Effective T3 output to make it tabular."""
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
46 count = 0
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
47 positive = 0
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
48 errors = 0
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
49 for line in raw_handle:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
50 if not line or line.startswith("#") \
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
51 or line.startswith("Id; Description; Score;"):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
52 continue
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
53 assert line.count(";") >= 3, repr(line)
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
54 # Normally there will just be three semi-colons, however the
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
55 # original FASTA file's ID or description might have had
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
56 # semi-colons in it as well, hence the following hackery:
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
57 try:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
58 id_descr, score, effective = line.rstrip("\r\n").rsplit(";",2)
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
59 # Cope when there was no FASTA description
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
60 if "; " not in id_descr and id_descr.endswith(";"):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
61 id = id_descr[:-1]
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
62 descr = ""
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
63 else:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
64 id, descr = id_descr.split("; ",1)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
65 except ValueError:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
66 stop_err("Problem parsing line:\n%s\n" % line)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
67 parts = [s.strip() for s in [id, descr, score, effective]]
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
68 out_handle.write("\t".join(parts) + "\n")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
69 count += 1
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
70 if float(score) < 0:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
71 errors += 1
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
72 if effective.lower() == "true":
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
73 positive += 1
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
74 return count, positive, errors
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
75
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
76 def run(cmd):
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
77 # Avoid using shell=True when we call subprocess to ensure if the Python
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
78 # script is killed, so too is the child process.
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
79 try:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
80 child = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
81 except Exception, err:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
82 stop_err("Error invoking command:\n%s\n\n%s\n" % (" ".join(cmd), err))
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
83 # Use .communicate as can get deadlocks with .wait(),
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
84 stdout, stderr = child.communicate()
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
85 return_code = child.returncode
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
86 if return_code:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
87 cmd_str= " ".join(cmd) # doesn't quote spaces etc
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
88 if stderr and stdout:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
89 stop_err("Return code %i from command:\n%s\n\n%s\n\n%s" % (return_code, cmd_str, stdout, stderr))
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
90 else:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
91 stop_err("Return code %i from command:\n%s\n%s" % (return_code, cmd_str, stderr))
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
92
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
93 if not os.path.isdir(effectiveT3_dir):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
94 stop_err("Effective T3 folder not found: %r" % effectiveT3_dir)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
95
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
96 if not os.path.isfile(effectiveT3_jar):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
97 stop_err("Effective T3 JAR file not found: %r" % effectiveT3_jar)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
98
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
99 if not os.path.isdir(os.path.join(effectiveT3_dir, "module")):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
100 stop_err("Effective T3 module folder not found: %r" % os.path.join(effectiveT3_dir, "module"))
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
101
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
102 effectiveT3_model = os.path.join(effectiveT3_dir, "module", model)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
103 if not os.path.isfile(effectiveT3_model):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
104 sys.stderr.write("Contents of %r is %s\n"
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
105 % (os.path.join(effectiveT3_dir, "module"),
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
106 ", ".join(repr(p) for p in os.listdir(os.path.join(effectiveT3_dir, "module")))))
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
107 sys.stderr.write("Main JAR was found: %r\n" % effectiveT3_jar)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
108 stop_err("Effective T3 model JAR file not found: %r" % effectiveT3_model)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
109
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
110 # We will have write access whereever the output should be,
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
111 temp_file = os.path.abspath(tabular_file + ".tmp")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
112
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
113 # Use absolute paths since will change current directory...
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
114 tabular_file = os.path.abspath(tabular_file)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
115 fasta_file = os.path.abspath(fasta_file)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
116
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
117 cmd = ["java", "-jar", effectiveT3_jar,
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
118 "-f", fasta_file,
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
119 "-m", model,
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
120 "-t", threshold,
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
121 "-o", temp_file,
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
122 "-q"]
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
123
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
124 try:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
125 # Must run from directory above the module subfolder:
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
126 os.chdir(effectiveT3_dir)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
127 except:
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
128 stop_err("Could not change to Effective T3 folder: %s" % effectiveT3_dir)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
129
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
130 run(cmd)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
131
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
132 if not os.path.isfile(temp_file):
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
133 stop_err("ERROR - No output file from Effective T3")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
134
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
135 out_handle = open(tabular_file, "w")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
136 out_handle.write("#ID\tDescription\tScore\tEffective\n")
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
137 data_handle = open(temp_file)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
138 count, positive, errors = clean_tabular(data_handle, out_handle)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
139 data_handle.close()
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
140 out_handle.close()
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
141
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
142 os.remove(temp_file)
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
143
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
144 if errors:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
145 print("%i sequences, %i positive, %i errors"
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
146 % (count, positive, errors))
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
147 else:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
148 print("%i/%i sequences positive" % (positive, count))
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
149
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
150 if count and count==errors:
5
1ea715da1879 Uploaded v0.0.13, embed citation, relax test for floating point differences
peterjc
parents: 3
diff changeset
151 # Galaxy will still allow them to see the output file
3
b0b927299aee Uploaded v0.0.11 with automatic dependency installation.
peterjc
parents:
diff changeset
152 stop_err("All your sequences gave an error code")