Mercurial > repos > jjohnson > qiime

annotate qiime_wrapper.py @ 0:e5c3175506b7 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Initial tool configs for qiime, most need work.
author Jim Johnson <jj@umn.edu>
date Sun, 17 Jul 2011 10:30:11 -0500
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
1 #!/usr/bin/env python
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
2 import logging, os, string, sys, tempfile, glob, shutil, types, urllib, optparse, re
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
3 import shlex, subprocess
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
4
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
5 """
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
6 sys.argv
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
7 this --galaxy_datasets= --quime_script
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
8
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
9 alpha_rarefaction
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
10 output html
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
11 wf_arare/alpha_rarefaction_plots/rarefaction_plots.html
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
12 wf_arare/alpha_rarefaction_plots/html_plots/
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
13 wf_arare/alpha_div
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
14 wf_arare/alpha_div/alpha_rarefaction_101_0.txt
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
15
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
16 --galaxy_summary_html=$output_html
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
17 --galaxy_summary_template=$output_template
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
18 --galaxy_summary_links='label:link,label:link'
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
19 --galaxy_outputdir=$output_html.extra_files_path
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
20
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
21
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
22 """
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
23
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
24 def stop_err( msg ):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
25 sys.stderr.write( "%s\n" % msg )
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
26 sys.exit()
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
27
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
28 def __main__():
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
29 debug = False
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
30 tmp_dir = None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
31 inputdir = None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
32 outputdir = None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
33 dataset_patterns = None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
34 datasetid = None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
35 new_dataset_patterns = None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
36 new_files_path = None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
37 summary_html=None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
38 summary_template=None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
39 summary_links=None
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
40 ## check if there are files to generate
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
41 cmd_args = []
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
42 for arg in sys.argv[1:]:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
43 if arg.startswith('--galaxy_'):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
44 (opt,val) = arg.split('=') if arg.find('=') > 0 else (arg,None)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
45 if opt == '--galaxy_tmpdir':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
46 try:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
47 if not os.path.exists(val):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
48 os.makedirs(val)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
49 tmp_dir = val
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
50 except Exception, ex:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
51 stop_err(ex)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
52 if opt == '--galaxy_outputdir':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
53 try:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
54 if not os.path.exists(val):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
55 os.makedirs(val)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
56 outputdir = val
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
57 except Exception, ex:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
58 stop_err(ex)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
59 if opt == '--galaxy_datasets':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
60 dataset_patterns = val.split(',')
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
61 if opt == '--galaxy_datasetid':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
62 datasetid = val
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
63 if opt == '--galaxy_new_datasets':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
64 new_dataset_patterns = val.split(',')
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
65 if opt == '--galaxy_new_files_path':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
66 new_dataset_patterns = val
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
67 if opt == '--galaxy_summary_html':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
68 summary_html=val
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
69 if opt == '--galaxy_summary_template':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
70 summary_template=val
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
71 if opt == '--galaxy_summary_links':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
72 summary_links=val
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
73 if opt == '--galaxy_debug':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
74 debug = True
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
75 else:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
76 cmd_args.append(arg)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
77 if debug: print >> sys.stdout, '\n : '.join(cmd_args)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
78 try:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
79 cmdline = ' '.join(cmd_args)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
80 if debug: print >> sys.stdout, cmdline
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
81 if tmp_dir == None or not os.path.isdir(tmp_dir):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
82 tmp_dir = tempfile.mkdtemp()
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
83 if outputdir == None or not os.path.isdir(outputdir):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
84 outputdir = tmp_dir
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
85 tmp_stderr_name = tempfile.NamedTemporaryFile( dir=tmp_dir,suffix='.err' ).name
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
86 tmp_stderr = open( tmp_stderr_name, 'wb' )
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
87 tmp_stdout_name = tempfile.NamedTemporaryFile( dir=tmp_dir,suffix='.out' ).name
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
88 tmp_stdout = open( tmp_stdout_name, 'wb' )
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
89 proc = subprocess.Popen( args=cmdline, shell=True, cwd=tmp_dir, stderr=tmp_stderr.fileno(), stdout=tmp_stdout.fileno() )
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
90 returncode = proc.wait()
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
91 tmp_stderr.close()
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
92 # get stderr, allowing for case where it's very large
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
93 tmp_stderr = open( tmp_stderr_name, 'rb' )
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
94 stderr = ''
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
95 buffsize = 1048576
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
96 try:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
97 while True:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
98 stderr += tmp_stderr.read( buffsize )
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
99 if not stderr or len( stderr ) % buffsize != 0:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
100 break
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
101 if debug: print >> sys.stderr, stderr
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
102 except OverflowError:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
103 pass
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
104 tmp_stderr.close()
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
105 if returncode != 0:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
106 if debug: print >> sys.stderr, "returncode = %d" % returncode
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
107 raise Exception, stderr
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
108 # collect results
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
109 if dataset_patterns != None:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
110 for root, dirs, files in os.walk(outputdir):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
111 for fname in files:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
112 fpath = os.path.join(root,fname)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
113 if dataset_patterns != None:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
114 for output in dataset_patterns:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
115 (pattern,path) = output.split(':')
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
116 if debug: print >> sys.stdout, '%s -> %s' % (pattern,path)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
117 if path == None or path == 'None':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
118 continue
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
119 if debug: print >> sys.stdout, 'outdir %s match: %s' % (fname,re.match(pattern,fname))
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
120 if re.match(pattern,fname):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
121 found = True
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
122 # flist.remove(fname)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
123 try:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
124 shutil.copy2(fpath, path)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
125 except Exception, ex:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
126 stop_err('%s' % ex)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
127 # move result to outdir
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
128 # Need to flatten the dir hierachy in order for galaxy to serve the href links
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
129 if summary_html != None:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
130 """
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
131 for root, dirs, files in os.walk(outputdir):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
132 if root != outputdir:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
133 for fname in files:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
134 fpath = os.path.join(root,fname)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
135 """
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
136 ## move everything up one level
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
137 dlist = os.listdir(outputdir)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
138 for dname in dlist:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
139 dpath = os.path.join(outputdir,dname)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
140 if os.path.isdir(dpath):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
141 flist = os.listdir(dpath)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
142 for fname in flist:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
143 fpath = os.path.join(dpath,fname)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
144 shutil.move(fpath,outputdir)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
145 if summary_template != None:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
146 shutil.copy(summary_template,summary_html)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
147 """
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
148 flist = os.listdir(outputdir)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
149 if debug: print >> sys.stdout, 'outputdir: %s' % outputdir
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
150 if debug: print >> sys.stdout, 'files: %s' % ','.join(flist)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
151 if dataset_patterns != None:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
152 for output in dataset_patterns:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
153 (pattern,path) = output.split(':')
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
154 if debug: print >> sys.stdout, '%s -> %s' % (pattern,path)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
155 if path == None or path == 'None':
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
156 continue
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
157 for fname in flist:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
158 if debug: print >> sys.stdout, 'outdir %s match: %s' % (fname,re.match(pattern,fname))
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
159 if re.match(pattern,fname):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
160 found = True
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
161 flist.remove(fname)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
162 fpath = os.path.join(outputdir,fname)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
163 try:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
164 shutil.copy2(fpath, path)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
165 except Exception, ex:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
166 stop_err('%s' % ex)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
167 """
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
168 # Handle the dynamically generated galaxy datasets
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
169 # http://bitbucket.org/galaxy/galaxy-central/wiki/ToolsMultipleOutput
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
170 # --new_datasets= specifies files to copy to the new_file_path
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
171 # The list items are separated by commas
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
172 # Each item conatins: a regex pattern for matching filenames and a galaxy datatype (separated by :)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
173 # The regex match.groups()[0] is used as the id name of the dataset, and must result in unique name for each output
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
174 if new_dataset_patterns != None and new_files_path != None and datasetid != None:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
175 for output in new_dataset_patterns(','):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
176 (pattern,ext) = output.split(':');
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
177 for fname in flist:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
178 m = re.match(pattern,fname)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
179 if m:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
180 fpath = os.path.join(outputdir,fname)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
181 if len(m.groups()) > 0:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
182 root = m.groups()[0]
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
183 else:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
184 # remove the ext from the name if it exists, galaxy will add back later
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
185 # remove underscores since galaxy uses that as a field separator for dynamic datasets
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
186 root = re.sub('\.?'+ext+'$','',fname).replace('_','').replace('.','')
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
187 # filename pattern required by galaxy
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
188 fn = "%s_%s_%s_%s_%s" % ( 'primary', datasetid, root, 'visible', ext )
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
189 if debug: print >> sys.stdout, '> %s' % fpath
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
190 if debug: print >> sys.stdout, '< %s' % os.path.join(new_files_path,fn)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
191 try:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
192 os.link(fpath, os.path.join(new_files_path,fn))
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
193 except:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
194 shutil.copy2(fpath, os.path.join(new_files_path,fn))
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
195
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
196 except Exception, e:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
197 msg = str(e) + stderr
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
198 stop_err( 'Error running ' + msg)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
199 finally:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
200 # Only remove temporary directories
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
201 # Enclose in try block, so we don't report error on stale nfs handles
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
202
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
203 try:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
204 if inputdir != None and os.path.exists(inputdir):
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
205 shutil.rmtree(inputdir)
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
206 except:
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
207 pass
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
208
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
209 if __name__ == "__main__": __main__()
e5c3175506b7 Initial tool configs for qiime, most need work.
Jim Johnson <jj@umn.edu>
parents:
diff changeset
210