Mercurial > repos > fubar > tool_factory_2

comparison README.md @ 26:db35d39e1de9 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Passes planemo test Uses galaxyxml to generate new tool. More outputs will be added...
author fubar
date Thu, 30 Jul 2020 06:48:45 -0400
parents 9fe74bd23af2
children
comparison
equal deleted inserted replaced
25:9fe74bd23af2 26:db35d39e1de9
32 a Foo section taking text and results from text (foo_whatever.log) and 32 a Foo section taking text and results from text (foo_whatever.log) and
33 artifacts (eg foo_MDS_plot.pdf) file names. All artifacts are linked for download. 33 artifacts (eg foo_MDS_plot.pdf) file names. All artifacts are linked for download.
34 A copy of the actual script is provided for provenance - be warned, it exposes 34 A copy of the actual script is provided for provenance - be warned, it exposes
35 real file paths. 35 real file paths.
36 36
37 37 **WARNING before you start**
38 tldr; 38
39 39 Install this tool on a private Galaxy ONLY
40 ``` 40 Please NEVER on a public or production instance
41 41 Please cite the resource at
42 # WARNING before you start
43 # Install this tool on a private Galaxy ONLY
44 # Please NEVER on a public or production instance
45 # updated august 2014 by John Chilton adding citation support
46 #
47 # updated august 8 2014 to fix bugs reported by Marius van den Beek
48 # please cite the resource at
49 http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref 42 http://bioinformatics.oxfordjournals.org/cgi/reprint/bts573?ijkey=lczQh1sWrMwdYWJ&keytype=ref
50 # if you use this tool in your published work. 43 if you use this tool in your published work.
44
51 45
52 *Short Story* 46 *Short Story*
53 47
54 This is an unusual Galaxy tool capable of generating new Galaxy tools. 48 This is an unusual Galaxy tool capable of generating new Galaxy tools.
55 It works by exposing *unrestricted* and therefore extremely dangerous scripting 49 It works by exposing *unrestricted* and therefore extremely dangerous scripting
76 70
77 If your script writes to the current directory path, arbitrary mix of (eg) 71 If your script writes to the current directory path, arbitrary mix of (eg)
78 pdfs, tabular analysis results and run logs,the tool factory can optionally 72 pdfs, tabular analysis results and run logs,the tool factory can optionally
79 auto-generate a linked Html page with separate sections showing a thumbnail 73 auto-generate a linked Html page with separate sections showing a thumbnail
80 grid for all pdfs and the log text, grouping all artifacts sharing a file 74 grid for all pdfs and the log text, grouping all artifacts sharing a file
81 name and log name prefix:: 75 name and log name prefix.if "foo.log" is emitted then *all* other outputs matching foo_* will
82 76 all be grouped together - eg
83 eg: if "foo.log" is emitted then *all* other outputs matching foo_* will 77 - foo_baz.pdf
84 all be grouped together - eg 78 - foo_bar.pdf and
85 foo_baz.pdf 79 - foo_zot.xls
86 foo_bar.pdf and 80
87 foo_zot.xls 81 would all be displayed and linked in the same section with foo.log's contents to form the "Foo" section of the Html page.
88 would all be displayed and linked in the same section with foo.log's contents 82 Sections appear in alphabetic order and there are no limits on the number of files or sections.
89 - to form the "Foo" section of the Html page. Sections appear in alphabetic
90 order and there are no limits on the number of files or sections.
91 83
92 *Automated generation of new Galaxy tools for installation into any Galaxy* 84 *Automated generation of new Galaxy tools for installation into any Galaxy*
93 85
94 Once a script is working correctly, this tool optionally generates a 86 Once a script is working correctly, this tool optionally generates a
95 new Galaxy tool, effectively freezing the supplied script into a new, 87 new Galaxy tool, effectively freezing the supplied script into a new,
96 ordinary Galaxy tool that runs it over one or more input files selected by 88 ordinary Galaxy tool that runs it over one or more input files selected by
97 the user. Generated tools are installed via a tool shed by an administrator 89 the user. Generated tools are installed via a tool shed by an administrator
98 and work exactly like all other Galaxy tools for your users. 90 and work exactly like all other Galaxy tools for your users.
99 91
100 If you use the Html output option, please ensure that sanitize_all_html is 92 If you use the Html output option, please ensure that sanitize_all_html is
101 set to False and uncommented in universe_wsgi.ini - it should show:: 93 set to False and uncommented in universe_wsgi.ini - it should show
102 94
103 # By default, all tool output served as 'text/html' will be sanitized 95 By default, all tool output served as 'text/html' will be sanitized
104 sanitize_all_html = False 96 Change ```sanitize_all_html = False```
105 97
106 This opens potential security risks and may not be acceptable for public 98 This opens potential security risks and may not be acceptable for public
107 sites where the lack of stylesheets may make Html pages damage onlookers' 99 sites where the lack of stylesheets may make Html pages damage onlookers'
108 eyeballs but should still be correct. 100 eyeballs but should still be correct.
109
110 101
111 *More Detail* 102 *More Detail*
112 103
113 To use the ToolFactory, you should have prepared a script to paste into a 104 To use the ToolFactory, you should have prepared a script to paste into a
114 text box, and a small test input example ready to select from your history 105 text box, and a small test input example ready to select from your history
150 This is a Galaxy tool. You can install it most conveniently using the 141 This is a Galaxy tool. You can install it most conveniently using the
151 administrative "Search and browse tool sheds" link. Find the Galaxy Main 142 administrative "Search and browse tool sheds" link. Find the Galaxy Main
152 toolshed at https://toolshed.g2.bx.psu.edu/ and search for the toolfactory 143 toolshed at https://toolshed.g2.bx.psu.edu/ and search for the toolfactory
153 repository. Open it and review the code and select the option to install it. 144 repository. Open it and review the code and select the option to install it.
154 145
155 ( 146
156 If you can't get the tool that way, the xml and py files here need to be 147 If you can't get the tool that way, the xml and py files here need to be
157 copied into a new tools 148 copied into a new tools subdirectory such as tools/toolfactory
158 subdirectory such as tools/toolfactory Your tool_conf.xml needs a new entry 149 Your tool_conf.xml needs a new entry pointing to the xml \file - something like
159 pointing to the xml 150 ```
160 file - something like::
161
162 <section name="Tool building tools" id="toolbuilders"> 151 <section name="Tool building tools" id="toolbuilders">
163 <tool file="toolfactory/rgToolFactory.xml"/> 152 <tool file="toolfactory/rgToolFactory.xml"/>
164 </section> 153 </section>
165 154 ```
166 If not already there (I just added it to datatypes_conf.xml.sample), 155 If not already there (I just added it to datatypes_conf.xml.sample),
167 please add: 156 please add:
157
158 ```
168 <datatype extension="toolshed.gz" type="galaxy.datatypes.binary:Binary" 159 <datatype extension="toolshed.gz" type="galaxy.datatypes.binary:Binary"
169 mimetype="multipart/x-gzip" subclass="True" /> 160 mimetype="multipart/x-gzip" subclass="True" />
161 ```
170 to your local data_types_conf.xml. 162 to your local data_types_conf.xml.
171 ) 163
172 164
173 Of course, R, python, perl etc are needed on your path if you want to test 165 Of course, R, python, perl etc are needed on your path if you want to test
174 scripts using those interpreters. Adding new ones to this tool code should 166 scripts using those interpreters. Adding new ones to this tool code should
175 be easy enough. Please make suggestions as bitbucket issues and code. The 167 be easy enough. Please make suggestions as bitbucket issues and code. The
176 HTML file code automatically shrinks R's bloated pdfs, and depends on 168 HTML file code automatically shrinks R's bloated pdfs, and depends on
273 Rscript will need to use commandArgs(TRUE) - see the example below - additional arguments will 265 Rscript will need to use commandArgs(TRUE) - see the example below - additional arguments will
274 appear as themselves - eg foo="bar" will mean that foo is defined as "bar" for the script. 266 appear as themselves - eg foo="bar" will mean that foo is defined as "bar" for the script.
275 267
276 Bash and sh will see any additional parameters on their command lines and the 3 named parameters 268 Bash and sh will see any additional parameters on their command lines and the 3 named parameters
277 in their environment magically - well, using env on the CL 269 in their environment magically - well, using env on the CL
278 270 ```
279 ***python***:: 271 ***python***::
280 272
281 # argparse for 3 possible comma separated lists 273 # argparse for 3 possible comma separated lists
282 # additional parameters need to be parsed ! 274 # additional parameters need to be parsed !
283 # then echo parameters to the output file 275 # then echo parameters to the output file
343 (my $INPATHS,my $INNAMES,my $OUTPATH ) = @ARGV; 335 (my $INPATHS,my $INNAMES,my $OUTPATH ) = @ARGV;
344 open(my $fh, '>', $OUTPATH) or die "Could not open file '$OUTPATH' $!"; 336 open(my $fh, '>', $OUTPATH) or die "Could not open file '$OUTPATH' $!";
345 print $fh "INPATHS=$INPATHS\n INNAMES=$INNAMES\n OUTPATH=$OUTPATH\n"; 337 print $fh "INPATHS=$INPATHS\n INNAMES=$INNAMES\n OUTPATH=$OUTPATH\n";
346 close $fh; 338 close $fh;
347 339
348 340 ```
349 341
350 Galaxy as an IDE for developing API scripts 342 Galaxy as an IDE for developing API scripts
351 If you need to develop Galaxy API scripts and you like to live dangerously, 343 If you need to develop Galaxy API scripts and you like to live dangerously,
352 please read on. 344 please read on.
353 345
381 373
382 Give the below a whirl on a private clone (install the tool factory from 374 Give the below a whirl on a private clone (install the tool factory from
383 the main toolshed) and try adding complexity with few rerun/edit/rerun cycles. 375 the main toolshed) and try adding complexity with few rerun/edit/rerun cycles.
384 376
385 Eg tool factory api script 377 Eg tool factory api script
378 ```
386 import sys 379 import sys
387 from blend.galaxy import GalaxyInstance 380 from blend.galaxy import GalaxyInstance
388 ourGal = 'http://x.x.x.x:xxxx' 381 ourGal = 'http://x.x.x.x:xxxx'
389 ourKey = 'xxx' 382 ourKey = 'xxx'
390 gi = GalaxyInstance(ourGal, key=ourKey) 383 gi = GalaxyInstance(ourGal, key=ourKey)
397 res.append('%s:\n' % lib['name']) 390 res.append('%s:\n' % lib['name'])
398 res.append(str(gi.libraries.show_library(lib['id'],contents=True))) 391 res.append(str(gi.libraries.show_library(lib['id'],contents=True)))
399 outf=open(sys.argv[2],'w') 392 outf=open(sys.argv[2],'w')
400 outf.write('\n'.join(res)) 393 outf.write('\n'.join(res))
401 outf.close() 394 outf.close()
395 ```
402 396
403 **Attribution** 397 **Attribution**
404 Creating re-usable tools from scripts: The Galaxy Tool Factory 398 Creating re-usable tools from scripts: The Galaxy Tool Factory
405 Ross Lazarus; Antony Kaspi; Mark Ziemann; The Galaxy Team 399 Ross Lazarus; Antony Kaspi; Mark Ziemann; The Galaxy Team
406 Bioinformatics 2012; doi: 10.1093/bioinformatics/bts573 400 Bioinformatics 2012; doi: 10.1093/bioinformatics/bts573
413 407
414 All rights reserved. 408 All rights reserved.
415 409
416 Licensed under the LGPL 410 Licensed under the LGPL
417 411
418 **Obligatory screenshot** 412 **screenshot**
419 413
420 http://bitbucket.org/fubar/galaxytoolmaker/src/fda8032fe989/images/dynamicScriptTool.png 414 ![example run](/images/dynamicScriptTool.png)
421 415
422 416
423 ``` 417 ```
424 418