Mercurial > repos > frogs > frogs_3_1_0

comparison test-data/references/01-prepro-flash.log @ 0:59bc96331073 draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/geraldinepascal/FROGS-wrappers/tree/v3.1.0 commit 08296fc88e3e938c482c631bd515b3b7a0499647
author frogs
date Thu, 28 Feb 2019 10:14:49 -0500
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:59bc96331073
1 ## Application
2 Software: preprocess.py (version: 3.1)
3 Command: /home/maria/workspace/git/FROGS/FROGS_master/test/../app/preprocess.py illumina --min-amplicon-size 44 --max-amplicon-size 490 --five-prim-primer GGCGVACGGGTGAGTAA --three-prim-primer GTGCCAGCNGCNGCGG --R1-size 267 --R2-size 266 --expected-amplicon-size 420 --merge-software flash --nb-cpus 4 --mismatch-rate 0.15 --keep-unmerged --input-archive /home/maria/workspace/git/FROGS/FROGS-wrapper_dev/test-data/input/temp/test_dataset.tar.gz --output-dereplicated res/01-prepro-flash.fasta --output-count res/01-prepro-flash.tsv --summary res/01-prepro-flash.html --log-file res/01-prepro-flash.log
4
5
6 ##Sample
7 R1 : res/1550052675.22_3437_01_R1.fastq
8 R2 : res/1550052675.22_3437_01_R2.fastq
9 Sample name : 01
10 nb seq before process : 30000
11 ##Commands
12 ########################################################################################################
13 # Join overlapping paired reads. (flash version : v1.2.11)
14 Command:
15 flash --threads 1 --allow-outies --min-overlap 43 --max-overlap 133 --max-mismatch-density 0.15 --compress res/1550052675.22_3437_01_R1.fastq res/1550052675.22_3437_01_R2.fastq --output-directory res --output-prefix 1550052676.19_3438_01_flash 2> res/1550052676.19_3438_01_flash.stderr
16
17 Execution:
18 start: 13 Feb 2019 11:11:16
19 end: 13 Feb 2019 11:11:22
20
21 Results:
22 nb seq paired-end assembled: 17622
23 ########################################################################################################
24 # Removes reads without the 5' primer and removes primer sequence. (cutadapt version : 1.18)
25 Command:
26 cutadapt -g GGCGVACGGGTGAGTAA --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 16 -o res/1550052676.19_3438_01_cutadapt_5prim_trim.fastq.gz res/1550052676.19_3438_01_flash.extendedFrags.fastq.gz > res/1550052676.19_3438_01_cutadapt_5prim_log.txt
27
28 Execution:
29 start: 13 Feb 2019 11:11:22
30 end: 13 Feb 2019 11:11:25
31
32 Results:
33 nb seq with 5' primer : 17622
34 ########################################################################################################
35 # Removes reads without the 3' primer and removes primer sequence. (cutadapt version : 1.18)
36 Command:
37 cutadapt -a GTGCCAGCNGCNGCGG --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 15 -o res/1550052676.19_3438_01_cutadapt.fastq.gz res/1550052676.19_3438_01_cutadapt_5prim_trim.fastq.gz > res/1550052676.19_3438_01_cutadapt_3prim_log.txt
38
39 Execution:
40 start: 13 Feb 2019 11:11:26
41 end: 13 Feb 2019 11:11:30
42
43 Results:
44 nb seq with 3' primer : 17622
45 ########################################################################################################
46 # Filters amplicons without primers by length and N count. (filterSeq.py version : 1.5.0)
47 Command:
48 filterSeq.py --force-fasta --max-N 0 --min-length 11 --max-length 457 --input-file res/1550052676.19_3438_01_cutadapt.fastq.gz --output-file res/1550052676.19_3438_01_N_and_length_filter.fasta --log-file res/1550052676.19_3438_01_N_and_length_filter_log.txt
49
50 Execution:
51 start: 13 Feb 2019 11:11:30
52 end: 13 Feb 2019 11:11:31
53
54 Results:
55 nb seq with expected length : 17622
56 nb seq without N : 17622
57 ########################################################################################################
58 # Concatenate paired reads. (combine_and_split.py version : )
59 Command:
60 combine_and_split.py --reads1 res/1550052676.19_3438_01_flash.notCombined_1.fastq.gz --reads2 res/1550052676.19_3438_01_flash.notCombined_2.fastq.gz -c XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX --combined-output res/1550052676.19_3438_01_artificial_combined.fastq.gz
61
62 Execution:
63 start: 13 Feb 2019 11:11:31
64 end: 13 Feb 2019 11:11:38
65
66 Results:
67 nb seq paired-end assembled: 12378
68 ########################################################################################################
69 # Removes reads without the 5' primer and removes primer sequence. (cutadapt version : 1.18)
70 Command:
71 cutadapt -g GGCGVACGGGTGAGTAA --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 16 -o res/1550052676.19_3438_01_art_comb_cutadapt_5prim_trim.fastq.gz res/1550052676.19_3438_01_artificial_combined.fastq.gz > res/1550052676.19_3438_01_art_comb_cutadapt_5prim_log.txt
72
73 Execution:
74 start: 13 Feb 2019 11:11:39
75 end: 13 Feb 2019 11:11:41
76
77 Results:
78 nb seq with 5' primer : 12378
79 ########################################################################################################
80 # Removes reads without the 3' primer and removes primer sequence. (cutadapt version : 1.18)
81 Command:
82 cutadapt -a GTGCCAGCNGCNGCGG --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 15 -o res/1550052676.19_3438_01_art_comb_cutadapt.fastq.gz res/1550052676.19_3438_01_art_comb_cutadapt_5prim_trim.fastq.gz > res/1550052676.19_3438_01_art_comb_cutadapt_3prim_log.txt
83
84 Execution:
85 start: 13 Feb 2019 11:11:42
86 end: 13 Feb 2019 11:11:45
87
88 Results:
89 nb seq with 3' primer : 12378
90 ########################################################################################################
91 # Filters amplicons without primers by length and N count. (filterSeq.py version : 1.5.0)
92 Command:
93 filterSeq.py --force-fasta --max-N 0 --min-length 267 --input-file res/1550052676.19_3438_01_art_comb_cutadapt.fastq.gz --output-file res/1550052676.19_3438_01_art_N_filter.fasta --log-file res/1550052676.19_3438_01_art_N_filter_log.txt
94
95 Execution:
96 start: 13 Feb 2019 11:11:45
97 end: 13 Feb 2019 11:11:46
98
99 Results:
100 nb seq with expected length : 12378
101 nb seq without N : 12378
102 ########################################################################################################
103 # Replace join tag. (combine_and_split.py version : )
104 Command:
105 combine_and_split.py --reads1 res/1550052676.19_3438_01_art_N_filter.fasta -s XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -c NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN --combined-output res/1550052676.19_3438_01_art_XtoN.fasta
106
107 Execution:
108 start: 13 Feb 2019 11:11:46
109 end: 13 Feb 2019 11:11:50
110
111 ########################################################################################################
112 # Dereplicates sample sequences. (derepSamples.py version : 1.6.1)
113 Command:
114 derepSamples.py --sequences-files res/1550052676.19_3438_01_N_and_length_filter.fasta res/1550052676.19_3438_01_art_XtoN.fasta --dereplicated-file res/1550052675.22_3437_01_filtered.fasta --count-file res/1550052676.19_3438_01_derep_count.tsv
115
116 Execution:
117 start: 13 Feb 2019 11:11:50
118 end: 13 Feb 2019 11:11:52
119
120
121 ##Sample
122 R1 : res/1550052675.22_3437_02_R1.fastq
123 R2 : res/1550052675.22_3437_02_R2.fastq
124 Sample name : 02
125 nb seq before process : 30000
126 ##Commands
127 ########################################################################################################
128 # Join overlapping paired reads. (flash version : v1.2.11)
129 Command:
130 flash --threads 1 --allow-outies --min-overlap 43 --max-overlap 133 --max-mismatch-density 0.15 --compress res/1550052675.22_3437_02_R1.fastq res/1550052675.22_3437_02_R2.fastq --output-directory res --output-prefix 1550052676.18_3439_02_flash 2> res/1550052676.18_3439_02_flash.stderr
131
132 Execution:
133 start: 13 Feb 2019 11:11:16
134 end: 13 Feb 2019 11:11:22
135
136 Results:
137 nb seq paired-end assembled: 17500
138 ########################################################################################################
139 # Removes reads without the 5' primer and removes primer sequence. (cutadapt version : 1.18)
140 Command:
141 cutadapt -g GGCGVACGGGTGAGTAA --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 16 -o res/1550052676.18_3439_02_cutadapt_5prim_trim.fastq.gz res/1550052676.18_3439_02_flash.extendedFrags.fastq.gz > res/1550052676.18_3439_02_cutadapt_5prim_log.txt
142
143 Execution:
144 start: 13 Feb 2019 11:11:22
145 end: 13 Feb 2019 11:11:25
146
147 Results:
148 nb seq with 5' primer : 17500
149 ########################################################################################################
150 # Removes reads without the 3' primer and removes primer sequence. (cutadapt version : 1.18)
151 Command:
152 cutadapt -a GTGCCAGCNGCNGCGG --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 15 -o res/1550052676.18_3439_02_cutadapt.fastq.gz res/1550052676.18_3439_02_cutadapt_5prim_trim.fastq.gz > res/1550052676.18_3439_02_cutadapt_3prim_log.txt
153
154 Execution:
155 start: 13 Feb 2019 11:11:26
156 end: 13 Feb 2019 11:11:30
157
158 Results:
159 nb seq with 3' primer : 17500
160 ########################################################################################################
161 # Filters amplicons without primers by length and N count. (filterSeq.py version : 1.5.0)
162 Command:
163 filterSeq.py --force-fasta --max-N 0 --min-length 11 --max-length 457 --input-file res/1550052676.18_3439_02_cutadapt.fastq.gz --output-file res/1550052676.18_3439_02_N_and_length_filter.fasta --log-file res/1550052676.18_3439_02_N_and_length_filter_log.txt
164
165 Execution:
166 start: 13 Feb 2019 11:11:30
167 end: 13 Feb 2019 11:11:31
168
169 Results:
170 nb seq with expected length : 17500
171 nb seq without N : 17500
172 ########################################################################################################
173 # Concatenate paired reads. (combine_and_split.py version : )
174 Command:
175 combine_and_split.py --reads1 res/1550052676.18_3439_02_flash.notCombined_1.fastq.gz --reads2 res/1550052676.18_3439_02_flash.notCombined_2.fastq.gz -c XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX --combined-output res/1550052676.18_3439_02_artificial_combined.fastq.gz
176
177 Execution:
178 start: 13 Feb 2019 11:11:31
179 end: 13 Feb 2019 11:11:38
180
181 Results:
182 nb seq paired-end assembled: 12500
183 ########################################################################################################
184 # Removes reads without the 5' primer and removes primer sequence. (cutadapt version : 1.18)
185 Command:
186 cutadapt -g GGCGVACGGGTGAGTAA --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 16 -o res/1550052676.18_3439_02_art_comb_cutadapt_5prim_trim.fastq.gz res/1550052676.18_3439_02_artificial_combined.fastq.gz > res/1550052676.18_3439_02_art_comb_cutadapt_5prim_log.txt
187
188 Execution:
189 start: 13 Feb 2019 11:11:39
190 end: 13 Feb 2019 11:11:42
191
192 Results:
193 nb seq with 5' primer : 12500
194 ########################################################################################################
195 # Removes reads without the 3' primer and removes primer sequence. (cutadapt version : 1.18)
196 Command:
197 cutadapt -a GTGCCAGCNGCNGCGG --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 15 -o res/1550052676.18_3439_02_art_comb_cutadapt.fastq.gz res/1550052676.18_3439_02_art_comb_cutadapt_5prim_trim.fastq.gz > res/1550052676.18_3439_02_art_comb_cutadapt_3prim_log.txt
198
199 Execution:
200 start: 13 Feb 2019 11:11:42
201 end: 13 Feb 2019 11:11:45
202
203 Results:
204 nb seq with 3' primer : 12500
205 ########################################################################################################
206 # Filters amplicons without primers by length and N count. (filterSeq.py version : 1.5.0)
207 Command:
208 filterSeq.py --force-fasta --max-N 0 --min-length 267 --input-file res/1550052676.18_3439_02_art_comb_cutadapt.fastq.gz --output-file res/1550052676.18_3439_02_art_N_filter.fasta --log-file res/1550052676.18_3439_02_art_N_filter_log.txt
209
210 Execution:
211 start: 13 Feb 2019 11:11:45
212 end: 13 Feb 2019 11:11:46
213
214 Results:
215 nb seq with expected length : 12500
216 nb seq without N : 12500
217 ########################################################################################################
218 # Replace join tag. (combine_and_split.py version : )
219 Command:
220 combine_and_split.py --reads1 res/1550052676.18_3439_02_art_N_filter.fasta -s XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -c NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN --combined-output res/1550052676.18_3439_02_art_XtoN.fasta
221
222 Execution:
223 start: 13 Feb 2019 11:11:46
224 end: 13 Feb 2019 11:11:50
225
226 ########################################################################################################
227 # Dereplicates sample sequences. (derepSamples.py version : 1.6.1)
228 Command:
229 derepSamples.py --sequences-files res/1550052676.18_3439_02_N_and_length_filter.fasta res/1550052676.18_3439_02_art_XtoN.fasta --dereplicated-file res/1550052675.22_3437_02_filtered.fasta --count-file res/1550052676.18_3439_02_derep_count.tsv
230
231 Execution:
232 start: 13 Feb 2019 11:11:51
233 end: 13 Feb 2019 11:11:52
234
235
236 ##Sample
237 R1 : res/1550052675.22_3437_03_R1.fastq
238 R2 : res/1550052675.22_3437_03_R2.fastq
239 Sample name : 03
240 nb seq before process : 30000
241 ##Commands
242 ########################################################################################################
243 # Join overlapping paired reads. (flash version : v1.2.11)
244 Command:
245 flash --threads 1 --allow-outies --min-overlap 43 --max-overlap 133 --max-mismatch-density 0.15 --compress res/1550052675.22_3437_03_R1.fastq res/1550052675.22_3437_03_R2.fastq --output-directory res --output-prefix 1550052676.18_3440_03_flash 2> res/1550052676.18_3440_03_flash.stderr
246
247 Execution:
248 start: 13 Feb 2019 11:11:16
249 end: 13 Feb 2019 11:11:22
250
251 Results:
252 nb seq paired-end assembled: 17464
253 ########################################################################################################
254 # Removes reads without the 5' primer and removes primer sequence. (cutadapt version : 1.18)
255 Command:
256 cutadapt -g GGCGVACGGGTGAGTAA --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 16 -o res/1550052676.18_3440_03_cutadapt_5prim_trim.fastq.gz res/1550052676.18_3440_03_flash.extendedFrags.fastq.gz > res/1550052676.18_3440_03_cutadapt_5prim_log.txt
257
258 Execution:
259 start: 13 Feb 2019 11:11:22
260 end: 13 Feb 2019 11:11:24
261
262 Results:
263 nb seq with 5' primer : 17464
264 ########################################################################################################
265 # Removes reads without the 3' primer and removes primer sequence. (cutadapt version : 1.18)
266 Command:
267 cutadapt -a GTGCCAGCNGCNGCGG --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 15 -o res/1550052676.18_3440_03_cutadapt.fastq.gz res/1550052676.18_3440_03_cutadapt_5prim_trim.fastq.gz > res/1550052676.18_3440_03_cutadapt_3prim_log.txt
268
269 Execution:
270 start: 13 Feb 2019 11:11:25
271 end: 13 Feb 2019 11:11:28
272
273 Results:
274 nb seq with 3' primer : 17464
275 ########################################################################################################
276 # Filters amplicons without primers by length and N count. (filterSeq.py version : 1.5.0)
277 Command:
278 filterSeq.py --force-fasta --max-N 0 --min-length 11 --max-length 457 --input-file res/1550052676.18_3440_03_cutadapt.fastq.gz --output-file res/1550052676.18_3440_03_N_and_length_filter.fasta --log-file res/1550052676.18_3440_03_N_and_length_filter_log.txt
279
280 Execution:
281 start: 13 Feb 2019 11:11:29
282 end: 13 Feb 2019 11:11:30
283
284 Results:
285 nb seq with expected length : 17464
286 nb seq without N : 17464
287 ########################################################################################################
288 # Concatenate paired reads. (combine_and_split.py version : )
289 Command:
290 combine_and_split.py --reads1 res/1550052676.18_3440_03_flash.notCombined_1.fastq.gz --reads2 res/1550052676.18_3440_03_flash.notCombined_2.fastq.gz -c XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX --combined-output res/1550052676.18_3440_03_artificial_combined.fastq.gz
291
292 Execution:
293 start: 13 Feb 2019 11:11:30
294 end: 13 Feb 2019 11:11:37
295
296 Results:
297 nb seq paired-end assembled: 12536
298 ########################################################################################################
299 # Removes reads without the 5' primer and removes primer sequence. (cutadapt version : 1.18)
300 Command:
301 cutadapt -g GGCGVACGGGTGAGTAA --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 16 -o res/1550052676.18_3440_03_art_comb_cutadapt_5prim_trim.fastq.gz res/1550052676.18_3440_03_artificial_combined.fastq.gz > res/1550052676.18_3440_03_art_comb_cutadapt_5prim_log.txt
302
303 Execution:
304 start: 13 Feb 2019 11:11:38
305 end: 13 Feb 2019 11:11:40
306
307 Results:
308 nb seq with 5' primer : 12536
309 ########################################################################################################
310 # Removes reads without the 3' primer and removes primer sequence. (cutadapt version : 1.18)
311 Command:
312 cutadapt -a GTGCCAGCNGCNGCGG --error-rate 0.1 --discard-untrimmed --match-read-wildcards --overlap 15 -o res/1550052676.18_3440_03_art_comb_cutadapt.fastq.gz res/1550052676.18_3440_03_art_comb_cutadapt_5prim_trim.fastq.gz > res/1550052676.18_3440_03_art_comb_cutadapt_3prim_log.txt
313
314 Execution:
315 start: 13 Feb 2019 11:11:41
316 end: 13 Feb 2019 11:11:43
317
318 Results:
319 nb seq with 3' primer : 12536
320 ########################################################################################################
321 # Filters amplicons without primers by length and N count. (filterSeq.py version : 1.5.0)
322 Command:
323 filterSeq.py --force-fasta --max-N 0 --min-length 267 --input-file res/1550052676.18_3440_03_art_comb_cutadapt.fastq.gz --output-file res/1550052676.18_3440_03_art_N_filter.fasta --log-file res/1550052676.18_3440_03_art_N_filter_log.txt
324
325 Execution:
326 start: 13 Feb 2019 11:11:44
327 end: 13 Feb 2019 11:11:45
328
329 Results:
330 nb seq with expected length : 12536
331 nb seq without N : 12536
332 ########################################################################################################
333 # Replace join tag. (combine_and_split.py version : )
334 Command:
335 combine_and_split.py --reads1 res/1550052676.18_3440_03_art_N_filter.fasta -s XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -c NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN --combined-output res/1550052676.18_3440_03_art_XtoN.fasta
336
337 Execution:
338 start: 13 Feb 2019 11:11:45
339 end: 13 Feb 2019 11:11:49
340
341 ########################################################################################################
342 # Dereplicates sample sequences. (derepSamples.py version : 1.6.1)
343 Command:
344 derepSamples.py --sequences-files res/1550052676.18_3440_03_N_and_length_filter.fasta res/1550052676.18_3440_03_art_XtoN.fasta --dereplicated-file res/1550052675.22_3437_03_filtered.fasta --count-file res/1550052676.18_3440_03_derep_count.tsv
345
346 Execution:
347 start: 13 Feb 2019 11:11:49
348 end: 13 Feb 2019 11:11:51
349
350
351
352 ##Sample
353 All
354 ##Commands
355 ########################################################################################################
356 # Dereplicates together sequences from several samples. (derepSamples.py version : 1.6.1)
357 Command:
358 derepSamples.py --nb-cpus 4 --size-separator ';size=' --samples-ref res/1550052675.22_3437_derep_inputs.tsv --dereplicated-file res/01-prepro-flash.fasta --count-file res/01-prepro-flash.tsv
359
360 Execution:
361 start: 13 Feb 2019 11:11:52
362 end: 13 Feb 2019 11:11:53
363