Mercurial > repos > iuc > stacks2_shortreads

comparison test-data/denovo_map/denovo_map.log @ 0:ad7a60726fc3 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/stacks2 commit b395fa36fa826e26085820ba3a9faacaeddcb460
author iuc
date Mon, 01 Jul 2019 11:02:39 -0400
parents
children 43e3eeb2e0ec
comparison
equal deleted inserted replaced
-1:000000000000 0:ad7a60726fc3
1 denovo_map.pl version 2.4 started at 2019-06-18 10:34:45
2 /home/berntm/miniconda3/envs/mulled-v1-2b57e7596f85ebb3b321e6c9681e8fd9250523a80d97945c46ac7743359454e7/bin/denovo_map.pl --samples demultiplexed --popmap denovo_map/popmap_cstacks.tsv -o stacks_outputs --paired
3
4 ustacks
5 ==========
6
7 Sample 1 of 2 'PopA_01'
8 ----------
9 ustacks -t fastq -f demultiplexed/PopA_01.1.fq -o stacks_outputs -i 1 --name PopA_01
10 ustacks parameters selected:
11 Input file: 'demultiplexed/PopA_01.1.fq'
12 Sample ID: 1
13 Min depth of coverage to create a stack (m): 3
14 Repeat removal algorithm: enabled
15 Max distance allowed between stacks (M): 2
16 Max distance allowed to align secondary reads: 4
17 Max number of stacks allowed per de novo locus: 3
18 Deleveraging algorithm: disabled
19 Gapped assembly: enabled
20 Minimum alignment length: 0.8
21 Model type: SNP
22 Alpha significance level for model: 0.05
23
24 Loading RAD-Tags...
25
26 Loaded 66 reads; formed:
27 4 stacks representing 63 primary reads (95.5%)
28 3 secondary stacks representing 3 secondary reads (4.5%)
29
30 Stack coverage: mean=15.75; stdev=7.46; max=27; n_reads=63(95.5%)
31 Removing repetitive stacks: cov > 39 (mean+3*stdev)...
32 Blacklisted 0 stacks.
33 Coverage after repeat removal: mean=15.75; stdev=7.46; max=27; n_reads=63(95.5%)
34
35 Assembling stacks (max. dist. M=2)...
36 Assembled 4 stacks into 3; blacklisted 0 stacks.
37 Coverage after assembling stacks: mean=21.00; stdev=4.24; max=27; n_reads=63(95.5%)
38
39 Merging secondary stacks (max. dist. N=4 from consensus)...
40 Merged 3 out of 3 secondary reads (100.0%), 1 merged with gapped alignments.
41 Coverage after merging secondary stacks: mean=22.00; stdev=4.32; max=28; n_reads=66(100.0%)
42
43 Assembling stacks, allowing for gaps (min. match length 80.0%)...
44 Assembled 3 stacks into 3 stacks.
45 Coverage after gapped assembly: mean=22.00; stdev=4.32; max=28; n_reads=66(100.0%)
46
47 Merging secondary stacks, allowing for gaps (min. match length 80.0%)...
48 Merged 0 out of 0 secondary reads (-nan%).
49 Coverage after merging gapped secondary stacks: mean=22.00; stdev=4.32; max=28; n_reads=66(100.0%)
50
51 Final coverage: mean=22.00; stdev=4.32; max=28; n_reads=66(100.0%)
52 Calling consensus sequences and haplotypes for catalog assembly...
53 Writing tags, SNPs, and alleles files...
54 Refetching read IDs...done.
55 ustacks is done.
56
57 Sample 2 of 2 'PopA_02'
58 ----------
59 ustacks -t fastq -f demultiplexed/PopA_02.1.fq -o stacks_outputs -i 2 --name PopA_02
60 ustacks parameters selected:
61 Input file: 'demultiplexed/PopA_02.1.fq'
62 Sample ID: 2
63 Min depth of coverage to create a stack (m): 3
64 Repeat removal algorithm: enabled
65 Max distance allowed between stacks (M): 2
66 Max distance allowed to align secondary reads: 4
67 Max number of stacks allowed per de novo locus: 3
68 Deleveraging algorithm: disabled
69 Gapped assembly: enabled
70 Minimum alignment length: 0.8
71 Model type: SNP
72 Alpha significance level for model: 0.05
73
74 Loading RAD-Tags...
75
76 Loaded 60 reads; formed:
77 4 stacks representing 55 primary reads (91.7%)
78 5 secondary stacks representing 5 secondary reads (8.3%)
79
80 Stack coverage: mean=13.75; stdev=9.12; max=26; n_reads=55(91.7%)
81 Removing repetitive stacks: cov > 42 (mean+3*stdev)...
82 Blacklisted 0 stacks.
83 Coverage after repeat removal: mean=13.75; stdev=9.12; max=26; n_reads=55(91.7%)
84
85 Assembling stacks (max. dist. M=2)...
86 Assembled 4 stacks into 3; blacklisted 0 stacks.
87 Coverage after assembling stacks: mean=18.33; stdev=6.55; max=26; n_reads=55(91.7%)
88
89 Merging secondary stacks (max. dist. N=4 from consensus)...
90 Merged 5 out of 5 secondary reads (100.0%), 0 merged with gapped alignments.
91 Coverage after merging secondary stacks: mean=20.00; stdev=6.53; max=28; n_reads=60(100.0%)
92
93 Assembling stacks, allowing for gaps (min. match length 80.0%)...
94 Assembled 3 stacks into 3 stacks.
95 Coverage after gapped assembly: mean=20.00; stdev=6.53; max=28; n_reads=60(100.0%)
96
97 Merging secondary stacks, allowing for gaps (min. match length 80.0%)...
98 Merged 0 out of 0 secondary reads (-nan%).
99 Coverage after merging gapped secondary stacks: mean=20.00; stdev=6.53; max=28; n_reads=60(100.0%)
100
101 Final coverage: mean=20.00; stdev=6.53; max=28; n_reads=60(100.0%)
102 Calling consensus sequences and haplotypes for catalog assembly...
103 Writing tags, SNPs, and alleles files...
104 Refetching read IDs...done.
105 ustacks is done.
106
107 Depths of Coverage for Processed Samples:
108 PopA_01: 22.00x
109 PopA_02: 20.00x
110
111 cstacks
112 ==========
113 cstacks -P stacks_outputs -M denovo_map/popmap_cstacks.tsv
114
115 cstacks parameters selected:
116 Loci matched based on sequence identity.
117 Number of mismatches allowed between stacks: 1
118 Gapped alignments: enabled
119 Constructing catalog from 2 samples.
120
121 Initializing new catalog...
122 Parsing stacks_outputs/PopA_01.tags.tsv
123 Parsing stacks_outputs/PopA_01.snps.tsv
124 Parsing stacks_outputs/PopA_01.alleles.tsv
125 3 loci were newly added to the catalog.
126
127 Processing sample stacks_outputs/PopA_02 [2 of 2]
128 Parsing stacks_outputs/PopA_02.tags.tsv
129 Parsing stacks_outputs/PopA_02.snps.tsv
130 Parsing stacks_outputs/PopA_02.alleles.tsv
131 Searching for sequence matches...
132 3 loci in the catalog, 184 kmers in the catalog hash.
133 Searching for gapped alignments...
134 Merging matches into catalog...
135 3 loci were matched to a catalog locus.
136 0 loci were matched to a catalog locus using gapped alignments.
137 0 loci were newly added to the catalog.
138 0 loci matched more than one catalog locus, linking them.
139 0 linked catalog loci were merged into 0 loci.
140
141 Writing catalog in directory 'stacks_outputs/'.
142 Final catalog contains 3 loci.
143 cstacks is done.
144
145 sstacks
146 ==========
147 sstacks -P stacks_outputs -M denovo_map/popmap_cstacks.tsv
148
149 Searching for matches by sequence identity...
150 Parsing stacks_outputs/catalog.tags.tsv
151 Parsing stacks_outputs/catalog.snps.tsv
152 Parsing stacks_outputs/catalog.alleles.tsv
153 Populating kmer dictionary for exact matches...done.
154 Populating kmer dictionary for gapped alignments...done.
155
156 Processing sample 'stacks_outputs/PopA_01' [1 of 2]
157 Parsing stacks_outputs/PopA_01.tags.tsv
158 Parsing stacks_outputs/PopA_01.snps.tsv
159 Parsing stacks_outputs/PopA_01.alleles.tsv
160 Searching for sequence matches...
161 3 sample loci compared against the catalog containing 3 loci.
162 3 matching loci, 0 contained no verified haplotypes.
163 0 loci matched more than one catalog locus and were excluded.
164 0 loci contained SNPs unaccounted for in the catalog and were excluded.
165 4 total haplotypes examined from matching loci, 4 verified.
166 Searching for gapped alignments...
167 Out of 3 query loci, 0 gapped alignments attempted.
168 0 loci matched one catalog locus; 0 total haplotypes examined, 0 verified.
169 0 loci matched no catalog locus;
170 0 loci matched more than one catalog locus and were excluded.
171 0 loci contained SNPs unaccounted for in the catalog and were excluded.
172 0 loci had no verified haplotypes.
173 0 loci had inconsistent alignments to a catalog locus and were excluded.
174 Outputing to file stacks_outputs/PopA_01.matches.tsv
175
176 Processing sample 'stacks_outputs/PopA_02' [2 of 2]
177 Parsing stacks_outputs/PopA_02.tags.tsv
178 Parsing stacks_outputs/PopA_02.snps.tsv
179 Parsing stacks_outputs/PopA_02.alleles.tsv
180 Searching for sequence matches...
181 3 sample loci compared against the catalog containing 3 loci.
182 3 matching loci, 0 contained no verified haplotypes.
183 0 loci matched more than one catalog locus and were excluded.
184 0 loci contained SNPs unaccounted for in the catalog and were excluded.
185 4 total haplotypes examined from matching loci, 4 verified.
186 Searching for gapped alignments...
187 Out of 3 query loci, 0 gapped alignments attempted.
188 0 loci matched one catalog locus; 0 total haplotypes examined, 0 verified.
189 0 loci matched no catalog locus;
190 0 loci matched more than one catalog locus and were excluded.
191 0 loci contained SNPs unaccounted for in the catalog and were excluded.
192 0 loci had no verified haplotypes.
193 0 loci had inconsistent alignments to a catalog locus and were excluded.
194 Outputing to file stacks_outputs/PopA_02.matches.tsv
195
196 sstacks is done.
197
198 tsv2bam
199 ==========
200 tsv2bam -P stacks_outputs -M denovo_map/popmap_cstacks.tsv -R demultiplexed/
201
202 Logging to 'stacks_outputs/tsv2bam.log'.
203 Configuration for this run:
204 Stacks directory: 'stacks_outputs/'
205 Population map: 'denovo_map/popmap_cstacks.tsv'
206 Num. samples: 2
207 Paired-end reads directory: 'demultiplexed/'
208
209 Paired-end reads files found, e.g. 'demultiplexed/PopA_01.2.fq'.
210 Loading the catalog...
211 Processing sample 'PopA_01'...
212 Processing sample 'PopA_02'...
213
214 Sample 'PopA_01': matched 3 sample loci to 3 catalog loci; found a paired-end read for 66 (100.0%) of the assembled forward reads; wrote 132 records.
215 Sample 'PopA_02': matched 3 sample loci to 3 catalog loci; found a paired-end read for 60 (100.0%) of the assembled forward reads; wrote 120 records.
216
217 tsv2bam is done.
218
219 gstacks
220 ==========
221 gstacks -P stacks_outputs -M denovo_map/popmap_cstacks.tsv
222
223 Logging to 'stacks_outputs/gstacks.log'.
224 Locus/sample distributions will be written to 'stacks_outputs/gstacks.log.distribs'.
225
226 Configuration for this run:
227 Input mode: denovo
228 Population map: 'denovo_map/popmap_cstacks.tsv'
229 Input files: 2, e.g. 'stacks_outputs/PopA_01.matches.bam'
230 Output to: 'stacks_outputs/'
231 Model: marukilow (var_alpha: 0.01, gt_alpha: 0.05)
232
233 Reading BAM headers...
234 Processing all loci...
235 20%...
236 50%...
237 100%
238
239 Attempted to assemble and align paired-end reads for 3 loci:
240 0 loci had no or almost no paired-end reads (0.0%);
241 0 loci had paired-end reads that couldn't be assembled into a contig (0.0%);
242 For the remaining 3 loci (100.0%), a paired-end contig was assembled;
243 Average contig size was 204.3 bp;
244 0 paired-end contigs overlapped the forward region (0.0%)
245 Mean overlap: -nanbp; mean size of overlapped loci after merging: -nan;
246 Out of 252 paired-end reads in these loci (mean 84.0 reads per locus),
247 252 were successfuly aligned (100.0%);
248 Mean insert length was -nan, stdev: nan (based on aligned reads in overlapped loci).
249
250 Genotyped 3 loci:
251 effective per-sample coverage: mean=31.0x, stdev=1.0x, min=30.0x, max=32.0x
252 mean number of sites per locus: 194.3
253 a consistent phasing was found for 5 of out 5 (100.0%) diploid loci needing phasing
254
255 gstacks is done.
256
257 populations
258 ==========
259 populations -P stacks_outputs -M denovo_map/popmap_cstacks.tsv
260
261 Logging to 'stacks_outputs/populations.log'.
262 Locus/sample distributions will be written to 'stacks_outputs/populations.log.distribs'.
263 populations parameters selected:
264 Percent samples limit per population: 0
265 Locus Population limit: 1
266 Percent samples overall: 0
267 Minor allele frequency cutoff: 0
268 Maximum observed heterozygosity cutoff: 1
269 Applying Fst correction: none.
270 Pi/Fis kernel smoothing: off
271 Fstats kernel smoothing: off
272 Bootstrap resampling: off
273
274 Parsing population map...
275 The population map contained 2 samples, 1 population(s), 1 group(s).
276 Working on 2 samples.
277 Working on 1 population(s):
278 1: PopA_01, PopA_02
279 Working on 1 group(s) of populations:
280 defaultgrp: 1
281
282 Genotyping markers will be written to 'stacks_outputs/populations.markers.tsv'
283 Raw Genotypes/Haplotypes will be written to 'stacks_outputs/populations.haplotypes.tsv'
284 Population-level summary statistics will be written to 'stacks_outputs/populations.sumstats.tsv'
285 Population-level haplotype summary statistics will be written to 'stacks_outputs/populations.hapstats.tsv'
286
287 Processing data in batches:
288 * load a batch of catalog loci and apply filters
289 * compute SNP- and haplotype-wise per-population statistics
290 * write the above statistics in the output files
291 * export the genotypes/haplotypes in specified format(s)
292 More details in 'stacks_outputs/populations.log.distribs'.
293 Now processing...
294 Batch 1
295
296 Removed 0 loci that did not pass sample/population constraints from 3 loci.
297 Kept 3 loci, composed of 613 sites; 0 of those sites were filtered, 6 variant sites remained.
298 Number of loci with PE contig: 3.00 (100.0%);
299 Mean length of loci: 194.33bp (stderr 0.33);
300 Number of loci with SE/PE overlap: 0.00 (0.0%);
301 Mean length of overlapping loci: -nanbp (stderr -0.00); mean overlap: -nanbp (stderr -0.00);
302 Mean genotyped sites per locus: 194.33bp (stderr 0.33).
303
304 Population summary statistics (more detail in populations.sumstats_summary.tsv):
305 1: 2 samples per locus; pi: 0.61111; all/variant/polymorphic sites: 583/6/6; private alleles: 0
306 Populations is done.
307 denovo_map.pl is done.
308
309 denovo_map.pl completed at 2019-06-18 10:34:45