Mercurial > repos > pjbriggs > amplicon_analysis_pipeline

annotate updating-to-pipeline-1.3-DADA2.txt @ 4:86a12d75ebe4 draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
author pjbriggs
date Fri, 20 Dec 2019 06:59:49 -0500
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
1 Notes on updating Galaxy tool to pipeline 1.3 (DADA2)
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
2 =====================================================
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
3
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
4 Where stuff is:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
5
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
6 * projects/Amplicon_analysis-galaxy: git repo for Galaxy tool (these
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
7 developments are in the 'update-to-Amplicon_analysis_pipeline-1.3'
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
8 branch, PR #50:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
9 https://github.com/pjbriggs/Amplicon_analysis-galaxy/pull/50)
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
10
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
11 * scratchpad/test_Amplicon_analysis_pipeline_DADA2: directory for
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
12 running/testing the updates
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
13
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
14 So far:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
15
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
16 * Updated the installer for pipeline version 1.3.2
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
17
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
18 * Have been trying to run the pipeline manually outside of Galaxy
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
19 on popov & CSF3:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
20 -- DADA2 works on popov (can't remember if it works on CSF3)
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
21 -- Vsearch pipeline fails on popov and CSF3 (but errors are
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
22 different)
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
23
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
24 * Mauro is looking at fixing the errors while I carry on trying
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
25 to update the Galaxy tool
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
26
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
27 Random notes from my notebook:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
28
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
29 p44:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
30
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
31 * DADA2 uses NSLOTS environment variable from the local environment
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
32 (so can get number of cores on cluster; if NSLOTS not set then
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
33 gets number of cores on local machine)
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
34
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
35 * DADA2 has new outputs:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
36 -- DADA2_OTU_tables/Error_rate_plots/ <-- need to capture all
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
37 PDFs from this folder
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
38
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
39 pp78-79:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
40
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
41 * Galaxy wrapper could check that 'Run' column is in supplied
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
42 metatable file (if it's not present then pipeline will fail
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
43 now)
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
44
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
45 * DADA2 has its own reference database
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
46
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
47 * DADA2 produces same outputs as Vsearch (with name changed from
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
48 "Vsearch_*" to "DADA2_*", plus extras:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
49 -- Vsearch_OTUs.tre -> otus.tre
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
50 -- Vsearch_multiplexed_linearised_dereplicated_mc2_repset_nonchimeras_OTUS.fasta -> seqs.fa
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
51 -- There might be issues with the heatmap
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
52
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
53 p83: notes on progress...
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
54
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
55 p95:
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
56
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
57 * Confirms heatmap is now e.g. RESULTS/Vsearch_silva/Heatmap.pdf
86a12d75ebe4 planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit 7be61b7ed35ca3deaad68d2eae384c8cd365bcb8
pjbriggs
parents:
diff changeset
58 (instead of HTML output)