<?xml version="1.0"?>
<tool id="mageck_count" name="MAGeCK count" version="@VERSION@.3" >
    <description>- collect sgRNA read counts from read mapping files</description>
    <macros>
        <import>mageck_macros.xml</import>
    </macros>
    <expand macro="requirements">
        <requirement type="package" version="9.22">ghostscript</requirement>
    </expand>
    <expand macro="version" />
    <command detect_errors="exit_code"><![CDATA[

#if str($reads.format_select) == "files":
    #import re
    #set $names = []
    #set $files = []

    #for $sample in $reads.sample:

        #set name = re.sub('[^\w\-\s]', '_', str($sample.element_identifier))

        #if $sample.is_of_type('fastq.gz', 'fastqsanger.gz'):
            #set infile = $name + ".fq.gz"
        #elif $sample.is_of_type('fastq'):
            #set infile = $name + ".fq"
        #elif $sample.is_of_type('bam'):
            #set infile = $name + ".bam"
        #end if

        ln -s '${sample}' '$infile' &&

        #silent $files.append($infile)
        #silent $names.append($name)

    #end for
#end if

mageck count

#if str($reads.format_select) == "files":

    -l '$reads.sgrna_library_file'

    --fastq ${ ' '.join( ["'%s'" % $x for $x in $files] )  }

    #if $reads.sample_label:
        --sample-label '$reads.sample_label'
    #else:
        --sample-label ${ ','.join( ["'%s'" % $x for $x in $names] )  }
    #end if

#elif str($reads.format_select) == "table":
    -k '$reads.counts'
    #if '$sgrna_library_file':
        -l '$sgrna_library_file'
    #end if
#end if

-n output

#if $out.pdfreportOpt:
    $out.pdfreportOpt
    --keep-tmp
#end if

$out.unmappedOpt

#if $adv.trim5:
   --trim-5 $adv.trim5
#end if

#if $adv.norm_method:
    --norm-method $adv.norm_method
#end if

#if $adv.control_sgrna:
    --control-sgrna $adv.control_sgrna
#end if

#if $adv.sgrna_len:
    --sgrna-len $adv.sgrna_len
#end if

$adv.count_n

$adv.reverse_complement

$adv.test_run

#if $adv.gmt_file:
    --gmt-file '$adv.gmt_file'
#end if

#if $out.pdfreportOpt:
  &&
  gs -dBATCH -dNOPAUSE -q -dPDFSETTINGS=/prepress -sDEVICE=pdfwrite -sOutputFile=merged.pdf *.pdf
#end if

    ]]></command>
    <inputs>
        <conditional name="reads">
            <param name="format_select" type="select" label="Reads Files or Count Table?" help="You can choose to input either separate files of reads (one per sample) or a single count table">
                <option value="files">Separate Reads files</option>
                <option value="table">Single Count table</option>
            </param>
            <when value="files">
                <param name="sample" argument="--fastq" type="data" format="fastq,fastq.gz,bam" multiple="true" label="Sample reads" help="The input reads must be in FASTQ, FASTQ.GZ or BAM format and all files must be in the same format." />
                <param name="sgrna_library_file" type="data" argument="--list-seq" format="txt,tabular,tsv,csv" label="sgRNA library file" help="A library file must be provided with three columns containing the sgRNA ID, sequence, and gene it is targeting, see Help below for more information." />
                <param name="sample_label" argument="--sample-label" type="text" optional="true" value="" label="Specify sample labels" help="By default, the input filenames will be used as the sample labels. Optionally you can specify different sample labels to use which must be separated by comma (,). Must be equal to the number of samples provided in --fastq option.">
                    <validator type="regex" message="Please only use letters, numbers or underscores in sample labels, and separate labels by commas">^[\w,]+$</validator>
                </param>
            </when>
            <when value="table">
                <param name="counts" argument="-k"  type="data" format="tabular" optional="true" label="Counts Table" help="Alternatively, a tab-separated file of read counts can be used as input. See Help below for format" />
                <param name="sgrna_library_file" type="data" argument="--list-seq" format="txt,tabular,tsv,csv" optional="True" label="sgRNA library file" help="Optionally, a library file can be provided with three columns containing the sgRNA ID, sequence, and gene it is targeting, see Help below for more information." />
            </when>
        </conditional>

        <section name="out" title="Output Options">
            <param name="countsummaryOpt" type="boolean" truevalue="True" falsevalue="" checked="false" optional="true" label="Output Count Summary file" help="Output summary statistics of the fastq files. Default: No" />
            <param name="normcountsOpt" type="boolean" truevalue="True" falsevalue="" checked="false" optional="true" label="Output Normalized Counts file" help="Default: No" />
            <param name="pdfreportOpt" argument="--pdf-report" type="boolean" truevalue="--pdf-report" falsevalue="" checked="false" optional="true"  label="Output plots" help="Generate PDF of the plots. Default: No" />
            <param name="unmappedOpt" argument="--unmapped-to-file" type="boolean" truevalue="--unmapped-to-file" falsevalue="" checked="false" optional="true" label="Output Unmapped reads" help="Save nmapped reads to file. Default: No" />
            <param name="rfilesOpt" type="boolean" truevalue="True" falsevalue="" checked="false" optional="true" label="Output R files" help="Output the .R and .Rnw files used to generate the plots in the PDF report. The median-normalized read counts file will also be output as it is required to regenerate the plots. Default: No" />
            <param name="logOpt" type="boolean" truevalue="True" falsevalue="" checked="false" label="Output Log file" help="This file includes the logging information, it will list some basic statistics of the dataset at the end. Default: No" />
        </section>

        <section name="adv" title="Advanced Options">
            <param name="gmt_file" argument="--gmt-file" type="data" format="tabular" optional="true" value="" label="Pathway file for QC" help="The pathway file used for QC, in GMT format. By default it will use the GMT file provided by MAGeCK" />
            <param name="trim5" argument="--trim-5" type="integer" min="0" optional="true" label="5' Trim length" help="Length of trimming the 5' of the reads. Default: 0" />
            <param name="norm_method" argument="--norm-method" type="select" optional="true" label="Method for normalization" help="Methods include: None (no normalization), Median (median normalization), Total (normalization by total read counts), Control (normalization by control sgRNAs specified by the --control-sgrna option). Default: Median" >
                <option value="median" selected="True">Median</option>
                <option value="none">None</option>
                <option value="total">Total</option>
                <option value="control">Control</option>
            </param>
            <param name="control_sgrna" argument="--control-sgrna" type="data" format="tabular" optional="true" label="Control sgRNAs file" help="A file of control sgRNA IDs for normalization and for generating the null distribution of RRA" />
            <param name="sgrna_len" argument="--sgrna-len" type="integer" min="0" optional="true" label="Length of the sgRNA" help="The program will automatically determine the sgRNA length from the library file, so only use this if you turn on the --unmapped-to-file option. Default: autodetected" />
            <param name="count_n" argument="--count-n" type="boolean" truevalue="--count-n" falsevalue="" checked="false" optional="true" label="Count sgRNAs with Ns" help="By default, sgRNAs containing Ns will be discarded" />
            <param name="reverse_complement" argument="--reverse-complement" type="boolean" truevalue="--reverse-complement" falsevalue="" checked="false" optional="true" label="Reverse complement" help="Reverse complement the sequences in library for read mapping" />
            <param name="test_run" argument="--test-run" type="boolean" truevalue="--test-run" falsevalue="" checked="false" optional="true" label="Test running" help="If this option is on, MAGeCK will only process the first 1M records for each file" />
        </section>
    </inputs>

    <outputs>
        <data name="counts" format="tabular" from_work_dir="output.count.txt" label="${tool.name} on ${on_string}: sgRNA Counts" />
        <data name="countsummary" format="tabular" from_work_dir="output.countsummary.txt" label="${tool.name} on ${on_string}: sgRNA Count Summary" >
            <filter>out['countsummaryOpt'] is True</filter>
        </data>
        <data name="normcounts" format="tabular" from_work_dir="output.count_normalized.txt" label="${tool.name} on ${on_string}: Normalized counts" >
            <filter>out['normcountsOpt'] is True or out['rfilesOpt'] is True</filter>
        </data>
        <data name="unmapped" format="tabular" from_work_dir="output.unmapped.txt" label="${tool.name} on ${on_string}: Unmapped" >
            <filter>out['unmappedOpt'] is True</filter>
        </data>
        <data name="pdfreport" format="pdf" from_work_dir="merged.pdf" label="${tool.name} on ${on_string}: PDF Report" >
            <filter>out['pdfreportOpt'] is True</filter>
        </data>
        <data name="log" format="txt" from_work_dir="output.log" label="${tool.name} on ${on_string}: Log" >
            <filter>out['logOpt'] is True</filter>
        </data>
        <data name="rscript" format="txt" from_work_dir="output_countsummary.R" label="${tool.name} on ${on_string}: R file"  >
            <filter>out['rfilesOpt'] is True</filter>
        </data>
        <data name="rnwfile" format="txt" from_work_dir="output_countsummary.Rnw" label="${tool.name} on ${on_string}: Rnw file"  >
            <filter>out['rfilesOpt'] is True</filter>
        </data>
    </outputs>

    <tests>
        <!-- Ensure fastq.gz input works -->
        <test expect_num_outputs="1">
            <param name="sgrna_library_file" value="demo/demo2/library.txt" ftype="tabular" />
            <param name="format_select" value="files" />
            <param name="sample" value="test1.fastq.gz" ftype="fastq.gz"/>
            <output name="counts" file="out.count.txt"/>
        </test>
        <!-- Ensure multiple fastq.gz input works with report -->
        <test expect_num_outputs="2">
            <param name="sgrna_library_file" value="demo/demo2/library.txt" ftype="tabular" />
            <param name="format_select" value="files" />
            <param name="sample" value="test1.fastq.gz,test2.fastq.gz" ftype="fastq.gz"/>
            <param name="pdfreportOpt" value="True" />
            <output name="counts" file="out.count_multi.txt"/>
            <output name="pdfreport" file="out.countsummary_multi.pdf" compare="sim_size" />
        </test>
         <!-- Ensure fastq input works -->
        <test expect_num_outputs="1">
            <param name="sgrna_library_file" value="demo/demo2/library.txt" ftype="tabular" />
            <param name="format_select" value="files" />
            <param name="sample" value="demo/demo2/test1.fastq" ftype="fastq"/>
            <param name="sample_label" value="test1_fastq_gz" />
            <output name="counts" file="out.count.txt"/>
        </test>
        <!-- Ensure BAM input works -->
        <test expect_num_outputs="1">
            <param name="sgrna_library_file" value="demo/demo2/library.txt" ftype="tabular" />
            <param name="format_select" value="files" />
            <param name="sample" value="test1.bam" ftype="bam"/>
            <output name="counts" file="out.count.bam.txt"/>
        </test>
        <!-- Ensure optional outputs work -->
        <test expect_num_outputs="8">
            <param name="sgrna_library_file" value="demo/demo2/library.txt" ftype="tabular" />
            <param name="format_select" value="files" />
            <param name="sample" value="test1.fastq.gz" ftype="fastq.gz"/>
            <param name="countsummaryOpt" value="True" />
            <param name="unmappedOpt" value="True" />
            <param name="pdfreportOpt" value="True" />
            <param name="rfilesOpt" value="True" />
            <param name="logOpt" value="True" />
            <output name="counts" file="out.count.fastq.txt"/>
            <output name="countsummary" file="out.countsummary.txt" compare="contains"/>
            <output name="normcounts" file="output.count_normalized.txt"/>
            <output name="log">
                <assert_contents>
                    <has_text_matching expression="Welcome to MAGeCK"/>
                </assert_contents>
            </output>
            <output name="unmapped" file="out.count.unmapped.txt" />
            <output name="pdfreport" file="out.countsummary.pdf" compare="sim_size" />
            <output name="rscript" file="out.count.R" />
            <output name="rnwfile">
                <assert_contents>
                    <has_text_matching expression="This is a template file for Sweave"/>
                </assert_contents>
            </output>
        </test>
    </tests>

    <help><![CDATA[
.. class:: infomark

**What it does**

Model-based Analysis of Genome-wide CRISPR-Cas9 Knockout (MAGeCK_) is a computational tool to identify important genes from the recent genome-scale CRISPR-Cas9 knockout screens (or GeCKO) technology. MAGeCK can be used for prioritizing single-guide RNAs, genes and pathways in genome-scale CRISPR/Cas9 knockout screens. MAGeCK identifies both positively and negatively selected genes simultaneously and reports robust results across different experimental conditions. MAGeCK is developed and maintained by Wei Li and Han Xu from `Prof. Xiaole Shirley Liu's lab`_ at the Department of Biostatistics and Computational Biology, Dana-Farber Cancer Institute and Harvard School of Public Health. MAGeCK has been used to identify functional lncRNAs from screens with close to `100% validation rate`_.

-----

**Inputs**

**Read files**

**MAGeCK count** accepts one or more FASTQ.GZ, FASTQ or BAM files as input.

Since version 0.5.5, MAGeCK count module supports collecting read counts from BAM files. This will allow you to use a third-party aligner to map reads to the library with mismatches, providing more usable reads for the analysis. However, it is still recommended to directly use the fastq file in the count module (which does not allow any mismatches), because:

* Some mismatches in the sgRNAs may have unwanted behaviors (have no on-target cleavages or have other off-target cleavages);
* In most cases the read counts are enough if we allow no mismatches;
* The mapping procedure is more complicated; for example, you need to know the exact length of 3' adapter sequence.

It is also possible to input a Count Table to normalize counts and get statistics.

**sgRNA library file**

When starting from FASTQ, FASTQ.GZ or BAM files, MAGeCK needs to know the sgRNA sequences and targeting genes. Such information is provided in the sgRNA library file and can be specified in the tool form above. The sgRNA library file can be provided in .tsv or .csv format. There are three columns in the library file: the sgRNA ID, the sequence, and the gene it is targeting.

Example:

    ============ ==================== ========
    **sgRNA ID** **Sequence**         **Gene**
    ------------ -------------------- --------
    s_10007      TGTTCACAGTATAGTTTGCC CCNA1
    s_10008      TTCTCCCTAATTGCTTGCTG CCNA1
    s_10027      ACATGTTGCTTCCCCTTGCA CCNC
    ============ ==================== ========

**Control sgRNA file**

The optional Control sgRNAs file is used to generate null distribution when calculating the p values. If this option is not specified, MAGeCK generates the null distribution of RRA scores by assuming all of the genes in the library are non-essential, see **More Information** below. This approach is sometimes over-conservative, and you can improve this if you know some genes are not essential. By providing the corresponding sgRNA IDs in this option, MAGeCK will have a better estimation of p values. To use this option, you need to prepare a text file specifying the IDs of control sgRNAs, one line for one sgRNA ID.

-----

**Outputs**

This tool outputs

    * an sgRNA Counts table

Optionally, under **Output Options** you can choose to output

    * a Count Summary file
    * a PDF report
    * a Normalized Counts table
    * an Unmapped reads file
    * the .R and .Rnw files used to generate the plots and PDF
    * a Log file of the analysis

**sgRNA Count file**

An example of the sgRNA count output file is shown below. This file can be used with **MAGeCK test**.

Example:

    ==============  ========    =========== ===========
    **sgRNA**       **Gene**    **Sample1** **Sample2**
    --------------  --------    ----------- -----------
    A1CF_m52595977  A1CF        213         199    
    A1CF_m52596017  A1CF        294         164    
    A1CF_m52596056  A1CF        421         378    
    A1CF_m52603842  A1CF        274         281    
    A1CF_m52603847  A1CF        0           0    
    ==============  ========    =========== ===========


**Count Summary**

MAGeCK can produce a **Count Summary** file containing statistics of the input files (the statistics of fastq files are also in the PDF report). An example count summary file is shown below.

Example:

    ==========  =====  =====   ======  ==========  =========== ========== ========= ======== ============ ======================= ========================== ============
    File        Label  Reads   Mapped  Percentage  TotalsgRNAs Zerocounts GiniIndex NegSelQC NegSelQCPval NegSelQCPvalPermutation NegSelQCPvalPermutationFDR NegSelQCGene
    ==========  =====  =====   ======  ==========  =========== ========== ========= ======== ============ ======================= ========================== ============
    InputFile1  L1     2500    1453    0.5812      2550        1276       0.5267    0        1            1                       1                          0.0
    ==========  =====  =====   ======  ==========  =========== ========== ========= ======== ============ ======================= ========================== ============

-----

**More Information**

**Overview of the MAGeCK algorithm**

Briefly, read counts from different samples are first median-normalized to adjust for the effect of library sizes and read count distributions. Then the variance of read counts is estimated by sharing information across features, and a negative binomial (NB) model is used to test whether sgRNA abundance differs significantly between treatments and controls. This approach is similar to those used for differential RNA-Seq analysis. We rank sgRNAs based on P-values calculated from the NB model, and use a modified robust ranking aggregation (RRA) algorithm named α-RRA to identify positively or negatively selected genes. More specifically, α-RRA assumes that if a gene has no effect on selection, then sgRNAs targeting this gene should be uniformly distributed across the ranked list of all the sgRNAs. α-RRA ranks genes by comparing the skew in rankings to the uniform null model, and prioritizes genes whose sgRNA rankings are consistently higher than expected. α-RRA calculates the statistical significance of the skew by permutation, and a detailed description of the algorithm is presented in the Materials and methods section of the MAGeCK paper. Finally, MAGeCK reports positively and negatively selected pathways by applying α-RRA to the rankings of genes in a pathway.

**MAGeCK FAQs**

**The 5' trim length option can only trim a fixed length of nucleotides before sgRNA, but what if the trimming length is different in different reads?**
MAGeCK enables automatically determining trimming length, even the length may be different within the same fastq files.
Alternatively, you can use **cutadapt** to trim the adaptor sequences of variable length before running MAGeCK.

**How do I get the simple statistics of my input files?**
MAGeCK produces a **Count Summary** file containing the statistics of the input files, the statistics are also in the PDF report. The statistics can also be found in the log file for **MAGeCK** count.

**How do I know the quality of my samples?**
For simple QC terms, you can just take a look at the sample statistics. Generally in a good negative selection sample:

#. the mapped reads should be over 60 percent of the total number reads
#. the number of zero-count sgRNAs should be few (<5%, and prefered <1%). One exception is in positive selection experiments, where the number of zero-count sgRNAs may be much higher, but the percentage of mapped reads should be reasonably high.

For more information on using MAGeCK, see the `MAGeCK website here`_.

.. _MAGeCK: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-014-0554-4
.. _`100% validation rate`: https://sourceforge.net/p/mageck/wiki/Home/
.. _`Prof. Xiaole Shirley Liu's lab`: http://liulab.dfci.harvard.edu/
.. _`MAGeCK website here`: https://sourceforge.net/p/mageck/wiki/QA/#using-mageck

    ]]></help>
      <expand macro="citations" />
</tool>