# RepeatExplorer2 with TAREAN (Tandem Repeat Analyzer) #
-------------------------------------------------------------------------------
New version of RepeatExplorer with TAndem REpeat ANalyzer 

## Authors
Petr Novak, Jiri Macas, Pavel Neumann
Biology Centre CAS, Czech Republic

## Change log

[link](CHANGELOG.md)


  
## Instalation ##
To use RepeatExplorer without installation, We recommend to use our freely
available galaxy server at
[https://repeatexplorer-elixir.cerit-sc.cz](https://repeatexplorer-elixir.cerit-sc.cz).
This server is provided in frame of ELIXIR-CZ project. Additionally, the galaxy
server includs also additional tools useful data preprocessing, quality contraol
and genome annotation.

For command line version from standalone installation, follow the instruction below:


To download source using git command:

	git clone https://bitbucket.org/petrnovak/repex_tarean.git
	cd repex_tarean
    
We recommend to install dependencies using conda (conda can be installed using [miniconda](https://docs.conda.io/en/latest/miniconda.html)). The required environment can be prepared using command:

    conda env create -f environment.yml

activate prepared environment using:
   
    conda activate repeatexplorer

In the `repex_tarean` direcory compile source and prepare databases using:

	make

Support for 32-bit executables is required. If you are using Ubuntu distribution you can add 32-bit support by running:

    sudo dpkg --add-architecture i386
    sudo apt-get update
    sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
   

to verify installation you can run clustering on example data:

    ./seqclust -p -v tmp/clustering_output test_data/LAS_paired_10k.fas
    
   
## Protein databases

Repeatexplorer2 utilize REXdb database of protein domains for repeat annotation and classification. Structure of database is described on [http://repeatexplorer.org/](http://repeatexplorer.org/). Current version of database for repeatexplorer is fetched from bitbucket repository [https://bitbucket.org/petrnovak/re_databases]https://bitbucket.org/petrnovak/re_databases() during compilation using make command


## RepeatExplorer command line options

    usage: seqclust [-h] [-p] [-A] [-t] [-l LOGFILE] [-m {float range 0.0..100.0}]
                    [-M {0,float range 0.1..1}] [-o {float range 30.0..80.0}]
                    [-c CPU] [-s SAMPLE] [-P PREFIX_LENGTH] [-v OUTPUT_DIR]
                    [-r MAX_MEMORY] [-d DATABASE DATABASE] [-C] [-k]
                    [-a {2,3,4,5}]
                    [-tax {VIRIDIPLANTAE3.0,VIRIDIPLANTAE2.2,METAZOA2.0,METAZOA3.0}]
                    [-opt {ILLUMINA,ILLUMINA_DUST_OFF,ILLUMINA_SHORT,OXFORD_NANOPORE}]
                    [-D {BLASTX_W2,BLASTX_W3,DIAMOND}]
                    sequences
    
    RepeatExplorer:
        Repetitive sequence discovery and clasification from NGS data
    
        
    
    positional arguments:
      sequences
    
    optional arguments:
      -h, --help            show this help message and exit
      -p, --paired
      -A, --automatic_filtering
      -t, --tarean_mode     analyze only tandem reapeats without additional classification
      -l LOGFILE, --logfile LOGFILE
                            log file, logging goes to stdout if not defines
      -m {float range 0.0..100.0}, --mincl {float range 0.0..100.0}
      -M {0,float range 0.1..1}, --merge_threshold {0,float range 0.1..1}
                            threshold for mate-pair based cluster merging, default 0 - no merging
      -o {float range 30.0..80.0}, --min_lcov {float range 30.0..80.0}
                            minimal overlap coverage - relative to longer sequence length, default 55
      -c CPU, --cpu CPU     number of cpu to use, if 0 use max available
      -s SAMPLE, --sample SAMPLE
                            use only sample of input data[by default max reads is used
      -P PREFIX_LENGTH, --prefix_length PREFIX_LENGTH
                            If you wish to keep part of the sequences name,
                             enter the number of characters which should be 
                            kept (1-10) instead of zero. Use this setting if
                             you are doing comparative analysis
      -v OUTPUT_DIR, --output_dir OUTPUT_DIR
      -r MAX_MEMORY, --max_memory MAX_MEMORY
                            Maximal amount of available RAM in kB if not set
                            clustering tries to use whole available RAM
      -d DATABASE DATABASE, --database DATABASE DATABASE
                            fasta file with database for annotation and name of database
      -C, --cleanup         remove unncessary large files from working directory
      -k, --keep_names      keep sequence names, by default sequences are renamed
      -a {2,3,4,5}, --assembly_min {2,3,4,5}
                            Assembly is performed on individual clusters, by default 
                            clusters with size less then 5 are not assembled. If you 
                            want need assembly of smaller cluster set *assmbly_min* 
                            accordingly
      -tax {VIRIDIPLANTAE3.0,VIRIDIPLANTAE2.2,METAZOA2.0,METAZOA3.0}, --taxon {VIRIDIPLANTAE3.0,VIRIDIPLANTAE2.2,METAZOA2.0,METAZOA3.0}
                            Select taxon and protein database version
      -opt {ILLUMINA,ILLUMINA_DUST_OFF,ILLUMINA_SHORT,OXFORD_NANOPORE}, --options {ILLUMINA,ILLUMINA_DUST_OFF,ILLUMINA_SHORT,OXFORD_NANOPORE}
      -D {BLASTX_W2,BLASTX_W3,DIAMOND}, --domain_search {BLASTX_W2,BLASTX_W3,DIAMOND}
                            Detection of protein domains can be performed by either blastx or
                             diamond" program. options are:
                              BLASTX_W2 - blastx with word size 2 (slowest, the most sesitive)
                              BLASTX_W3 - blastx with word size 3 (default)
                              DIAMOND   - diamond program (significantly faster, less sensitive)
                            To use this option diamond program must be installed in your PATH
    


## Galaxy toolshed
TODO

## Reproducibility
To make clustering reproducible between runs with the 
same data, environment variable PYTHONHASHSEED must be set:

	export PYTHONHASHSEED=0
    
## Disk space requirements
Large sqlite database for temporal data is created in OS specific temp directory- usually /tmp/ 
To use alternative location, it is necessary specify `TEMP` environment variable.

## CPU and RAM requirements

Resources requirements can be set either from command line arguments `--max-memory` and `--cpu` or
using environment variables `TAREAN_MAX_MEM` and `TAREAN_CPU`. If not set, pipeline use all
available resources

## How cite

If you use RepeatExplorer for general repeat characterization in your work please cite:

 - [Novak, P., Neumann, P., Pech, J., Steinhaisl, J., Macas, J. (2013) - RepeatExplorer: a Galaxy-based web server for genome-wide characterization of eukaryotic repetitive elements from next generation sequence read. Bioinformatics 29:792-793](http://bioinformatics.oxfordjournals.org/content/29/6/792)

or

 - [Novak, P., Neumann, P., Macas, J. (2010) - Graph-based clustering and characterization of repetitive sequences in next-generation sequencing data. BMC Bioinformatics 11 :37](http://www.biomedcentral.com/1471-2105/11/378)

If you use TAREAN for satellite detection and characterization please cite:

 - [Novak, P., Robledillo, L.A.,Koblizkova, A., Vrbova, I., Neumann, P., Macas, J. (2017) - TAREAN: a computational tool for identification and characterization of satellite DNA from unassembled short reads. Nucleic Acid Research](https://doi.org/10.1093/nar/gkx257)