<tool id="sklearn_ensemble" name="Ensemble methods" version="@VERSION@">
    <description>for classification and regression</description>
    <macros>
        <import>main_macros.xml</import>
    </macros>
    <expand macro="python_requirements"/>
    <expand macro="macro_stdio"/>
    <version_command>echo "@VERSION@"</version_command>
    <command><![CDATA[
    python "$ensemble_script" '$inputs'
]]>
    </command>
    <configfiles>
        <inputs name="inputs"/>
        <configfile name="ensemble_script">
<![CDATA[
import json
import numpy as np
import pandas
import pickle
import sys

from scipy.io import mmread
import sklearn.ensemble
from galaxy_ml.utils import load_model, get_X_y


N_JOBS = int(__import__('os').environ.get('GALAXY_SLOTS', 1))

# Get inputs, outputs.
input_json_path = sys.argv[1]
with open(input_json_path, "r") as param_handler:
    params = json.load(param_handler)
print(params)

# Put all cheetah up here to avoid confusion.
#if $selected_tasks.selected_task == "train":
infile1 = "$selected_tasks.selected_algorithms.input_options.infile1"
infile2 = "$selected_tasks.selected_algorithms.input_options.infile2"
#else:
infile_model = "$selected_tasks.infile_model"
infile_data = "$selected_tasks.infile_data"
#end if
outfile_fit = "$outfile_fit"
outfile_predict = "$outfile_predict"

# All Python from here on out:

if params["selected_tasks"]["selected_task"] == "train":
    algorithm = params["selected_tasks"]["selected_algorithms"]["selected_algorithm"]
    options = params["selected_tasks"]["selected_algorithms"]["options"]
    if algorithm in ['RandomForestClassifier', 'RandomForestRegressor']:
        options['n_jobs'] = N_JOBS
    if "select_max_features" in options:
        if options["select_max_features"]["max_features"] == "number_input":
            options["select_max_features"]["max_features"] = options["select_max_features"]["num_max_features"]
            options["select_max_features"].pop("num_max_features")
        options["max_features"] = options["select_max_features"]["max_features"]
        options.pop("select_max_features")
    if "presort" in options:
        if options["presort"] == "true":
            options["presort"] = True
        if options["presort"] == "false":
            options["presort"] = False
    if "min_samples_leaf" in options and options["min_samples_leaf"] == 1.0:
        options["min_samples_leaf"] = 1
    if "min_samples_split" in options and options["min_samples_split"] > 1.0:
        options["min_samples_split"] = int(options["min_samples_split"])

    X, y = get_X_y(params, infile1, infile2)

    my_class = getattr(sklearn.ensemble, algorithm)
    estimator = my_class(**options)
    estimator.fit(X,y)
    with open(outfile_fit, 'wb') as out_handler:
        pickle.dump(estimator, out_handler, pickle.HIGHEST_PROTOCOL)

else:
    with open(infile_model, 'rb') as model_handler:
        classifier_object = load_model(model_handler)
    header = 'infer' if params["selected_tasks"]["header"] else None
    data = pandas.read_csv(infile_data, sep='\t', header=header, index_col=None, parse_dates=True, encoding=None, tupleize_cols=False)
    prediction = classifier_object.predict(data)
    prediction_df = pandas.DataFrame(prediction, columns=["predicted"])
    res = pandas.concat([data, prediction_df], axis=1)
    res.to_csv(path_or_buf = outfile_predict, sep="\t", index=False)

]]>
        </configfile>
    </configfiles>
    <inputs>
        <expand macro="sl_Conditional" model="zip">
            <param name="selected_algorithm" type="select" label="Select an ensemble method:">
                <option value="RandomForestClassifier" selected="true">Random forest classifier</option>
                <option value="AdaBoostClassifier">Ada boost classifier</option>
                <option value="GradientBoostingClassifier">Gradient Boosting Classifier</option>
                <option value="RandomForestRegressor">Random forest regressor</option>
                <option value="AdaBoostRegressor">Ada boost regressor</option>
                <option value="GradientBoostingRegressor">Gradient Boosting Regressor</option>
            </param>
            <when value="RandomForestClassifier">
                <expand macro="sl_mixed_input"/>
                <section name="options" title="Advanced Options" expanded="False">
                    <expand macro="n_estimators" default_value="100"/>
                    <expand macro="criterion"/>
                    <expand macro="max_features"/>
                    <expand macro="max_depth"/>
                    <expand macro="min_samples_split"/>
                    <expand macro="min_samples_leaf"/>
                    <expand macro="min_weight_fraction_leaf"/>
                    <expand macro="max_leaf_nodes"/>
                    <expand macro="bootstrap"/>
                    <expand macro="warm_start" checked="false"/>
                    <expand macro="random_state"/>
                    <expand macro="oob_score"/>
                    <!--class_weight=None-->
                </section>
            </when>
            <when value="AdaBoostClassifier">
                <expand macro="sl_mixed_input"/>
                <section name="options" title="Advanced Options" expanded="False">
                    <!--base_estimator=None-->
                    <expand macro="n_estimators" default_value="50"/>
                    <expand macro="learning_rate"/>
                    <param argument="algorithm" type="select" label="Boosting algorithm"  help=" ">
                        <option value="SAMME.R" selected="true">SAMME.R</option>
                        <option value="SAMME">SAMME</option>
                    </param>
                    <expand macro="random_state"/>
                </section>
            </when>
            <when value="GradientBoostingClassifier">
                <expand macro="sl_mixed_input"/>
                <section name="options" title="Advanced Options" expanded="False">
                    <!--base_estimator=None-->
                    <param argument="loss" type="select" label="Loss function">
                        <option value="deviance" selected="true">deviance - logistic regression with probabilistic outputs</option>
                        <option value="exponential">exponential - gradient boosting recovers the AdaBoost algorithm</option>
                    </param>
                    <expand macro="learning_rate" default_value='0.1'/>
                    <expand macro="n_estimators" default_value="100" help="The number of boosting stages to perform"/>
                    <expand macro="max_depth" default_value="3" help="maximum depth of the individual regression estimators"/>
                    <expand macro="criterion2">
                        <option value="friedman_mse" selected="true">friedman_mse - mean squared error with improvement score by Friedman</option>
                    </expand>
                    <expand macro="min_samples_split" type="float"/>
                    <expand macro="min_samples_leaf" type="float" label="The minimum number of samples required to be at a leaf node"/>
                    <expand macro="min_weight_fraction_leaf"/>
                    <expand macro="subsample"/>
                    <expand macro="max_features"/>
                    <expand macro="max_leaf_nodes"/>
                    <expand macro="min_impurity_decrease"/>
                    <expand macro="verbose"/>
                    <expand macro="warm_start" checked="false"/>
                    <expand macro="random_state"/>
                    <expand macro="presort"/>
                </section>
            </when>
            <when value="RandomForestRegressor">
                <expand macro="sl_mixed_input"/>
                <section name="options" title="Advanced Options" expanded="False">
                    <expand macro="n_estimators" default_value="100"/>
                    <expand macro="criterion2"/>
                    <expand macro="max_features"/>
                    <expand macro="max_depth"/>
                    <expand macro="min_samples_split"/>
                    <expand macro="min_samples_leaf"/>
                    <expand macro="min_weight_fraction_leaf"/>
                    <expand macro="max_leaf_nodes"/>
                    <expand macro="min_impurity_decrease"/>
                    <expand macro="bootstrap"/>
                    <expand macro="oob_score"/>
                    <expand macro="random_state"/>
                    <expand macro="verbose"/>
                    <expand macro="warm_start" checked="false"/>
                </section>
            </when>
            <when value="AdaBoostRegressor">
                <expand macro="sl_mixed_input"/>
                <section name="options" title="Advanced Options" expanded="False">
                    <!--base_estimator=None-->
                    <expand macro="n_estimators" default_value="50"/>
                    <expand macro="learning_rate"/>
                    <param argument="loss" type="select" label="Loss function"  optional="true" help="Used when updating the weights after each boosting iteration. ">
                        <option value="linear" selected="true">linear</option>
                        <option value="square">square</option>
                        <option value="exponential">exponential</option>
                    </param>
                    <expand macro="random_state"/>
                </section>
            </when>
            <when value="GradientBoostingRegressor">
                <expand macro="sl_mixed_input"/>
                <section name="options" title="Advanced Options" expanded="False">
                    <param argument="loss" type="select" label="Loss function">
                        <option value="ls" selected="true">ls - least squares regression</option>
                        <option value="lad">lad - least absolute deviation</option>
                        <option value="huber">huber - combination of least squares regression and least absolute deviation</option>
                        <option value="quantile">quantile - use alpha to specify the quantile</option>
                    </param>
                    <expand macro="learning_rate" default_value="0.1"/>
                    <expand macro="n_estimators" default_value="100" help="The number of boosting stages to perform"/>
                    <expand macro="max_depth" default_value="3" help="maximum depth of the individual regression estimators"/>
                    <expand macro="criterion2">
                        <option value="friedman_mse" selected="true">friedman_mse - mean squared error with improvement score by Friedman</option>
                    </expand>
                    <expand macro="min_samples_split" type="float"/>
                    <expand macro="min_samples_leaf" type="float" label="The minimum number of samples required to be at a leaf node"/>
                    <expand macro="min_weight_fraction_leaf"/>
                    <expand macro="subsample"/>
                    <expand macro="max_features"/>
                    <expand macro="max_leaf_nodes"/>
                    <expand macro="min_impurity_decrease"/>
                    <param argument="alpha" type="float" value="0.9" label="alpha" help="The alpha-quantile of the huber loss function and the quantile loss function" />
                    <!--base_estimator=None-->
                    <expand macro="verbose"/>
                    <expand macro="warm_start" checked="false"/>
                    <expand macro="random_state"/>
                    <expand macro="presort"/>
                </section>
            </when>
        </expand>
    </inputs>

    <expand macro="output"/>

    <tests>
        <test>
            <param name="infile1" value="train.tabular" ftype="tabular"/>
            <param name="infile2" value="train.tabular" ftype="tabular"/>
            <param name="col1" value="1,2,3,4"/>
            <param name="col2" value="5"/>
            <param name="selected_task" value="train"/>
            <param name="selected_algorithm" value="RandomForestClassifier"/>
            <param name="random_state" value="10"/>
            <output name="outfile_fit" file="rfc_model01" compare="sim_size" delta="5"/>
        </test>
        <test>
            <param name="infile_model" value="rfc_model01" ftype="zip"/>
            <param name="infile_data" value="test.tabular" ftype="tabular"/>
            <param name="selected_task" value="load"/>
            <output name="outfile_predict" file="rfc_result01"/>
        </test>
        <test>
            <param name="infile1" value="regression_train.tabular" ftype="tabular"/>
            <param name="infile2" value="regression_train.tabular" ftype="tabular"/>
            <param name="col1" value="1,2,3,4,5"/>
            <param name="col2" value="6"/>
            <param name="selected_task" value="train"/>
            <param name="selected_algorithm" value="RandomForestRegressor"/>
            <param name="random_state" value="10"/>
            <output name="outfile_fit" file="rfr_model01" compare="sim_size" delta="5"/>
        </test>
        <test>
            <param name="infile_model" value="rfr_model01" ftype="zip"/>
            <param name="infile_data" value="regression_test.tabular" ftype="tabular"/>
            <param name="selected_task" value="load"/>
            <output name="outfile_predict" file="rfr_result01"/>
        </test>
        <test>
            <param name="infile1" value="regression_X.tabular" ftype="tabular"/>
            <param name="infile2" value="regression_y.tabular" ftype="tabular"/>
            <param name="header1" value="True"/>
            <param name="selected_column_selector_option" value="all_columns"/>
            <param name="header2" value="True"/>
            <param name="col2" value="1"/>
            <param name="selected_task" value="train"/>
            <param name="selected_algorithm" value="GradientBoostingRegressor"/>
            <param name="max_features" value="number_input"/>
            <param name="num_max_features" value="0.5"/>
            <param name="random_state" value="42"/>
            <output name="outfile_fit" file="gbr_model01" compare="sim_size" delta="5"/>
        </test>
        <test>
            <param name="infile_model" value="gbr_model01" ftype="zip"/>
            <param name="infile_data" value="regression_test_X.tabular" ftype="tabular"/>
            <param name="selected_task" value="load"/>
            <param name="header" value="True"/>
            <output name="outfile_predict" file="gbr_prediction_result01.tabular"/>
        </test>
        <test>
            <param name="infile1" value="train.tabular" ftype="tabular"/>
            <param name="infile2" value="train.tabular" ftype="tabular"/>
            <param name="col1" value="1,2,3,4"/>
            <param name="col2" value="5"/>
            <param name="selected_task" value="train"/>
            <param name="selected_algorithm" value="GradientBoostingClassifier"/>
            <output name="outfile_fit" file="gbc_model01" compare="sim_size" delta="5"/>
        </test>
        <test>
            <param name="infile_model" value="gbc_model01" ftype="zip"/>
            <param name="infile_data" value="test.tabular" ftype="tabular"/>
            <param name="selected_task" value="load"/>
            <output name="outfile_predict" file="gbc_result01"/>
        </test>
        <test>
            <param name="infile1" value="train.tabular" ftype="tabular"/>
            <param name="infile2" value="train.tabular" ftype="tabular"/>
            <param name="col1" value="1,2,3,4"/>
            <param name="col2" value="5"/>
            <param name="selected_task" value="train"/>
            <param name="selected_algorithm" value="AdaBoostClassifier"/>
            <param name="random_state" value="10"/>
            <output name="outfile_fit" file="abc_model01" compare="sim_size" delta="5"/>
        </test>
        <test>
            <param name="infile_model" value="abc_model01" ftype="zip"/>
            <param name="infile_data" value="test.tabular" ftype="tabular"/>
            <param name="selected_task" value="load"/>
            <output name="outfile_predict" file="abc_result01"/>
        </test>
        <test>
            <param name="infile1" value="regression_train.tabular" ftype="tabular"/>
            <param name="infile2" value="regression_train.tabular" ftype="tabular"/>
            <param name="col1" value="1,2,3,4,5"/>
            <param name="col2" value="6"/>
            <param name="selected_task" value="train"/>
            <param name="selected_algorithm" value="AdaBoostRegressor"/>
            <param name="random_state" value="10"/>
            <output name="outfile_fit" file="abr_model01" compare="sim_size" delta="5"/>
        </test>
        <test>
            <param name="infile_model" value="abr_model01" ftype="zip"/>
            <param name="infile_data" value="regression_test.tabular" ftype="tabular"/>
            <param name="selected_task" value="load"/>
            <output name="outfile_predict" file="abr_result01"/>
        </test>
    </tests>
    <help><![CDATA[
***What it does***
The goal of ensemble methods is to combine the predictions of several base estimators built with a given learning algorithm in order to improve generalizability / robustness over a single estimator. This tool offers two sets of ensemble algorithms for classification and regression: random forests and ADA boosting which are based on sklearn.ensemble library from Scikit-learn. Here you can find out about the input, output and methods presented in the tools. For information about ensemble methods and parameters settings please refer to `Scikit-learn ensemble`_.

.. _`Scikit-learn ensemble`: http://scikit-learn.org/stable/modules/ensemble.html

 **1 - Methods**
 There are two groups of operations available:

  1 - Train a model : A training set containing samples and their respective labels (or predicted values) are input. Based on the selected algorithm and options, an estimator object is fit to the data and is returned.

  2 - Load a model and predict : An existing model predicts the class labels (or regression values) for a new dataset.

 **2 - Trainig input**
 When you choose to train a model, you need a features dataset X and a labels set y. This tool expects tabular or sparse data for X and a single column for y (tabular). You can select a subset of columns in a tabular dataset as your features dataset or labels column. Below you find some examples:

 **Sample tabular features dataset**
 The following training dataset contains 3 feature columns and a column containing class labels. You can simply select the first 3 columns as features and the last column as labels:

 ::

  4.01163365529    -6.10797684314    8.29829894763     1
  10.0788438916    1.59539821454     10.0684278289     0
  -5.17607775503   -0.878286135332   6.92941850665     2
  4.00975406235    -7.11847496542    9.3802423585      1
  4.61204065139    -5.71217537352    9.12509610964     1


 **Sample sparse features dataset**
 In this case you cannot specifiy a column range.

 ::

  4 1048577 8738
  1 271 0.02083333333333341
  1 1038 0.02461995616119806
  2 829017 0.01629088031127686
  2 829437 0.01209127083516686
  2 830752 0.02535100632816968
  3 1047487 0.01485722929945572
  3 1047980 0.02640566620767753
  3 1048475 0.01665869913262564
  4 608 0.01662975263094352
  4 1651 0.02519674277562741
  4 4053 0.04223659971350601


 **2 - Trainig output**
 The trained model is generated and output in the form of a binary file.


 **3 - Prediction input**

 When you choose to load a model and do prediction, the tool expects an already trained estimator and a tabular dataset as input. The dataset contains new samples which you want to classify or predict regression values for.


 .. class:: warningmark

 The number of feature columns must be the same in training and prediction datasets!


 **3 - Prediction output**
 The tool predicts the class labels for new samples and adds them as the last column to the prediction dataset. The new dataset then is output as a tabular file. The prediction output format should look like the training dataset.

    ]]></help>
    <expand macro="sklearn_citation"/>
</tool>