sklearn_svm_classifier: train_test

annotate train_test_eval.py @ 11:7d7379dfef8b draft

"planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"

author	bgruening
date	Fri, 13 Sep 2019 12:12:23 -0400
parents	153f237ddb36
children	2df8f5c30edc

rev	line source
10 153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	1 import argparse
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	2 import joblib
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	3 import json
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	4 import numpy as np
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	5 import pandas as pd
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	6 import pickle
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	7 import warnings
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	8 from itertools import chain
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	9 from scipy.io import mmread
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	10 from sklearn.base import clone
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	11 from sklearn import (cluster, compose, decomposition, ensemble,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	12 feature_extraction, feature_selection,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	13 gaussian_process, kernel_approximation, metrics,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	14 model_selection, naive_bayes, neighbors,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	15 pipeline, preprocessing, svm, linear_model,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	16 tree, discriminant_analysis)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	17 from sklearn.exceptions import FitFailedWarning
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	18 from sklearn.metrics.scorer import _check_multimetric_scoring
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	19 from sklearn.model_selection._validation import _score, cross_validate
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	20 from sklearn.model_selection import _search, _validation
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	21 from sklearn.utils import indexable, safe_indexing
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	22
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	23 from galaxy_ml.model_validations import train_test_split
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	24 from galaxy_ml.utils import (SafeEval, get_scoring, load_model,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	25 read_columns, try_get_attr, get_module)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	26
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	27
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	28 _fit_and_score = try_get_attr('galaxy_ml.model_validations', '_fit_and_score')
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	29 setattr(_search, '_fit_and_score', _fit_and_score)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	30 setattr(_validation, '_fit_and_score', _fit_and_score)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	31
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	32 N_JOBS = int(__import__('os').environ.get('GALAXY_SLOTS', 1))
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	33 CACHE_DIR = './cached'
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	34 NON_SEARCHABLE = ('n_jobs', 'pre_dispatch', 'memory', '_path',
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	35 'nthread', 'callbacks')
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	36 ALLOWED_CALLBACKS = ('EarlyStopping', 'TerminateOnNaN', 'ReduceLROnPlateau',
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	37 'CSVLogger', 'None')
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	38
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	39
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	40 def _eval_swap_params(params_builder):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	41 swap_params = {}
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	42
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	43 for p in params_builder['param_set']:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	44 swap_value = p['sp_value'].strip()
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	45 if swap_value == '':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	46 continue
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	47
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	48 param_name = p['sp_name']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	49 if param_name.lower().endswith(NON_SEARCHABLE):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	50 warnings.warn("Warning: `%s` is not eligible for search and was "
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	51 "omitted!" % param_name)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	52 continue
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	53
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	54 if not swap_value.startswith(':'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	55 safe_eval = SafeEval(load_scipy=True, load_numpy=True)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	56 ev = safe_eval(swap_value)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	57 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	58 # Have `:` before search list, asks for estimator evaluatio
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	59 safe_eval_es = SafeEval(load_estimators=True)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	60 swap_value = swap_value[1:].strip()
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	61 # TODO maybe add regular express check
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	62 ev = safe_eval_es(swap_value)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	63
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	64 swap_params[param_name] = ev
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	65
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	66 return swap_params
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	67
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	68
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	69 def train_test_split_none(arrays, *kwargs):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	70 """extend train_test_split to take None arrays
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	71 and support split by group names.
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	72 """
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	73 nones = []
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	74 new_arrays = []
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	75 for idx, arr in enumerate(arrays):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	76 if arr is None:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	77 nones.append(idx)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	78 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	79 new_arrays.append(arr)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	80
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	81 if kwargs['shuffle'] == 'None':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	82 kwargs['shuffle'] = None
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	83
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	84 group_names = kwargs.pop('group_names', None)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	85
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	86 if group_names is not None and group_names.strip():
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	87 group_names = [name.strip() for name in
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	88 group_names.split(',')]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	89 new_arrays = indexable(*new_arrays)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	90 groups = kwargs['labels']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	91 n_samples = new_arrays[0].shape[0]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	92 index_arr = np.arange(n_samples)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	93 test = index_arr[np.isin(groups, group_names)]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	94 train = index_arr[~np.isin(groups, group_names)]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	95 rval = list(chain.from_iterable(
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	96 (safe_indexing(a, train),
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	97 safe_indexing(a, test)) for a in new_arrays))
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	98 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	99 rval = train_test_split(new_arrays, *kwargs)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	100
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	101 for pos in nones:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	102 rval[pos * 2: 2] = [None, None]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	103
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	104 return rval
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	105
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	106
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	107 def main(inputs, infile_estimator, infile1, infile2,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	108 outfile_result, outfile_object=None,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	109 outfile_weights=None, groups=None,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	110 ref_seq=None, intervals=None, targets=None,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	111 fasta_path=None):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	112 """
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	113 Parameter
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	114 ---------
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	115 inputs : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	116 File path to galaxy tool parameter
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	117
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	118 infile_estimator : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	119 File path to estimator
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	120
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	121 infile1 : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	122 File path to dataset containing features
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	123
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	124 infile2 : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	125 File path to dataset containing target values
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	126
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	127 outfile_result : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	128 File path to save the results, either cv_results or test result
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	129
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	130 outfile_object : str, optional
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	131 File path to save searchCV object
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	132
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	133 outfile_weights : str, optional
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	134 File path to save deep learning model weights
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	135
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	136 groups : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	137 File path to dataset containing groups labels
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	138
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	139 ref_seq : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	140 File path to dataset containing genome sequence file
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	141
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	142 intervals : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	143 File path to dataset containing interval file
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	144
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	145 targets : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	146 File path to dataset compressed target bed file
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	147
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	148 fasta_path : str
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	149 File path to dataset containing fasta file
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	150 """
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	151 warnings.simplefilter('ignore')
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	152
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	153 with open(inputs, 'r') as param_handler:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	154 params = json.load(param_handler)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	155
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	156 # load estimator
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	157 with open(infile_estimator, 'rb') as estimator_handler:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	158 estimator = load_model(estimator_handler)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	159
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	160 # swap hyperparameter
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	161 swapping = params['experiment_schemes']['hyperparams_swapping']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	162 swap_params = _eval_swap_params(swapping)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	163 estimator.set_params(**swap_params)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	164
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	165 estimator_params = estimator.get_params()
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	166
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	167 # store read dataframe object
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	168 loaded_df = {}
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	169
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	170 input_type = params['input_options']['selected_input']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	171 # tabular input
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	172 if input_type == 'tabular':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	173 header = 'infer' if params['input_options']['header1'] else None
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	174 column_option = (params['input_options']['column_selector_options_1']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	175 ['selected_column_selector_option'])
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	176 if column_option in ['by_index_number', 'all_but_by_index_number',
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	177 'by_header_name', 'all_but_by_header_name']:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	178 c = params['input_options']['column_selector_options_1']['col1']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	179 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	180 c = None
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	181
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	182 df_key = infile1 + repr(header)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	183 df = pd.read_csv(infile1, sep='\t', header=header,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	184 parse_dates=True)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	185 loaded_df[df_key] = df
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	186
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	187 X = read_columns(df, c=c, c_option=column_option).astype(float)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	188 # sparse input
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	189 elif input_type == 'sparse':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	190 X = mmread(open(infile1, 'r'))
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	191
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	192 # fasta_file input
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	193 elif input_type == 'seq_fasta':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	194 pyfaidx = get_module('pyfaidx')
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	195 sequences = pyfaidx.Fasta(fasta_path)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	196 n_seqs = len(sequences.keys())
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	197 X = np.arange(n_seqs)[:, np.newaxis]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	198 for param in estimator_params.keys():
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	199 if param.endswith('fasta_path'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	200 estimator.set_params(
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	201 **{param: fasta_path})
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	202 break
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	203 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	204 raise ValueError(
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	205 "The selected estimator doesn't support "
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	206 "fasta file input! Please consider using "
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	207 "KerasGBatchClassifier with "
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	208 "FastaDNABatchGenerator/FastaProteinBatchGenerator "
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	209 "or having GenomeOneHotEncoder/ProteinOneHotEncoder "
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	210 "in pipeline!")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	211
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	212 elif input_type == 'refseq_and_interval':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	213 path_params = {
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	214 'data_batch_generator__ref_genome_path': ref_seq,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	215 'data_batch_generator__intervals_path': intervals,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	216 'data_batch_generator__target_path': targets
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	217 }
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	218 estimator.set_params(**path_params)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	219 n_intervals = sum(1 for line in open(intervals))
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	220 X = np.arange(n_intervals)[:, np.newaxis]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	221
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	222 # Get target y
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	223 header = 'infer' if params['input_options']['header2'] else None
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	224 column_option = (params['input_options']['column_selector_options_2']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	225 ['selected_column_selector_option2'])
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	226 if column_option in ['by_index_number', 'all_but_by_index_number',
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	227 'by_header_name', 'all_but_by_header_name']:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	228 c = params['input_options']['column_selector_options_2']['col2']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	229 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	230 c = None
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	231
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	232 df_key = infile2 + repr(header)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	233 if df_key in loaded_df:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	234 infile2 = loaded_df[df_key]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	235 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	236 infile2 = pd.read_csv(infile2, sep='\t',
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	237 header=header, parse_dates=True)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	238 loaded_df[df_key] = infile2
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	239
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	240 y = read_columns(
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	241 infile2,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	242 c=c,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	243 c_option=column_option,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	244 sep='\t',
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	245 header=header,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	246 parse_dates=True)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	247 if len(y.shape) == 2 and y.shape[1] == 1:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	248 y = y.ravel()
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	249 if input_type == 'refseq_and_interval':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	250 estimator.set_params(
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	251 data_batch_generator__features=y.ravel().tolist())
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	252 y = None
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	253 # end y
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	254
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	255 # load groups
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	256 if groups:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	257 groups_selector = (params['experiment_schemes']['test_split']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	258 ['split_algos']).pop('groups_selector')
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	259
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	260 header = 'infer' if groups_selector['header_g'] else None
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	261 column_option = \
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	262 (groups_selector['column_selector_options_g']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	263 ['selected_column_selector_option_g'])
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	264 if column_option in ['by_index_number', 'all_but_by_index_number',
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	265 'by_header_name', 'all_but_by_header_name']:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	266 c = groups_selector['column_selector_options_g']['col_g']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	267 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	268 c = None
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	269
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	270 df_key = groups + repr(header)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	271 if df_key in loaded_df:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	272 groups = loaded_df[df_key]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	273
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	274 groups = read_columns(
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	275 groups,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	276 c=c,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	277 c_option=column_option,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	278 sep='\t',
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	279 header=header,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	280 parse_dates=True)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	281 groups = groups.ravel()
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	282
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	283 # del loaded_df
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	284 del loaded_df
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	285
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	286 # handle memory
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	287 memory = joblib.Memory(location=CACHE_DIR, verbose=0)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	288 # cache iraps_core fits could increase search speed significantly
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	289 if estimator.__class__.__name__ == 'IRAPSClassifier':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	290 estimator.set_params(memory=memory)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	291 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	292 # For iraps buried in pipeline
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	293 new_params = {}
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	294 for p, v in estimator_params.items():
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	295 if p.endswith('memory'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	296 # for case of `__irapsclassifier__memory`
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	297 if len(p) > 8 and p[:-8].endswith('irapsclassifier'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	298 # cache iraps_core fits could increase search
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	299 # speed significantly
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	300 new_params[p] = memory
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	301 # security reason, we don't want memory being
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	302 # modified unexpectedly
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	303 elif v:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	304 new_params[p] = None
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	305 # handle n_jobs
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	306 elif p.endswith('n_jobs'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	307 # For now, 1 CPU is suggested for iprasclassifier
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	308 if len(p) > 8 and p[:-8].endswith('irapsclassifier'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	309 new_params[p] = 1
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	310 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	311 new_params[p] = N_JOBS
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	312 # for security reason, types of callback are limited
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	313 elif p.endswith('callbacks'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	314 for cb in v:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	315 cb_type = cb['callback_selection']['callback_type']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	316 if cb_type not in ALLOWED_CALLBACKS:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	317 raise ValueError(
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	318 "Prohibited callback type: %s!" % cb_type)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	319
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	320 estimator.set_params(**new_params)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	321
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	322 # handle scorer, convert to scorer dict
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	323 scoring = params['experiment_schemes']['metrics']['scoring']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	324 scorer = get_scoring(scoring)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	325 scorer, _ = _check_multimetric_scoring(estimator, scoring=scorer)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	326
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	327 # handle test (first) split
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	328 test_split_options = (params['experiment_schemes']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	329 ['test_split']['split_algos'])
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	330
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	331 if test_split_options['shuffle'] == 'group':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	332 test_split_options['labels'] = groups
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	333 if test_split_options['shuffle'] == 'stratified':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	334 if y is not None:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	335 test_split_options['labels'] = y
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	336 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	337 raise ValueError("Stratified shuffle split is not "
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	338 "applicable on empty target values!")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	339
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	340 X_train, X_test, y_train, y_test, groups_train, groups_test = \
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	341 train_test_split_none(X, y, groups, **test_split_options)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	342
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	343 exp_scheme = params['experiment_schemes']['selected_exp_scheme']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	344
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	345 # handle validation (second) split
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	346 if exp_scheme == 'train_val_test':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	347 val_split_options = (params['experiment_schemes']
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	348 ['val_split']['split_algos'])
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	349
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	350 if val_split_options['shuffle'] == 'group':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	351 val_split_options['labels'] = groups_train
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	352 if val_split_options['shuffle'] == 'stratified':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	353 if y_train is not None:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	354 val_split_options['labels'] = y_train
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	355 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	356 raise ValueError("Stratified shuffle split is not "
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	357 "applicable on empty target values!")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	358
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	359 X_train, X_val, y_train, y_val, groups_train, groups_val = \
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	360 train_test_split_none(X_train, y_train, groups_train,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	361 **val_split_options)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	362
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	363 # train and eval
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	364 if hasattr(estimator, 'validation_data'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	365 if exp_scheme == 'train_val_test':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	366 estimator.fit(X_train, y_train,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	367 validation_data=(X_val, y_val))
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	368 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	369 estimator.fit(X_train, y_train,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	370 validation_data=(X_test, y_test))
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	371 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	372 estimator.fit(X_train, y_train)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	373
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	374 if hasattr(estimator, 'evaluate'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	375 scores = estimator.evaluate(X_test, y_test=y_test,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	376 scorer=scorer,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	377 is_multimetric=True)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	378 else:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	379 scores = _score(estimator, X_test, y_test, scorer,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	380 is_multimetric=True)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	381 # handle output
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	382 for name, score in scores.items():
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	383 scores[name] = [score]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	384 df = pd.DataFrame(scores)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	385 df = df[sorted(df.columns)]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	386 df.to_csv(path_or_buf=outfile_result, sep='\t',
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	387 header=True, index=False)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	388
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	389 memory.clear(warn=False)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	390
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	391 if outfile_object:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	392 main_est = estimator
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	393 if isinstance(estimator, pipeline.Pipeline):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	394 main_est = estimator.steps[-1][-1]
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	395
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	396 if hasattr(main_est, 'model_') \
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	397 and hasattr(main_est, 'save_weights'):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	398 if outfile_weights:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	399 main_est.save_weights(outfile_weights)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	400 del main_est.model_
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	401 del main_est.fit_params
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	402 del main_est.model_class_
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	403 del main_est.validation_data
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	404 if getattr(main_est, 'data_generator_', None):
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	405 del main_est.data_generator_
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	406
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	407 with open(outfile_object, 'wb') as output_handler:
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	408 pickle.dump(estimator, output_handler,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	409 pickle.HIGHEST_PROTOCOL)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	410
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	411
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	412 if __name__ == '__main__':
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	413 aparser = argparse.ArgumentParser()
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	414 aparser.add_argument("-i", "--inputs", dest="inputs", required=True)
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	415 aparser.add_argument("-e", "--estimator", dest="infile_estimator")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	416 aparser.add_argument("-X", "--infile1", dest="infile1")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	417 aparser.add_argument("-y", "--infile2", dest="infile2")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	418 aparser.add_argument("-O", "--outfile_result", dest="outfile_result")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	419 aparser.add_argument("-o", "--outfile_object", dest="outfile_object")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	420 aparser.add_argument("-w", "--outfile_weights", dest="outfile_weights")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	421 aparser.add_argument("-g", "--groups", dest="groups")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	422 aparser.add_argument("-r", "--ref_seq", dest="ref_seq")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	423 aparser.add_argument("-b", "--intervals", dest="intervals")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	424 aparser.add_argument("-t", "--targets", dest="targets")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	425 aparser.add_argument("-f", "--fasta_path", dest="fasta_path")
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	426 args = aparser.parse_args()
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	427
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	428 main(args.inputs, args.infile_estimator, args.infile1, args.infile2,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	429 args.outfile_result, outfile_object=args.outfile_object,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	430 outfile_weights=args.outfile_weights, groups=args.groups,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	431 ref_seq=args.ref_seq, intervals=args.intervals,
153f237ddb36 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	432 targets=args.targets, fasta_path=args.fasta_path)

Mercurial > repos > bgruening > sklearn_svm_classifier

annotate train_test_eval.py @ 11:7d7379dfef8b draft