Mercurial > repos > bgruening > sklearn_pairwise_metrics

annotate utils.py @ 20:9916ea4d57f4 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
author bgruening
date Thu, 23 Aug 2018 16:15:17 -0400
parents 8a9cd8e1ae5b
children ae8456eaf50c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
19
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
1 import sys
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
2 import os
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
3 import pandas
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
4 import re
20
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
5 import cPickle as pickle
19
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
6 import warnings
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
7 import numpy as np
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
8 import xgboost
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
9 import scipy
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
10 import sklearn
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
11 import ast
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
12 from asteval import Interpreter, make_symbol_table
20
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
13 from sklearn import (cluster, decomposition, ensemble, feature_extraction, feature_selection,
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
14 gaussian_process, kernel_approximation, linear_model, metrics,
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
15 model_selection, naive_bayes, neighbors, pipeline, preprocessing,
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
16 svm, linear_model, tree, discriminant_analysis)
19
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
17
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
18 N_JOBS = int( os.environ.get('GALAXY_SLOTS', 1) )
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
19
20
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
20 class SafePickler(object):
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
21 """
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
22 Used to safely deserialize scikit-learn model objects serialized by cPickle.dump
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
23 Usage:
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
24 eg.: SafePickler.load(pickled_file_object)
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
25 """
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
26 @classmethod
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
27 def find_class(self, module, name):
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
28
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
29 bad_names = ('and', 'as', 'assert', 'break', 'class', 'continue',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
30 'def', 'del', 'elif', 'else', 'except', 'exec',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
31 'finally', 'for', 'from', 'global', 'if', 'import',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
32 'in', 'is', 'lambda', 'not', 'or', 'pass', 'print',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
33 'raise', 'return', 'try', 'system', 'while', 'with',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
34 'True', 'False', 'None', 'eval', 'execfile', '__import__',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
35 '__package__', '__subclasses__', '__bases__', '__globals__',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
36 '__code__', '__closure__', '__func__', '__self__', '__module__',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
37 '__dict__', '__class__', '__call__', '__get__',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
38 '__getattribute__', '__subclasshook__', '__new__',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
39 '__init__', 'func_globals', 'func_code', 'func_closure',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
40 'im_class', 'im_func', 'im_self', 'gi_code', 'gi_frame',
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
41 '__asteval__', 'f_locals', '__mro__')
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
42 good_names = ('copy_reg._reconstructor', '__builtin__.object')
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
43
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
44 if re.match(r'^[a-zA-Z_][a-zA-Z0-9_]*$', name):
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
45 fullname = module + '.' + name
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
46 if (fullname in good_names)\
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
47 or ( ( module.startswith('sklearn.')
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
48 or module.startswith('xgboost.')
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
49 or module.startswith('skrebate.')
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
50 or module.startswith('numpy.')
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
51 or module == 'numpy'
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
52 )
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
53 and (name not in bad_names)
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
54 ) :
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
55 # TODO: replace with a whitelist checker
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
56 if fullname not in SK_NAMES + SKR_NAMES + XGB_NAMES + NUMPY_NAMES + good_names:
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
57 print("Warning: global %s is not in pickler whitelist yet and will loss support soon. Contact tool author or leave a message at github.com" % fullname)
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
58 mod = sys.modules[module]
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
59 return getattr(mod, name)
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
60
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
61 raise pickle.UnpicklingError("global '%s' is forbidden" % fullname)
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
62
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
63 @classmethod
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
64 def load(self, file):
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
65 obj = pickle.Unpickler(file)
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
66 obj.find_global = self.find_class
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
67 return obj.load()
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
68
19
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
69 def read_columns(f, c=None, c_option='by_index_number', return_df=False, **args):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
70 data = pandas.read_csv(f, **args)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
71 if c_option == 'by_index_number':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
72 cols = list(map(lambda x: x - 1, c))
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
73 data = data.iloc[:,cols]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
74 if c_option == 'all_but_by_index_number':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
75 cols = list(map(lambda x: x - 1, c))
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
76 data.drop(data.columns[cols], axis=1, inplace=True)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
77 if c_option == 'by_header_name':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
78 cols = [e.strip() for e in c.split(',')]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
79 data = data[cols]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
80 if c_option == 'all_but_by_header_name':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
81 cols = [e.strip() for e in c.split(',')]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
82 data.drop(cols, axis=1, inplace=True)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
83 y = data.values
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
84 if return_df:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
85 return y, data
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
86 else:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
87 return y
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
88 return y
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
89
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
90
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
91 ## generate an instance for one of sklearn.feature_selection classes
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
92 def feature_selector(inputs):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
93 selector = inputs["selected_algorithm"]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
94 selector = getattr(sklearn.feature_selection, selector)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
95 options = inputs["options"]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
96
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
97 if inputs['selected_algorithm'] == 'SelectFromModel':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
98 if not options['threshold'] or options['threshold'] == 'None':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
99 options['threshold'] = None
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
100 if inputs['model_inputter']['input_mode'] == 'prefitted':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
101 model_file = inputs['model_inputter']['fitted_estimator']
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
102 with open(model_file, 'rb') as model_handler:
20
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
103 fitted_estimator = SafePickler.load(model_handler)
19
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
104 new_selector = selector(fitted_estimator, prefit=True, **options)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
105 else:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
106 estimator_json = inputs['model_inputter']["estimator_selector"]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
107 estimator = get_estimator(estimator_json)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
108 new_selector = selector(estimator, **options)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
109
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
110 elif inputs['selected_algorithm'] == 'RFE':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
111 estimator=get_estimator(inputs["estimator_selector"])
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
112 new_selector = selector(estimator, **options)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
113
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
114 elif inputs['selected_algorithm'] == 'RFECV':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
115 options['scoring'] = get_scoring(options['scoring'])
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
116 options['n_jobs'] = N_JOBS
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
117 options['cv'] = get_cv( options['cv'].strip() )
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
118 estimator=get_estimator(inputs["estimator_selector"])
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
119 new_selector = selector(estimator, **options)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
120
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
121 elif inputs['selected_algorithm'] == "VarianceThreshold":
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
122 new_selector = selector(**options)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
123
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
124 else:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
125 score_func = inputs["score_func"]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
126 score_func = getattr(sklearn.feature_selection, score_func)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
127 new_selector = selector(score_func, **options)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
128
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
129 return new_selector
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
130
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
131
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
132 def get_X_y(params, file1, file2):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
133 input_type = params["selected_tasks"]["selected_algorithms"]["input_options"]["selected_input"]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
134 if input_type=="tabular":
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
135 header = 'infer' if params["selected_tasks"]["selected_algorithms"]["input_options"]["header1"] else None
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
136 column_option = params["selected_tasks"]["selected_algorithms"]["input_options"]["column_selector_options_1"]["selected_column_selector_option"]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
137 if column_option in ["by_index_number", "all_but_by_index_number", "by_header_name", "all_but_by_header_name"]:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
138 c = params["selected_tasks"]["selected_algorithms"]["input_options"]["column_selector_options_1"]["col1"]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
139 else:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
140 c = None
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
141 X = read_columns(
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
142 file1,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
143 c = c,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
144 c_option = column_option,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
145 sep='\t',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
146 header=header,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
147 parse_dates=True
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
148 )
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
149 else:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
150 X = mmread(file1)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
151
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
152 header = 'infer' if params["selected_tasks"]["selected_algorithms"]["input_options"]["header2"] else None
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
153 column_option = params["selected_tasks"]["selected_algorithms"]["input_options"]["column_selector_options_2"]["selected_column_selector_option2"]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
154 if column_option in ["by_index_number", "all_but_by_index_number", "by_header_name", "all_but_by_header_name"]:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
155 c = params["selected_tasks"]["selected_algorithms"]["input_options"]["column_selector_options_2"]["col2"]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
156 else:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
157 c = None
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
158 y = read_columns(
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
159 file2,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
160 c = c,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
161 c_option = column_option,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
162 sep='\t',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
163 header=header,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
164 parse_dates=True
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
165 )
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
166 y=y.ravel()
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
167 return X, y
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
168
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
169
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
170 class SafeEval(Interpreter):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
171
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
172 def __init__(self, load_scipy=False, load_numpy=False):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
173
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
174 # File opening and other unneeded functions could be dropped
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
175 unwanted = ['open', 'type', 'dir', 'id', 'str', 'repr']
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
176
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
177 # Allowed symbol table. Add more if needed.
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
178 new_syms = {
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
179 'np_arange': getattr(np, 'arange'),
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
180 'ensemble_ExtraTreesClassifier': getattr(ensemble, 'ExtraTreesClassifier')
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
181 }
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
182
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
183 syms = make_symbol_table(use_numpy=False, **new_syms)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
184
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
185 if load_scipy:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
186 scipy_distributions = scipy.stats.distributions.__dict__
20
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
187 for k, v in scipy_distributions.items():
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
188 if isinstance(v, (scipy.stats.rv_continuous, scipy.stats.rv_discrete)):
9916ea4d57f4 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 8cf3d813ec755166ee0bd517b4ecbbd4f84d4df1
bgruening
parents: 19
diff changeset
189 syms['scipy_stats_' + k] = v
19
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
190
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
191 if load_numpy:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
192 from_numpy_random = ['beta', 'binomial', 'bytes', 'chisquare', 'choice', 'dirichlet', 'division',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
193 'exponential', 'f', 'gamma', 'geometric', 'gumbel', 'hypergeometric',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
194 'laplace', 'logistic', 'lognormal', 'logseries', 'mtrand', 'multinomial',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
195 'multivariate_normal', 'negative_binomial', 'noncentral_chisquare', 'noncentral_f',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
196 'normal', 'pareto', 'permutation', 'poisson', 'power', 'rand', 'randint',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
197 'randn', 'random', 'random_integers', 'random_sample', 'ranf', 'rayleigh',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
198 'sample', 'seed', 'set_state', 'shuffle', 'standard_cauchy', 'standard_exponential',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
199 'standard_gamma', 'standard_normal', 'standard_t', 'triangular', 'uniform',
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
200 'vonmises', 'wald', 'weibull', 'zipf' ]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
201 for f in from_numpy_random:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
202 syms['np_random_' + f] = getattr(np.random, f)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
203
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
204 for key in unwanted:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
205 syms.pop(key, None)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
206
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
207 super(SafeEval, self).__init__( symtable=syms, use_numpy=False, minimal=False,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
208 no_if=True, no_for=True, no_while=True, no_try=True,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
209 no_functiondef=True, no_ifexp=True, no_listcomp=False,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
210 no_augassign=False, no_assert=True, no_delete=True,
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
211 no_raise=True, no_print=True)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
212
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
213
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
214 def get_search_params(params_builder):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
215 search_params = {}
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
216 safe_eval = SafeEval(load_scipy=True, load_numpy=True)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
217
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
218 for p in params_builder['param_set']:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
219 search_p = p['search_param_selector']['search_p']
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
220 if search_p.strip() == '':
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
221 continue
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
222 param_type = p['search_param_selector']['selected_param_type']
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
223
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
224 lst = search_p.split(":")
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
225 assert (len(lst) == 2), "Error, make sure there is one and only one colon in search parameter input."
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
226 literal = lst[1].strip()
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
227 ev = safe_eval(literal)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
228 if param_type == "final_estimator_p":
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
229 search_params["estimator__" + lst[0].strip()] = ev
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
230 else:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
231 search_params["preprocessing_" + param_type[5:6] + "__" + lst[0].strip()] = ev
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
232
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
233 return search_params
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
234
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
235
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
236 def get_estimator(estimator_json):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
237 estimator_module = estimator_json['selected_module']
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
238 estimator_cls = estimator_json['selected_estimator']
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
239
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
240 if estimator_module == "xgboost":
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
241 cls = getattr(xgboost, estimator_cls)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
242 else:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
243 module = getattr(sklearn, estimator_module)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
244 cls = getattr(module, estimator_cls)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
245
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
246 estimator = cls()
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
247
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
248 estimator_params = estimator_json['text_params'].strip()
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
249 if estimator_params != "":
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
250 try:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
251 params = safe_eval('dict(' + estimator_params + ')')
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
252 except ValueError:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
253 sys.exit("Unsupported parameter input: `%s`" %estimator_params)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
254 estimator.set_params(**params)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
255 if 'n_jobs' in estimator.get_params():
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
256 estimator.set_params( n_jobs=N_JOBS )
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
257
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
258 return estimator
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
259
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
260
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
261 def get_cv(literal):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
262 safe_eval = SafeEval()
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
263 if literal == "":
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
264 return None
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
265 if literal.isdigit():
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
266 return int(literal)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
267 m = re.match(r'^(?P<method>\w+)\((?P<args>.*)\)$', literal)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
268 if m:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
269 my_class = getattr( model_selection, m.group('method') )
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
270 args = safe_eval( 'dict('+ m.group('args') + ')' )
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
271 return my_class( **args )
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
272 sys.exit("Unsupported CV input: %s" %literal)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
273
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
274
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
275 def get_scoring(scoring_json):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
276 def balanced_accuracy_score(y_true, y_pred):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
277 C = metrics.confusion_matrix(y_true, y_pred)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
278 with np.errstate(divide='ignore', invalid='ignore'):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
279 per_class = np.diag(C) / C.sum(axis=1)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
280 if np.any(np.isnan(per_class)):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
281 warnings.warn('y_pred contains classes not in y_true')
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
282 per_class = per_class[~np.isnan(per_class)]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
283 score = np.mean(per_class)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
284 return score
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
285
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
286 if scoring_json['primary_scoring'] == "default":
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
287 return None
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
288
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
289 my_scorers = metrics.SCORERS
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
290 if 'balanced_accuracy' not in my_scorers:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
291 my_scorers['balanced_accuracy'] = metrics.make_scorer(balanced_accuracy_score)
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
292
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
293 if scoring_json['secondary_scoring'] != 'None'\
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
294 and scoring_json['secondary_scoring'] != scoring_json['primary_scoring']:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
295 scoring = {}
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
296 scoring['primary'] = my_scorers[ scoring_json['primary_scoring'] ]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
297 for scorer in scoring_json['secondary_scoring'].split(','):
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
298 if scorer != scoring_json['primary_scoring']:
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
299 scoring[scorer] = my_scorers[scorer]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
300 return scoring
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
301
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
302 return my_scorers[ scoring_json['primary_scoring'] ]
8a9cd8e1ae5b planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d00173591e4a783a4c1cb2664e4bb192ab5414f7
bgruening
parents:
diff changeset
303