Mercurial > repos > bgruening > scipy_sparse

annotate model_prediction.py @ 27:eb3091b4ee4b draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
"planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
author bgruening
date Fri, 13 Sep 2019 12:26:41 -0400
parents d0ed8e976b79
children a2da4cebc584
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
1 import argparse
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
2 import json
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
3 import numpy as np
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
4 import pandas as pd
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
5 import tabix
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
6 import warnings
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
7
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
8 from scipy.io import mmread
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
9 from sklearn.pipeline import Pipeline
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
10
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
11 from galaxy_ml.externals.selene_sdk.sequences import Genome
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
12 from galaxy_ml.utils import (load_model, read_columns,
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
13 get_module, try_get_attr)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
14
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
15
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
16 N_JOBS = int(__import__('os').environ.get('GALAXY_SLOTS', 1))
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
17
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
18
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
19 def main(inputs, infile_estimator, outfile_predict,
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
20 infile_weights=None, infile1=None,
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
21 fasta_path=None, ref_seq=None,
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
22 vcf_path=None):
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
23 """
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
24 Parameter
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
25 ---------
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
26 inputs : str
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
27 File path to galaxy tool parameter
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
28
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
29 infile_estimator : strgit
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
30 File path to trained estimator input
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
31
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
32 outfile_predict : str
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
33 File path to save the prediction results, tabular
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
34
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
35 infile_weights : str
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
36 File path to weights input
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
37
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
38 infile1 : str
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
39 File path to dataset containing features
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
40
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
41 fasta_path : str
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
42 File path to dataset containing fasta file
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
43
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
44 ref_seq : str
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
45 File path to dataset containing the reference genome sequence.
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
46
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
47 vcf_path : str
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
48 File path to dataset containing variants info.
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
49 """
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
50 warnings.filterwarnings('ignore')
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
51
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
52 with open(inputs, 'r') as param_handler:
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
53 params = json.load(param_handler)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
54
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
55 # load model
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
56 with open(infile_estimator, 'rb') as est_handler:
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
57 estimator = load_model(est_handler)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
58
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
59 main_est = estimator
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
60 if isinstance(estimator, Pipeline):
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
61 main_est = estimator.steps[-1][-1]
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
62 if hasattr(main_est, 'config') and hasattr(main_est, 'load_weights'):
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
63 if not infile_weights or infile_weights == 'None':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
64 raise ValueError("The selected model skeleton asks for weights, "
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
65 "but dataset for weights wan not selected!")
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
66 main_est.load_weights(infile_weights)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
67
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
68 # handle data input
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
69 input_type = params['input_options']['selected_input']
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
70 # tabular input
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
71 if input_type == 'tabular':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
72 header = 'infer' if params['input_options']['header1'] else None
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
73 column_option = (params['input_options']
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
74 ['column_selector_options_1']
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
75 ['selected_column_selector_option'])
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
76 if column_option in ['by_index_number', 'all_but_by_index_number',
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
77 'by_header_name', 'all_but_by_header_name']:
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
78 c = params['input_options']['column_selector_options_1']['col1']
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
79 else:
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
80 c = None
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
81
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
82 df = pd.read_csv(infile1, sep='\t', header=header, parse_dates=True)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
83
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
84 X = read_columns(df, c=c, c_option=column_option).astype(float)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
85
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
86 if params['method'] == 'predict':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
87 preds = estimator.predict(X)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
88 else:
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
89 preds = estimator.predict_proba(X)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
90
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
91 # sparse input
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
92 elif input_type == 'sparse':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
93 X = mmread(open(infile1, 'r'))
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
94 if params['method'] == 'predict':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
95 preds = estimator.predict(X)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
96 else:
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
97 preds = estimator.predict_proba(X)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
98
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
99 # fasta input
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
100 elif input_type == 'seq_fasta':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
101 if not hasattr(estimator, 'data_batch_generator'):
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
102 raise ValueError(
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
103 "To do prediction on sequences in fasta input, "
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
104 "the estimator must be a `KerasGBatchClassifier`"
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
105 "equipped with data_batch_generator!")
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
106 pyfaidx = get_module('pyfaidx')
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
107 sequences = pyfaidx.Fasta(fasta_path)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
108 n_seqs = len(sequences.keys())
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
109 X = np.arange(n_seqs)[:, np.newaxis]
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
110 seq_length = estimator.data_batch_generator.seq_length
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
111 batch_size = getattr(estimator, 'batch_size', 32)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
112 steps = (n_seqs + batch_size - 1) // batch_size
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
113
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
114 seq_type = params['input_options']['seq_type']
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
115 klass = try_get_attr(
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
116 'galaxy_ml.preprocessors', seq_type)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
117
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
118 pred_data_generator = klass(
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
119 fasta_path, seq_length=seq_length)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
120
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
121 if params['method'] == 'predict':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
122 preds = estimator.predict(
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
123 X, data_generator=pred_data_generator, steps=steps)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
124 else:
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
125 preds = estimator.predict_proba(
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
126 X, data_generator=pred_data_generator, steps=steps)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
127
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
128 # vcf input
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
129 elif input_type == 'variant_effect':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
130 klass = try_get_attr('galaxy_ml.preprocessors',
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
131 'GenomicVariantBatchGenerator')
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
132
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
133 options = params['input_options']
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
134 options.pop('selected_input')
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
135 if options['blacklist_regions'] == 'none':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
136 options['blacklist_regions'] = None
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
137
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
138 pred_data_generator = klass(
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
139 ref_genome_path=ref_seq, vcf_path=vcf_path, **options)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
140
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
141 pred_data_generator.fit()
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
142
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
143 variants = pred_data_generator.variants
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
144 # TODO : remove the following block after galaxy-ml v0.7.13
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
145 blacklist_tabix = getattr(pred_data_generator.reference_genome_,
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
146 '_blacklist_tabix', None)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
147 clean_variants = []
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
148 if blacklist_tabix:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
149 start_radius = pred_data_generator.start_radius_
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
150 end_radius = pred_data_generator.end_radius_
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
151
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
152 for chrom, pos, name, ref, alt, strand in variants:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
153 center = pos + len(ref) // 2
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
154 start = center - start_radius
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
155 end = center + end_radius
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
156
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
157 if isinstance(pred_data_generator.reference_genome_, Genome):
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
158 if "chr" not in chrom:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
159 chrom = "chr" + chrom
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
160 if "MT" in chrom:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
161 chrom = chrom[:-1]
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
162 try:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
163 rows = blacklist_tabix.query(chrom, start, end)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
164 found = 0
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
165 for row in rows:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
166 found = 1
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
167 break
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
168 if found:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
169 continue
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
170 except tabix.TabixError:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
171 pass
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
172
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
173 clean_variants.append((chrom, pos, name, ref, alt, strand))
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
174 else:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
175 clean_variants = variants
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
176
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
177 setattr(pred_data_generator, 'variants', clean_variants)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
178
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
179 variants = np.array(clean_variants)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
180 # predict 1600 sample at once then write to file
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
181 gen_flow = pred_data_generator.flow(batch_size=1600)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
182
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
183 file_writer = open(outfile_predict, 'w')
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
184 header_row = '\t'.join(['chrom', 'pos', 'name', 'ref',
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
185 'alt', 'strand'])
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
186 file_writer.write(header_row)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
187 header_done = False
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
188
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
189 steps_done = 0
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
190
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
191 # TODO: multiple threading
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
192 try:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
193 while steps_done < len(gen_flow):
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
194 index_array = next(gen_flow.index_generator)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
195 batch_X = gen_flow._get_batches_of_transformed_samples(
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
196 index_array)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
197
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
198 if params['method'] == 'predict':
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
199 batch_preds = estimator.predict(
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
200 batch_X,
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
201 # The presence of `pred_data_generator` below is to
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
202 # override model carrying data_generator if there
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
203 # is any.
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
204 data_generator=pred_data_generator)
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
205 else:
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
206 batch_preds = estimator.predict_proba(
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
207 batch_X,
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
208 # The presence of `pred_data_generator` below is to
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
209 # override model carrying data_generator if there
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
210 # is any.
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
211 data_generator=pred_data_generator)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
212
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
213 if batch_preds.ndim == 1:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
214 batch_preds = batch_preds[:, np.newaxis]
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
215
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
216 batch_meta = variants[index_array]
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
217 batch_out = np.column_stack([batch_meta, batch_preds])
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
218
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
219 if not header_done:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
220 heads = np.arange(batch_preds.shape[-1]).astype(str)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
221 heads_str = '\t'.join(heads)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
222 file_writer.write("\t%s\n" % heads_str)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
223 header_done = True
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
224
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
225 for row in batch_out:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
226 row_str = '\t'.join(row)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
227 file_writer.write("%s\n" % row_str)
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
228
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
229 steps_done += 1
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
230
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
231 finally:
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
232 file_writer.close()
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
233 # TODO: make api `pred_data_generator.close()`
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
234 pred_data_generator.close()
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
235 return 0
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
236 # end input
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
237
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
238 # output
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
239 if len(preds.shape) == 1:
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
240 rval = pd.DataFrame(preds, columns=['Predicted'])
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
241 else:
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
242 rval = pd.DataFrame(preds)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
243
27
eb3091b4ee4b "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ba6a47bdf76bbf4cb276206ac1a8cbf61332fd16"
bgruening
parents: 26
diff changeset
244 rval.to_csv(outfile_predict, sep='\t', header=True, index=False)
26
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
245
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
246
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
247 if __name__ == '__main__':
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
248 aparser = argparse.ArgumentParser()
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
249 aparser.add_argument("-i", "--inputs", dest="inputs", required=True)
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
250 aparser.add_argument("-e", "--infile_estimator", dest="infile_estimator")
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
251 aparser.add_argument("-w", "--infile_weights", dest="infile_weights")
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
252 aparser.add_argument("-X", "--infile1", dest="infile1")
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
253 aparser.add_argument("-O", "--outfile_predict", dest="outfile_predict")
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
254 aparser.add_argument("-f", "--fasta_path", dest="fasta_path")
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
255 aparser.add_argument("-r", "--ref_seq", dest="ref_seq")
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
256 aparser.add_argument("-v", "--vcf_path", dest="vcf_path")
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
257 args = aparser.parse_args()
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
258
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
259 main(args.inputs, args.infile_estimator, args.outfile_predict,
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
260 infile_weights=args.infile_weights, infile1=args.infile1,
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
261 fasta_path=args.fasta_path, ref_seq=args.ref_seq,
d0ed8e976b79 planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty
bgruening
parents:
diff changeset
262 vcf_path=args.vcf_path)