Mercurial > repos > bgruening > sklearn_pca

annotate train_test_split.py @ 6:3af626081467 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit f031d8ddfb73cec24572648666ac44ee47f08aad
author bgruening
date Thu, 11 Aug 2022 09:59:00 +0000
parents 24c1cc2dd4a4
children c16818ce0424
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
1 import argparse
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
2 import json
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
3 import warnings
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
4
2
e36ab18cbaca "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 208a8d348e7c7a182cfbe1b6f17868146428a7e2"
bgruening
parents: 1
diff changeset
5 import pandas as pd
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
6 from galaxy_ml.model_validations import train_test_split
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
7 from galaxy_ml.utils import get_cv, read_columns
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
8
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
9
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
10 def _get_single_cv_split(params, array, infile_labels=None, infile_groups=None):
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
11 """output (train, test) subset from a cv splitter
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
12
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
13 Parameters
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
14 ----------
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
15 params : dict
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
16 Galaxy tool inputs
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
17 array : pandas DataFrame object
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
18 The target dataset to split
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
19 infile_labels : str
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
20 File path to dataset containing target values
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
21 infile_groups : str
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
22 File path to dataset containing group values
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
23 """
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
24 y = None
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
25 groups = None
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
26
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
27 nth_split = params["mode_selection"]["nth_split"]
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
28
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
29 # read groups
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
30 if infile_groups:
3
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
31 header = (
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
32 "infer"
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
33 if (params["mode_selection"]["cv_selector"]["groups_selector"]["header_g"])
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
34 else None
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
35 )
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
36 column_option = params["mode_selection"]["cv_selector"]["groups_selector"][
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
37 "column_selector_options_g"
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
38 ]["selected_column_selector_option_g"]
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
39 if column_option in [
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
40 "by_index_number",
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
41 "all_but_by_index_number",
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
42 "by_header_name",
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
43 "all_but_by_header_name",
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
44 ]:
3
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
45 c = params["mode_selection"]["cv_selector"]["groups_selector"][
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
46 "column_selector_options_g"
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
47 ]["col_g"]
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
48 else:
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
49 c = None
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
50
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
51 groups = read_columns(
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
52 infile_groups,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
53 c=c,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
54 c_option=column_option,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
55 sep="\t",
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
56 header=header,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
57 parse_dates=True,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
58 )
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
59 groups = groups.ravel()
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
60
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
61 params["mode_selection"]["cv_selector"]["groups_selector"] = groups
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
62
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
63 # read labels
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
64 if infile_labels:
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
65 target_input = params["mode_selection"]["cv_selector"].pop("target_input")
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
66 header = "infer" if target_input["header1"] else None
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
67 col_index = target_input["col"][0] - 1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
68 df = pd.read_csv(infile_labels, sep="\t", header=header, parse_dates=True)
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
69 y = df.iloc[:, col_index].values
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
70
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
71 # construct the cv splitter object
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
72 splitter, groups = get_cv(params["mode_selection"]["cv_selector"])
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
73
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
74 total_n_splits = splitter.get_n_splits(array.values, y=y, groups=groups)
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
75 if nth_split > total_n_splits:
3
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
76 raise ValueError(
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
77 "Total number of splits is {}, but got `nth_split` "
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
78 "= {}".format(total_n_splits, nth_split)
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
79 )
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
80
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
81 i = 1
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
82 for train_index, test_index in splitter.split(array.values, y=y, groups=groups):
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
83 # suppose nth_split >= 1
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
84 if i == nth_split:
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
85 break
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
86 else:
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
87 i += 1
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
88
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
89 train = array.iloc[train_index, :]
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
90 test = array.iloc[test_index, :]
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
91
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
92 return train, test
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
93
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
94
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
95 def main(
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
96 inputs,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
97 infile_array,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
98 outfile_train,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
99 outfile_test,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
100 infile_labels=None,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
101 infile_groups=None,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
102 ):
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
103 """
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
104 Parameter
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
105 ---------
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
106 inputs : str
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
107 File path to galaxy tool parameter
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
108
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
109 infile_array : str
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
110 File paths of input arrays separated by comma
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
111
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
112 infile_labels : str
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
113 File path to dataset containing labels
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
114
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
115 infile_groups : str
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
116 File path to dataset containing groups
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
117
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
118 outfile_train : str
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
119 File path to dataset containing train split
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
120
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
121 outfile_test : str
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
122 File path to dataset containing test split
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
123 """
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
124 warnings.simplefilter("ignore")
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
125
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
126 with open(inputs, "r") as param_handler:
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
127 params = json.load(param_handler)
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
128
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
129 input_header = params["header0"]
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
130 header = "infer" if input_header else None
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
131 array = pd.read_csv(infile_array, sep="\t", header=header, parse_dates=True)
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
132
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
133 # train test split
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
134 if params["mode_selection"]["selected_mode"] == "train_test_split":
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
135 options = params["mode_selection"]["options"]
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
136 shuffle_selection = options.pop("shuffle_selection")
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
137 options["shuffle"] = shuffle_selection["shuffle"]
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
138 if infile_labels:
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
139 header = "infer" if shuffle_selection["header1"] else None
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
140 col_index = shuffle_selection["col"][0] - 1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
141 df = pd.read_csv(infile_labels, sep="\t", header=header, parse_dates=True)
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
142 labels = df.iloc[:, col_index].values
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
143 options["labels"] = labels
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
144
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
145 train, test = train_test_split(array, **options)
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
146
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
147 # cv splitter
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
148 else:
3
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
149 train, test = _get_single_cv_split(
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
150 params, array, infile_labels=infile_labels, infile_groups=infile_groups
24c1cc2dd4a4 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit ea12f973df4b97a2691d9e4ce6bf6fae59d57717"
bgruening
parents: 2
diff changeset
151 )
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
152
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
153 print("Input shape: %s" % repr(array.shape))
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
154 print("Train shape: %s" % repr(train.shape))
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
155 print("Test shape: %s" % repr(test.shape))
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
156 train.to_csv(outfile_train, sep="\t", header=input_header, index=False)
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
157 test.to_csv(outfile_test, sep="\t", header=input_header, index=False)
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
158
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
159
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
160 if __name__ == "__main__":
0
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
161 aparser = argparse.ArgumentParser()
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
162 aparser.add_argument("-i", "--inputs", dest="inputs", required=True)
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
163 aparser.add_argument("-X", "--infile_array", dest="infile_array")
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
164 aparser.add_argument("-y", "--infile_labels", dest="infile_labels")
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
165 aparser.add_argument("-g", "--infile_groups", dest="infile_groups")
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
166 aparser.add_argument("-o", "--outfile_train", dest="outfile_train")
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
167 aparser.add_argument("-t", "--outfile_test", dest="outfile_test")
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
168 args = aparser.parse_args()
2d7016b3ae92 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 2afb24f3c81d625312186750a714d702363012b5"
bgruening
parents:
diff changeset
169
1
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
170 main(
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
171 args.inputs,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
172 args.infile_array,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
173 args.outfile_train,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
174 args.outfile_test,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
175 args.infile_labels,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
176 args.infile_groups,
132805688fa3 "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit e2a5eade6d0e5ddf3a47630381a0ad90d80e8a04"
bgruening
parents: 0
diff changeset
177 )