plotly_ml_performance_plots: plot_ml

annotate plot_ml_performance.py @ 2:62e3a4e8c54c draft

"planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 3752bb86fb42f0c98b6d6cbbd75e8d987f17a3fd"

author	bgruening
date	Thu, 16 Jan 2020 13:49:49 -0500
parents	85da91bbdbfb
children	1c5dcef5ce0f

rev	line source
0 4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	1 import argparse
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	2 import pandas as pd
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	3 import plotly
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	4 import pickle
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	5 import plotly.graph_objs as go
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	6 from sklearn.metrics import confusion_matrix, precision_recall_fscore_support, roc_curve, auc
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	7 from sklearn.preprocessing import label_binarize
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	8
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	9
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	10 def main(infile_input, infile_output, infile_trained_model):
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	11 """
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	12 Produce an interactive confusion matrix (heatmap), precision, recall, fscore and auc plots
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	13 Args:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	14 infile_input: str, input tabular file with true labels
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	15 infile_output: str, input tabular file with predicted labels
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	16 infile_trained_model: str, input trained model file (zip)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	17 """
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	18
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	19 df_input = pd.read_csv(infile_input, sep='\t', parse_dates=True)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	20 df_output = pd.read_csv(infile_output, sep='\t', parse_dates=True)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	21 true_labels = df_input.iloc[:, -1].copy()
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	22 predicted_labels = df_output.iloc[:, -1].copy()
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	23 axis_labels = list(set(true_labels))
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	24 c_matrix = confusion_matrix(true_labels, predicted_labels)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	25 data = [
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	26 go.Heatmap(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	27 z=c_matrix,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	28 x=axis_labels,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	29 y=axis_labels,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	30 colorscale='Portland',
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	31 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	32 ]
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	33
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	34 layout = go.Layout(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	35 title='Confusion Matrix between true and predicted class labels',
1 85da91bbdbfb planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit bd5f722e867bcf00eba4b08a87fcce285486fe5b bgruening parents: 0 diff changeset	36 xaxis=dict(title='Predicted class labels'),
85da91bbdbfb planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit bd5f722e867bcf00eba4b08a87fcce285486fe5b bgruening parents: 0 diff changeset	37 yaxis=dict(title='True class labels')
0 4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	38 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	39
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	40 fig = go.Figure(data=data, layout=layout)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	41 plotly.offline.plot(fig, filename="output_confusion.html", auto_open=False)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	42
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	43 # plot precision, recall and f_score for each class label
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	44 precision, recall, f_score, _ = precision_recall_fscore_support(true_labels, predicted_labels)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	45
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	46 trace_precision = go.Scatter(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	47 x=axis_labels,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	48 y=precision,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	49 mode='lines+markers',
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	50 name='Precision'
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	51 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	52
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	53 trace_recall = go.Scatter(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	54 x=axis_labels,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	55 y=recall,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	56 mode='lines+markers',
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	57 name='Recall'
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	58 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	59
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	60 trace_fscore = go.Scatter(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	61 x=axis_labels,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	62 y=f_score,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	63 mode='lines+markers',
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	64 name='F-score'
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	65 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	66
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	67 layout_prf = go.Layout(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	68 title='Precision, recall and f-score of true and predicted class labels',
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	69 xaxis=dict(title='Class labels'),
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	70 yaxis=dict(title='Precision, recall and f-score')
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	71 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	72
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	73 data_prf = [trace_precision, trace_recall, trace_fscore]
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	74 fig_prf = go.Figure(data=data_prf, layout=layout_prf)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	75 plotly.offline.plot(fig_prf, filename="output_prf.html", auto_open=False)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	76
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	77 # plot roc and auc curves for different classes
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	78 with open(infile_trained_model, 'rb') as model_file:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	79 model = pickle.load(model_file)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	80
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	81 # remove the last column (label column)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	82 test_data = df_input.iloc[:, :-1]
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	83 model_items = dir(model)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	84
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	85 try:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	86 # find the probability estimating method
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	87 if 'predict_proba' in model_items:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	88 y_score = model.predict_proba(test_data)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	89 elif 'decision_function' in model_items:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	90 y_score = model.decision_function(test_data)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	91
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	92 true_labels_list = true_labels.tolist()
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	93 one_hot_labels = label_binarize(true_labels_list, classes=axis_labels)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	94 data_roc = list()
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	95
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	96 if len(axis_labels) > 2:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	97 fpr = dict()
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	98 tpr = dict()
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	99 roc_auc = dict()
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	100 for i in axis_labels:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	101 fpr[i], tpr[i], _ = roc_curve(one_hot_labels[:, i], y_score[:, i])
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	102 roc_auc[i] = auc(fpr[i], tpr[i])
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	103 for i in range(len(axis_labels)):
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	104 trace = go.Scatter(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	105 x=fpr[i],
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	106 y=tpr[i],
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	107 mode='lines+markers',
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	108 name='ROC curve of class {0} (AUC = {1:0.2f})'.format(i, roc_auc[i])
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	109 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	110 data_roc.append(trace)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	111 else:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	112 try:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	113 y_score_binary = y_score[:, 1]
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	114 except:
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	115 y_score_binary = y_score
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	116 fpr, tpr, _ = roc_curve(one_hot_labels, y_score_binary, pos_label=1)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	117 roc_auc = auc(fpr, tpr)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	118 trace = go.Scatter(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	119 x=fpr,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	120 y=tpr,
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	121 mode='lines+markers',
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	122 name='ROC curve (AUC = {0:0.2f})'.format(roc_auc)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	123 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	124 data_roc.append(trace)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	125
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	126 trace_diag = go.Scatter(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	127 x=[0, 1],
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	128 y=[0, 1],
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	129 mode='lines',
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	130 name='Chance'
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	131 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	132 data_roc.append(trace_diag)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	133 layout_roc = go.Layout(
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	134 title='Receiver operating characteristics (ROC) and area under curve (AUC)',
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	135 xaxis=dict(title='False positive rate'),
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	136 yaxis=dict(title='True positive rate')
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	137 )
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	138
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	139 fig_roc = go.Figure(data=data_roc, layout=layout_roc)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	140 plotly.offline.plot(fig_roc, filename="output_roc.html", auto_open=False)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	141
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	142 except Exception as exp:
2 62e3a4e8c54c "planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 3752bb86fb42f0c98b6d6cbbd75e8d987f17a3fd" bgruening parents: 1 diff changeset	143 print("Plotting the ROC-AUC graph failed. This exception was raised: {}".format(exp))
0 4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	144
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	145
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	146 if __name__ == "__main__":
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	147 aparser = argparse.ArgumentParser()
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	148 aparser.add_argument("-i", "--input", dest="infile_input", required=True)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	149 aparser.add_argument("-j", "--output", dest="infile_output", required=True)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	150 aparser.add_argument("-k", "--model", dest="infile_trained_model", required=True)
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	151 args = aparser.parse_args()
4fac53da862f planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/plotly_ml_performance_plots commit 8496ba724e35ba551172ea975b0fed091d4bbe88 bgruening parents: diff changeset	152 main(args.infile_input, args.infile_output, args.infile_trained_model)

Mercurial > repos > bgruening > plotly_ml_performance_plots

annotate plot_ml_performance.py @ 2:62e3a4e8c54c draft