PyCaret Model Training Report

Setup & Best Model
Best Model Plots
Feature Importance
Explainer

Setup Parameters

ParameterValue
target PCR
session_id 42
index False
train_size 0.8
normalize True
feature_selection True
fold 5
remove_outliers True
remove_multicollinearity True
If you want to know all the experiment setup parameters, please check the PyCaret documentation for the classification/regression exp function.

Best Model: LGBMClassifier

ParameterValue
boosting_type gbdt
class_weight None
colsample_bytree 1.0
importance_type split
learning_rate 0.1
max_depth -1
min_child_samples 20
min_child_weight 0.001
min_split_gain 0.0
n_estimators 100
n_jobs -1
num_leaves 31
objective None
random_state 42
reg_alpha 0.0
reg_lambda 0.0
subsample 1.0
subsample_for_bin 200000
subsample_freq 0

Comparison Results on the Cross-Validation Set

Model Accuracy ROC-AUC Recall Prec. F1 Kappa MCC PR-AUC-Weighted TT (Sec)
Light Gradient Boosting Machine 0.7091 0.6267 0.64 0.6895 0.6467 0.4056 0.4224 0.5918 0.322
Naive Bayes 0.6545 0.6800 0.72 0.6117 0.6498 0.3163 0.3232 0.6930 1.240
K Neighbors Classifier 0.6364 0.6467 0.56 0.6067 0.5743 0.2603 0.2660 0.6001 0.864
Ridge Classifier 0.6364 0.6467 0.64 0.5962 0.6048 0.2700 0.2835 0.0000 0.898
Random Forest Classifier 0.6364 0.6300 0.60 0.6343 0.6013 0.2688 0.2834 0.6539 0.906
Logistic Regression 0.6364 0.6400 0.64 0.5962 0.6048 0.2700 0.2835 0.6697 0.798
Quadratic Discriminant Analysis 0.6364 0.6933 0.72 0.5851 0.6353 0.2815 0.2899 0.7075 0.418
Linear Discriminant Analysis 0.6364 0.6467 0.64 0.5962 0.6048 0.2700 0.2835 0.6751 0.364
Gradient Boosting Classifier 0.6182 0.6333 0.60 0.5843 0.5846 0.2328 0.2389 0.6403 0.522
Ada Boost Classifier 0.6182 0.6567 0.60 0.5943 0.5891 0.2340 0.2415 0.6517 0.560
Extra Trees Classifier 0.6182 0.5800 0.56 0.5876 0.5622 0.2266 0.2347 0.6413 0.468
Decision Tree Classifier 0.6000 0.5967 0.56 0.5867 0.5533 0.1950 0.2060 0.5215 1.532
CatBoost Classifier 0.5818 0.6667 0.48 0.5133 0.4845 0.1454 0.1414 0.6991 3.426
SVM - Linear Kernel 0.5455 0.5000 0.40 0.5033 0.4332 0.0684 0.0685 0.0000 1.666
Dummy Classifier 0.5455 0.5000 0.00 0.0000 0.0000 0.0000 0.0000 0.4545 0.456
Extreme Gradient Boosting 0.5273 0.5600 0.52 0.4967 0.5042 0.0550 0.0564 0.5943 0.336

Results on the Test Set for the best model

Model Accuracy ROC-AUC Recall Prec. F1 Kappa MCC PR-AUC-Weighted
Light Gradient Boosting Machine 0.7857 0.7604 0.6667 0.8 0.7273 0.5532 0.5594 0.7502

Best Model Plots on the testing set

Confusion_matrix

confusion_matrix

Auc

auc

Threshold

threshold

Pr

pr

Error

error

Class_report

class_report

Learning

learning

Calibration

calibration

Vc

vc

Dimension

dimension

Manifold

manifold

Rfe

rfe

Feature

feature

Feature_all

feature_all

PyCaret Feature Importance Report

Feature importance analysis from atrained Random Forest

Use gini impurity forcalculating feature importance for classificationand Variance Reduction for regression

tree_importance

SHAP Summary from a trained lightgbm

shap_summary