Icon

kn_​forum_​72596_​binary_​classification_​lightgbm

KNIME forum - binary machine-learning task (72596)

KNIME forum - binary machine-learning task (72596)
https://forum.knime.com/t/outcome-of-a-neural-network-is-too-good-where-did-i-go-wrong/72596/7?u=mlauber71

# conda env create -f="/Users/m_lauber/Dropbox/knime-workspace/Machine_Learning/ml_binary/kn_example_ml_binary_lightgbm_hyper_parameter_opt/data/py3_knime_lightgbm.yml"# conda env create -f="C:\\Users\\x123456\\knime-workspace\\Machine_learning\\ml_binary\\kn_example_ml_binary_lightgbm_hyper_parameter_opt\\data\\py3_knime_lightgbm.yml"# conda activate py3_knime_lightgbm# conda update -n py3_knime_lightgbm --all# conda env update --name py3_knime_lightgbm --file "/Users/m_lauber/Dropbox/knime-workspace/Machine_Learning/ml_binary/kn_example_ml_binary_lightgbm_hyper_parameter_opt/data/py3_knime_lightgbm.yml" --prune# conda env update --name py3_knime_lightgbm --file "C:\\Users\\x123456\\knime-workspace\\Machine_learning\\ml_binary\\kn_example_ml_binary_lightgbm_hyper_parameter_opt\\data\\py3_knime_lightgbm.yml" --prune# conda env update --name py3_knime_lightgbm --file "/Users/m_lauber/Dropbox/knime-workspace/Machine_Learning/ml_binary/kn_example_ml_binary_lightgbm_hyper_parameter_opt/data/py3_knime_lightgbm.yml"# conda env update --name py3_knime_lightgbm --file "C:\\Users\\x123456\\knime-workspace\\Machine_learning\\ml_binary\\kn_example_ml_binary_lightgbm_hyper_parameter_opt\\data\\py3_knime_lightgbm.yml"# conda update -n base conda# KNIME official Python integration guide# https://docs.knime.com/latest/python_installation_guide/index.html#_introduction# KNIME and Python - Setting up and managing Conda environments# https://medium.com/p/2ac217792539# Hyperparameter optimization for LightGBM - wrapped in KNIME nodes# https://medium.com/p/ddb7ae1d7e2# conda activate py3_knime_lightgbm# file: py3_knime_lightgbm.yml with some modifications# THX Carsten Haubold (https://hub.knime.com/carstenhaubold) for hintsname: py3_knime_lightgbm # Name of the created environmentchannels: # Repositories to search for packages# - defaults # edit: removed to just use conda-forge# - anaconda # edit: removed to just use conda-forge - conda-forge# https://anaconda.org/knime - knime # conda search knime-python-base -c knime --info # to see what is in the packagedependencies: # List of packages that should be installed- python=3.9 # Python- knime-python-base # dependencies of KNIME - Python integration# - knime-python-scripting # everything you need to also build Python packages for KNIME- cairo # SVG support- pillow # Image inputs/outputs- matplotlib # Plotting- IPython # Notebook support- nbformat # Notebook support- scipy # Notebook support- jpype1 # A Python to Java bridge# Jupyter Notebook support- jupyter # Jupyter Notebook- pandas-profiling # create overview of your data- sweetviz # In-depth EDA (target analysis, comparison, feature analysis, correlation) in two lines of code!- plotly # An interactive, browser-based graphing library for Python- python-kaleido # Fast static image export for web-based visualization libraries# Machine Learning Modules- lightgbm- xgboost- hyperopt- scikit-optimize # skopt- optuna # A hyperparameter optimization framework- pip # Python installer- pip:# - JPype1 # Databases - vtreat # https://medium.com/low-code-for-advanced-data-science/data-preparation-for-machine-learning-with-knime-and-the-python-vtreat-package-efcaf58fa783 - h2o>=3.38 - boruta # Python Implementation of Boruta Feature Selection MEDIUM Blog: Hyperparameter optimization for LightGBM — wrapped in KNIME nodeshttps://medium.com/p/ddb7ae1d7e2GitHub Repositoryhttps://github.com/ml-score/knime_meets_python/tree/main/machine_learning/binary KNIME forum - binary machine-learning task (72596)https://forum.knime.com/t/outcome-of-a-neural-network-is-too-good-where-did-i-go-wrong/72596/7?u=mlauber71 locate and create/data/ folderwith absolute pathsmodel_results.xlsxtrain.parquetTarget is target"row_id" will not be usedhttps://medium.com/p/efcaf58fa783test.parquetPy_vtreat_LightGBMPy_vtreat_LightGBMPy_vtreat_LightGBMremoverow_idTestPy_LightGBMPy_LightGBMPy_LightGBMPy_LightGBMroc_aucPr (AUC)DESCENDINGpy3_knime_lightgbmyaml in node description!Apple Siliconlightgbm_feature_importance.parquetlightgbm_vtreat_feature_importance.parquetpy3_knime_lightgbmyaml in node description!Windowslightgbm_model_parameters.txtPy_vtreat_LightGBMroc_auclightgbm_vtreat_model_parameters.txtNode 3868Node 3869 Collect LocalMetadata Excel Writer Parquet Reader vtreat preparebinary data Merge Variables Parquet Reader Table to H2O H2O Binomial Scorer H2O Local Context ConstantValue Column Column Filter ReferenceColumn Filter Column Filter ConstantValue Column Table to H2O H2O Binomial Scorer Py_LightGBM Column Filter Concatenate RowID Sorter Conda EnvironmentPropagation Parquet Writer Parquet Writer Conda EnvironmentPropagation CSV Writer Py_vtreat_LightGBM CSV Writer Binary ClassificationInspector Binary ClassificationInspector Select Parametersfor Models Merge Variables Merge Variables # conda env create -f="/Users/m_lauber/Dropbox/knime-workspace/Machine_Learning/ml_binary/kn_example_ml_binary_lightgbm_hyper_parameter_opt/data/py3_knime_lightgbm.yml"# conda env create -f="C:\\Users\\x123456\\knime-workspace\\Machine_learning\\ml_binary\\kn_example_ml_binary_lightgbm_hyper_parameter_opt\\data\\py3_knime_lightgbm.yml"# conda activate py3_knime_lightgbm# conda update -n py3_knime_lightgbm --all# conda env update --name py3_knime_lightgbm --file "/Users/m_lauber/Dropbox/knime-workspace/Machine_Learning/ml_binary/kn_example_ml_binary_lightgbm_hyper_parameter_opt/data/py3_knime_lightgbm.yml" --prune# conda env update --name py3_knime_lightgbm --file "C:\\Users\\x123456\\knime-workspace\\Machine_learning\\ml_binary\\kn_example_ml_binary_lightgbm_hyper_parameter_opt\\data\\py3_knime_lightgbm.yml" --prune# conda env update --name py3_knime_lightgbm --file "/Users/m_lauber/Dropbox/knime-workspace/Machine_Learning/ml_binary/kn_example_ml_binary_lightgbm_hyper_parameter_opt/data/py3_knime_lightgbm.yml"# conda env update --name py3_knime_lightgbm --file "C:\\Users\\x123456\\knime-workspace\\Machine_learning\\ml_binary\\kn_example_ml_binary_lightgbm_hyper_parameter_opt\\data\\py3_knime_lightgbm.yml"# conda update -n base conda# KNIME official Python integration guide# https://docs.knime.com/latest/python_installation_guide/index.html#_introduction# KNIME and Python - Setting up and managing Conda environments# https://medium.com/p/2ac217792539# Hyperparameter optimization for LightGBM - wrapped in KNIME nodes# https://medium.com/p/ddb7ae1d7e2# conda activate py3_knime_lightgbm# file: py3_knime_lightgbm.yml with some modifications# THX Carsten Haubold (https://hub.knime.com/carstenhaubold) for hintsname: py3_knime_lightgbm # Name of the created environmentchannels: # Repositories to search for packages# - defaults # edit: removed to just use conda-forge# - anaconda # edit: removed to just use conda-forge - conda-forge# https://anaconda.org/knime - knime # conda search knime-python-base -c knime --info # to see what is in the packagedependencies: # List of packages that should be installed- python=3.9 # Python- knime-python-base # dependencies of KNIME - Python integration# - knime-python-scripting # everything you need to also build Python packages for KNIME- cairo # SVG support- pillow # Image inputs/outputs- matplotlib # Plotting- IPython # Notebook support- nbformat # Notebook support- scipy # Notebook support- jpype1 # A Python to Java bridge# Jupyter Notebook support- jupyter # Jupyter Notebook- pandas-profiling # create overview of your data- sweetviz # In-depth EDA (target analysis, comparison, feature analysis, correlation) in two lines of code!- plotly # An interactive, browser-based graphing library for Python- python-kaleido # Fast static image export for web-based visualization libraries# Machine Learning Modules- lightgbm- xgboost- hyperopt- scikit-optimize # skopt- optuna # A hyperparameter optimization framework- pip # Python installer- pip:# - JPype1 # Databases - vtreat # https://medium.com/low-code-for-advanced-data-science/data-preparation-for-machine-learning-with-knime-and-the-python-vtreat-package-efcaf58fa783 - h2o>=3.38 - boruta # Python Implementation of Boruta Feature Selection MEDIUM Blog: Hyperparameter optimization for LightGBM — wrapped in KNIME nodeshttps://medium.com/p/ddb7ae1d7e2GitHub Repositoryhttps://github.com/ml-score/knime_meets_python/tree/main/machine_learning/binary KNIME forum - binary machine-learning task (72596)https://forum.knime.com/t/outcome-of-a-neural-network-is-too-good-where-did-i-go-wrong/72596/7?u=mlauber71 locate and create/data/ folderwith absolute pathsmodel_results.xlsxtrain.parquetTarget is target"row_id" will not be usedhttps://medium.com/p/efcaf58fa783test.parquetPy_vtreat_LightGBMPy_vtreat_LightGBMPy_vtreat_LightGBMremoverow_idTestPy_LightGBMPy_LightGBMPy_LightGBMPy_LightGBMroc_aucPr (AUC)DESCENDINGpy3_knime_lightgbmyaml in node description!Apple Siliconlightgbm_feature_importance.parquetlightgbm_vtreat_feature_importance.parquetpy3_knime_lightgbmyaml in node description!Windowslightgbm_model_parameters.txtPy_vtreat_LightGBMroc_auclightgbm_vtreat_model_parameters.txtNode 3868Node 3869 Collect LocalMetadata Excel Writer Parquet Reader vtreat preparebinary data Merge Variables Parquet Reader Table to H2O H2O Binomial Scorer H2O Local Context ConstantValue Column Column Filter ReferenceColumn Filter Column Filter ConstantValue Column Table to H2O H2O Binomial Scorer Py_LightGBM Column Filter Concatenate RowID Sorter Conda EnvironmentPropagation Parquet Writer Parquet Writer Conda EnvironmentPropagation CSV Writer Py_vtreat_LightGBM CSV Writer Binary ClassificationInspector Binary ClassificationInspector Select Parametersfor Models Merge Variables Merge Variables

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: m_lauber
Created at: 2023-09-13
On NodePit since: 2023-09-15
Last update: 2024-04-27
Created with KNIME version: v4.7.8
Tags: lightgbmbinarymodelaucclassificationaucprvtreat

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!