Effortlessly tune LGBM with optuna

Perquisites:

• LGBM == lightgbm (python package): Microsoft’s implementation of gradient boosted machines
• optuna (python package): automated hyperparameter optimization framework favoured by Kaggle grandmasters. Being algorithm agnostic it can help find optimal hyperparameters for any model. Apart from gridsearch, it features tools for pruning the unpromising trails for faster results

So what’s the catch?

Complete model optimization includes many different operations:

• Choosing the optimal starting hyperparameters for your algorithm (conditional on the task type and data stats)
• Defining the hyperparameters to optimize, their grid and distribution
• Selecting the optimal loss function for optimization
• Configuring the validation strategy
• Further optimization (e.g. n_estimators tuning with early_stopping for tree ensembles like LGBM)
• Results analysis
• and much more…

Putting it all together even for a single task requires a lot of code and any subsequent tasks will require substantial modifications to this code

There is a reason this article includes two important keywords:

• LGBM — fastest gradient boosting framework
• optuna — fastest hyperparameter optimization framework