Stacking 101. All you need to know

image by author

This series is comprised of 2 articles which explain a fully manual approach and an automated open-source stacking tool that does it all for you

Part I— Stacking tips and tricks + code from scratch

In Part II I introduce a fully automated stacking framework that encapsulates everything necessary to create an ultimate multilayer stacking ensemble with a few one-liners

Where exactly can stacking be used in a machine learning project?

Let’s break down a machine learning into it’s essential steps:

1. Preprocess data — transform all variables in a way, that they could at least be used for training a machine learning algorithm (become numeric) and at most they would (apart from just being numeric) represent the upmost value in predicting the target variable
2. Create/extract/engineer new features:

• Can be done manually if one has domain knowledge and features are interpretable (E.g. divide area by price)
• Can be done blindly (recursively) combining anonymised important features and validating
• Can be done by stacking different models predictions of the target variable as features into the train/test sets