Tutorials: Feature Engineering in Python

Andrew Ng stated, “applied ML is basically just feature engineering.” In data science and ML, the most important, but oftentimes most overlooked, piece of the puzzle is feature engineering.

At Rasgo, we are data scientists on the mission to enable the global data science community to generate valuable and trusted insights from data in under 5 minutes. As we have marched forward on this mission, we’ve grown incredibly frustrated in the lack of helpful content and python functions that target feature engineering. We wrestle with these problems everyday and we wanted to provide a repository of recipes that showcase how to use the best tools available in this space. Additionally, we’ve built our own SDK (PyRasgo) for feature engineering that enables users to automatically track, visualize, and evaluate their feature engineering experiments to make more accurate and explainable feature engineering decisions.

In that vein, this repository contains tutorials and code to enable data scientists to easily create new ML features and evaluate their importance for supervised machine learning. We sincerely hope this is helpful and please leave comments with any questions on what we can do to improve!

Please join us on the

• Rasgo Forum for questions about these recipies and PyRasgo.
• Rasgo User Group Slack to join our community.
• Video Tutorials on YouTube (Coming Soon)

Table of Contents

• Feature Profiling

* pandas-profiling: * SweetViz:

• Data Cleaning

* Missing Data * pandas: * Duplicate Data * pandas: * Data Type Mismatch * pandas: * Date Gaps in Time Series * pandas:

• Feature Transformation

* Time-series * Lag: * Moving Average: * Weekly Resampled Aggregation: * Weekly Rolling Aggregation: * Velocity and Acceleration: * Energy: * Mean Difference: * Mean Absolute Difference: * tsfresh: * Categorical * One-hot encoding: * Target encoding: * Leave One Out encoding: * Numerical * Standard scaler: * Min-Max scaler: * Robust scaler:

• Model Selection

* Train-Test Split * Time Series Split * Scikit-learn: * Train-Test Split: * K-Fold or Cross-Validation * Random: * Stratified: * Group: * Model Comparison * PyCaret: * Model Training * Catboost * Classification: * Regression: * Model Metrics * Binary Classification * AUC: * Log Loss: * Regression * MAE: * MAPE: * RMSE: * R^2:

• Feature Importance

* Scikit-learn: * XGBoost: * catboost:

• Feature Selection

* Model Agnostic * Low Variance: * Univariate Feature Selection * F-test: * Mutual Information: * Model Based * Lasso-based Selection (Coming soon) * Feature Importance * Scikit-learn Tree-based: * Permutation Importance: * SHAP Values: * Sequential Feature Selection * Forward Stepwise Selection (Coming soon) * Backwards Stepwise Selection * Scikit-learn Tree-based: * catboost: