Chapter 5

Deepen the Pipeline

Presentation & objectives

This session lets you go deeper on specific aspects of the pipeline you built in Session 2 — on the full dataset (all four cities) rather than the two-city subsample used there. It is a learning path: 9 modules, each closing a real gap Session 2 left open or introducing a genuinely new tool. Pick your own order based on what you already know — just check each module’s Prerequisites first (and the diagram at the bottom of this page), since a few modules require another one to be done first.

Choose Your Learning Path

9 modules, no fixed order — pick the sequence that fits what you already know. Check each module’s Prerequisites before deciding where to start: the diagram below shows which modules require another one first.

graph TD
  m1["1 - Correlation Analysis"]
  m2["2 - Advanced Cleaning"] --> m3["3 - Robust Evaluation"]
  m3 --> m4["4 - Feature Selection"]
  m4 --> m5["5 - Hyperparameter Tuning"]
  m5 --> m6["6 - Experiment Tracking"]
  m4 --> m7["7 - Sklearn Transformer"]
  m8["8 - Data Visualization"]
  m6 --> m9["9 - Model Registry"]
  m7 --> m9

  classDef available stroke-width:2px;

  class m1,m2,m3,m4,m5,m6,m7,m8,m9 available

  click m1 href "practical_1"
  click m2 href "practical_2"
  click m3 href "practical_3"
  click m4 href "practical_4"
  click m5 href "practical_5"
  click m6 href "practical_6"
  click m7 href "practical_7"
  click m8 href "practical_8"
  click m9 href "practical_9"
  1. Correlation Analysis — Tell a real relationship in the data from one that only looks convincing, so your heatmap can actually be trusted.
  2. Advanced Cleaning — Handle cities with wildly different amounts of data, where a cleaning approach that works on one can silently fail on another.
  3. Robust Evaluation — Judge your model against every city in turn, instead of trusting whatever a single lucky (or unlucky) split happens to show.
  4. Feature Selection — Learn when adding more input columns makes a model worse, and how to pick only the ones that actually help.
  5. Hyperparameter Tuning — Search systematically for the settings that make a more powerful model worth using, instead of guessing.
  6. Experiment Tracking — Keep a record of every experiment you run, so results stay comparable and your best one is easy to reproduce later.
  7. A Scikit-learn Transformer — Turn data-preparation steps into reusable building blocks you can chain together in a real pipeline, trained and applied automatically like any other step.
  8. Data Visualization — Turn a result into a figure a decision-maker outside the project can actually act on, instead of a table only you can read.
  9. Model Registry — Give calling code a name for “the model to use” that you can repoint to a better version, without changing that code.