Competency Levels
This page breaks down each of the three competencies assessed in this course into critical learnings, and describes three levels of mastery for each of them. The Competent level is the target to reach by the end of Session 3.
The levels are cumulative: reaching Intermediate assumes the Novice descriptors are already acquired, and Competent assumes both. Each learning objective stated at the top of an activity page contributes to one of the critical learnings below; the “Evidenced in” line tells you which activities work on each competency.
| Level | Meaning |
|---|---|
| Novice | You apply a practice when it is guided and the expected result is given to you. |
| Intermediate | You apply the practice on your own and can explain why it matters. |
| Competent (target) | You adapt the practice to a new situation, anticipate its limits, and justify your choices. |
How to use this grid
- Before an activity, check which critical learning its objectives target, and read the descriptor of the level just above yours.
- In your reflections, back each claim with a concrete element from your own work (a commit, a test, a per-fold number, a chart): the grid describes what must be visible in your repository, not what you know.
- At the final assessment, your level on each critical learning is judged from your Git repository as a whole. Reaching Competent everywhere is the goal; a well-justified Intermediate on a module you deliberately skipped is better than an unexplained gap.
BC02: Data-Driven Communication
Effectively communicate insights derived from data through visualization and statistical analysis.
| Critical learning | Novice | Intermediate | Competent (target) |
|---|---|---|---|
| Form hypotheses from data | • Describe each variable’s type, measurement scale and analytical role. • Observe distributions and missing values. |
• Formulate justified hypotheses about which predictors are informative. • Identify concrete risks when generalizing from one city to another. |
• Revise hypotheses when the scope of the data changes (e.g. from 2 to 4 cities). • Judge whether the evidence (sample size n) is sufficient, and propose what to investigate next. |
| Quantify results with their uncertainty | • Report a result as metrics computed on a clearly identified test set. | • Report dispersion (standard deviation across folds), significance and sample size alongside the mean. | • Explain the limits of a number: an outlier fold, a p-value made meaningless by a large sample, a single split that cannot be trusted. |
| Design an effective visual | • Choose a chart type suited to the message. • Avoid pie charts, 3D, dual axes and truncated baselines. |
• Remove clutter using Gestalt principles (direct labels instead of legends, white space, alignment). | • Direct attention with preattentive attributes and a colorblind-safe palette. • Produce a chart whose message is the first thing seen (“close your eyes” test). |
| Turn a result into a decision | • Distinguish exploratory from explanatory analysis. • Identify the audience, the expected action and the supporting data (Who / What / How). |
• State a Big Idea as one actionable sentence naming a decision-maker. | • Translate technical results into a recommendation with its risks and limits (e.g. whether a model trained on one city can be deployed in another). • Do so consistently in your reflections and your README. |
Evidenced in: Session 2 (Understand the Air Quality Problem, Build a First Pipeline); Session 3 (Correlation Analysis, Robust Evaluation, A Scikit-learn Transformer, Data Visualization).
BC04: Software Engineering Best Practices
Apply best practices in software development to data science workflows.
| Critical learning | Novice | Intermediate | Competent (target) |
|---|---|---|---|
| Design modular, reusable code | • Extract repeated notebook logic into single-responsibility functions. • Pass everything a function needs as parameters, with no hidden global state. |
• Group functions into modules by responsibility, with docstrings and a configuration object (dataclass).• Recognize when a piece of logic should not be extracted. |
• Build components that follow an external contract (scikit-learn fit/transform) so they compose with third-party tools.• Keep the orchestration layer ( workflows.py) readable as a map of the whole pipeline. |
| Develop test-first | • Run a provided test suite. • Read a failing test as feedback on which behavior is still missing. |
• Read the test before writing the function it specifies. • Implement module by module until each test passes. |
• Use tests to harden a component against a real edge case (e.g. a city with no data for a column). • Use tests to compare two competing implementations fairly. |
| Version and collaborate with Git | • Commit regularly from your own account. • Write descriptive commit messages. |
• Make atomic commits following Conventional Commits (refactor, fix, test), one per module.• Keep contributions balanced within the pair. |
• Produce a history that tells the project’s story: which modules were completed, in which order, and why some were skipped. • Document your work well enough that your partner could take over any part of it. |
| Organize the project for reliable execution | • Install the environment with uv sync.• Run code from the project root. |
• Keep src/, tests/, scripts/ and notebooks/ clearly separated.• Verify the pipeline through a script, never through the notebook. |
• Handle optional dependencies (dependency groups, lazy imports). • Anchor paths to the source file rather than the working directory, so the code behaves the same wherever it is run. |
Evidenced in: Session 2 (Transfer to Python); Session 3 (Advanced Cleaning, Robust Evaluation, Feature Selection, Hyperparameter Tuning, Experiment Tracking, A Scikit-learn Transformer, Model Registry).
BC07: End-to-End ML Pipeline Design
Design and implement end-to-end data science projects using industry-standard tools and libraries.
| Critical learning | Novice | Intermediate | Competent (target) |
|---|---|---|---|
| Build an end-to-end pipeline | • Chain loading, cleaning, splitting, feature engineering, training and evaluation in a notebook. • Explain why these steps have to happen in that order. |
• Package the pipeline as a configurable workflow (run_baseline, run_advanced).• Run it outside any notebook, from a script. |
• Chain preparation, feature selection and model into a single scikit-learn Pipeline evaluated per fold.• Keep extending the pipeline as the project evolves, without breaking what already works. |
| Prepare data without leakage | • Fill missing values within each city, never across cities. • Exclude features that destabilize the model (e.g. raw site coordinates). |
• Adapt cleaning to imbalanced data (drop high-missing columns before filling). • Handle missing values pairwise, not globally, when comparing many variables. |
• Identify every step that learns from data (missingness threshold, feature selection) and guarantee it only learns from the training fold. • Explain why the evaluation would otherwise be overly optimistic. |
| Evaluate and improve a model rigorously | • Compute RMSE, MAE and R² on a held-out city. • Compare both directions of a train/test pair. |
• Generalize evaluation with GroupKFold, reading per-fold results as well as mean and standard deviation.• Test the statistical significance of a relationship before trusting it. |
• Select features and tune hyperparameters systematically (RFE vs. SelectKBest, GridSearchCV with GroupKFold).• Justify whether a more complex model is worth it compared to a simpler baseline. |
| Track, version and manage models | • Distinguish an experiment’s parameters (inputs) from its metrics (results). • Record both for every experiment you run. |
• Log every run with MLflow. • Compare runs through queries and the MLflow UI rather than terminal history. |
• Register several model versions and compare candidates on the same evidence before promoting one. • Promote through an alias so calling code never hardcodes a version, and explain what changes at team scale. |
Evidenced in: Session 2 (Build a First Pipeline, Transfer to Python); Session 3 (Correlation Analysis, Advanced Cleaning, Robust Evaluation, Feature Selection, Hyperparameter Tuning, Experiment Tracking, A Scikit-learn Transformer, Model Registry).