2. Production-Ready NLP Pipeline

Duration2h30 x 2 AI Allowed

Introduction

This practical session challenges you to transform your notebook-based spam detection analysis into a production-ready NLP pipeline. Building on the air quality project architecture, you’ll create a structured, testable, and maintainable text classification system while exploring advanced preprocessing techniques and model optimization strategies.

Unlike the previous guided notebook experience, this mini-project emphasizes independent problem-solving and professional software development practices. You’ll apply the architectural principles learned in the air quality sessions to the NLP domain, creating a comprehensive solution that demonstrates your ability to build enterprise-grade machine learning systems.

Learning Objectives

By completing this mini-project, you will demonstrate mastery of:

Software Architecture: Design modular, reusable components following separation of concerns principles
MLOps Integration: Implement systematic experiment tracking and model management with MLflow
Professional Documentation: Create comprehensive README files that communicate technical and business value
Version Control: Apply atomic, well-documented Git commits throughout the development process
NLP Engineering: Build production-ready text processing pipelines with optimal preprocessing configurations

This project serves as a portfolio piece demonstrating your ability to transition from experimental data science to production-ready ML engineering.

Prerequisites

This session assume you have completed and submitted Air Quality project.

Problem Statement

You have successfully explored spam detection fundamentals through notebook experimentation. Now comes the critical transition: How do you transform experimental NLP code into a production-ready, maintainable pipeline?

Your mission is to architect a robust text classification system that demonstrates professional software engineering practices while optimizing text preprocessing for maximum performance. The core challenge lies in balancing text data characteristics—high dimensionality, domain-specific vocabulary, and class imbalance—with clean, testable code architecture.

Unlike structured data, text processing involves interdependent decisions: tokenization patterns affect feature extraction, which influences model performance across different communication domains. You’ll systematically experiment with preprocessing strategies (tokenization, normalization, stop words, vocabulary sizing) and implement cross-domain analysis to understand how SMS-trained models perform on email data—a critical real-world scenario.

Your pipeline will enable data-driven preprocessing decisions, supported by comprehensive MLflow tracking to capture the complex parameter space of NLP systems.

Assessment

✅ Modular pipeline architecture with separated DataProcessor, TextPreprocessor, ModelTrainer, and Evaluator classes
✅ Cross-domain experimentation analyzing SMS, email, and combined dataset performance with transfer learning evaluation
✅ Advanced text preprocessing optimization including configurable tokenization patterns, case normalization, stop word handling, and vocabulary sizing
✅ Systematic hyperparameter exploration across preprocessing configurations and model parameters with statistical significance testing
✅ Comprehensive MLflow integration tracking text-specific parameters, cross-domain metrics, and preprocessing pipeline artifacts
✅ Professional documentation with clear README explaining NLP-specific design decisions and business impact of preprocessing choices

Tips for technical exploration

While building your structured pipeline, investigate these advanced text processing techniques to optimize performance:

Tokenization Strategy Optimization

Experiment with different token patterns

Modify the TOKEN_REGEX variable to test alternative tokenization approaches:

Option 1: Word characters only, minimum 2 character-
Option 2: Keep punctuation patterns -
Option 3: Include single characters

Questions to explore:

How does including/excluding punctuation affect spam detection?
Do minimum token length requirements improve or hurt performance?
Which pattern works best for each dataset?

Case Normalization Analysis

Test lowercase conversion impact

Modify the CountVectorizer to use lowercase=True and compare results:

1
2
3
4


vectorizer = CountVectorizer(max_features=NB_FEATURES, 
                           token_pattern=TOKEN_REGEX, 
                           lowercase=True,  # Changed from False
                           stop_words=None)

Questions to explore:

Does case normalization improve generalization?
Are there spam indicators that depend on capitalization?
How does this choice affect different domains (SMS vs email)?

Stop Word Removal Evaluation

Analyze stop word impact

Test stop word removal by changing the CountVectorizer configuration:

1
2
3
4


vectorizer = CountVectorizer(max_features=NB_FEATURES, 
                           token_pattern=TOKEN_REGEX, 
                           lowercase=False, 
                           stop_words='english')  # Changed from None

Questions to explore:

Do stop words provide useful information for spam detection?
How does stop word removal affect model interpretability?
Are there domain-specific differences in stop word utility?

Feature Dimensionality Optimization

Optimize vocabulary size

Test different values for NB_FEATURES to find the optimal vocabulary size:

1
2
3


# Test different vocabulary sizes
for nb_features in [1000, 2500, 5000, 10000, 15000]:
    # Retrain and evaluate model with each setting

Questions to explore:

What is the optimal trade-off between vocabulary size and performance?
How does optimal vocabulary size differ across domains?
At what point does additional vocabulary provide diminishing returns?

Tips for experiment tracking

Integrate comprehensive experiment tracking throughout your pipeline (these are suggested parameters you are free to adapt):

Parameters to Track:

Text preprocessing configuration (tokenization patterns, feature counts, normalization settings)
Model hyperparameters (regularization, solver type, iteration limits)
Dataset configurations (SMS, email, combined, cross-domain scenarios)
Evaluation strategies (train-test splits, cross-validation settings)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


# Example parameter logging structure
mlflow.log_params({
    'dataset_combination': 'sms_only',
    'tokenization_pattern': TOKEN_REGEX,
    'max_features': NB_FEATURES,
    'lowercase_conversion': False,
    'stop_words_removal': None,
    'model_type': 'logistic_regression',
    'model_max_iter': 1000,
    'train_test_split': 0.2,
    'class_balancing': True
})

Metrics to Log:

Performance metrics (accuracy, precision, recall, F1-score)
Cross-validation statistics (mean and standard deviation across folds)
Confusion matrix elements (true/false positives and negatives)
Training metadata (execution time, feature dimensionality, dataset sizes)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


# Required metrics logging
mlflow.log_metrics({
    'accuracy': accuracy_score,
    'precision': precision_score, 
    'recall': recall_score,
    'f1_score': f1_score,
    'confusion_matrix_tp': tp,
    'confusion_matrix_tn': tn,
    'confusion_matrix_fp': fp,
    'confusion_matrix_fn': fn,
    'training_time_seconds': training_duration,
    'feature_dimensionality': feature_count
})

Artifacts to Store:

Trained model instances with vectorizer configurations
Feature importance rankings and vocabulary analyses
Performance visualization plots and confusion matrices
Preprocessing pipeline configurations and parameters

1
2
3
4
5
6
7
8


# Model registration and artifact storage
mlflow.sklearn.log_model(
    model, 
    "spam_classifier",
    registered_model_name=f"SpamDetector_{dataset_type}"
)
mlflow.log_artifact("confusion_matrix.png")
mlflow.log_artifact("feature_importance.csv")

Submission process

Commit all pending files in your repository.
In the project folder root, run uv run python scripts/run_tests.py.
Add and commit the two new files test_proof.json and test_results.json without modifying them.
Publish on moodle the link toward your repository (https version). Please check your repository allow public access.