Chapter 3
Practice on a Natural Language Processing use case
Presentation & objectives
This session focuses on applying machine learning techniques to Natural Language Processing (NLP) challenges through hands-on spam detection projects. Students will learn to transform unstructured text data into actionable machine learning solutions while mastering the transition from experimental notebooks to production-ready NLP pipelines.
Through practical experience with authentic SMS and email datasets, you will explore the unique challenges of text classification including tokenization strategies, feature extraction techniques, cross-domain adaptation, and the critical importance of proper evaluation in imbalanced text datasets.
Assessment Information
This session contributes to your final assessment through two complementary mini-projects. For detailed evaluation criteria and requirements, please refer to the Assessment Guidelines section.
This work must be completed in groups of up to 5 people. Each student must have their own Git account and make commits using their own account.
Learning Objectives
By the end of this session, students will be able to:
- Process unstructured text data using industry-standard NLP preprocessing techniques including tokenization, normalization, and feature extraction
- Build robust text classification models with proper handling of class imbalance and domain-specific linguistic patterns
- Evaluate text classification performance using appropriate metrics (precision, recall, F1-score) and understanding their business implications
- Handle cross-domain challenges by analyzing performance transfer between different text types (SMS vs email)
- Architect production-ready NLP pipelines following software engineering best practices and modular design principles
- Apply MLOps practices to NLP workflows with systematic experiment tracking and model management
- Document NLP projects professionally with clear communication of technical methodology and business value
Activities
- NLP Introduction (Homework)
- Text Classification Fundamentals
- Production-Ready NLP Pipeline
Info
Before attending the practical session Text Classification Fundamentals ML Code, you must read at home NLP Introduction.
Submission Guidelines
- Deadline: December 14, 11:59 p.m.
- Format: Link to the project’s GitLab repository
- Deliverable: Complete Git repository with commit history
Warning
Important: Projects without appropriate Git history or with unbalanced commits between team members will be penalized. The quality of collaborative work is as important as the technical quality of the code.
Knowledge Check
Answer these questions once you have finished this course module.
---
primary_color: steelblue
secondary_color: lightgray
text_color: black
shuffle_questions: false
shuffle_answers: true
---
# Why is tokenization strategy crucial for text classification performance?
1. [ ] It determines the computational complexity of the machine learning algorithm
2. [x] It defines how text is split into features, directly affecting what patterns the model can learn
3. [ ] It automatically handles class imbalance in text datasets
4. [ ] It eliminates the need for feature selection in NLP tasks
# What is the primary challenge when transferring spam detection models from SMS to email data?
1. [ ] Email messages are always longer than SMS messages
2. [ ] Different programming languages are required for each domain
3. [x] Linguistic patterns, vocabulary, and communication styles differ significantly between domains
4. [ ] Email data requires different machine learning algorithms
# In spam detection, why might high precision be more important than high recall in certain scenarios?
1. [ ] High precision reduces computational costs during model training
2. [x] High precision minimizes false positives, preventing important emails from being incorrectly flagged as spam
3. [ ] High precision automatically improves recall in text classification tasks
4. [ ] High precision eliminates the need for cross-validation in model evaluation
# What is the main advantage of using CountVectorizer over simple word counting for text feature extraction?
1. [ ] CountVectorizer automatically removes all punctuation from text
2. [ ] CountVectorizer only works with English language text
3. [x] CountVectorizer provides configurable tokenization, vocabulary limits, and preprocessing options
4. [ ] CountVectorizer eliminates the need for train-test splitting
# Why is class balancing particularly important in spam detection datasets?
1. [ ] Balanced datasets always achieve higher accuracy scores
2. [x] Imbalanced datasets can lead to models that achieve high accuracy by always predicting the majority class
3. [ ] Class balancing eliminates the need for cross-validation
4. [ ] Balanced datasets require fewer features for optimal performance
# What is the primary benefit of stop word removal in bag-of-words text classification?
1. [ ] Stop words always indicate spam content in messages
2. [ ] Removing stop words automatically improves model accuracy
3. [x] Stop words are frequently occurring but often uninformative for classification tasks
4. [ ] Stop word removal reduces the need for tokenization
# When building production NLP pipelines, why is modular architecture particularly important?
1. [ ] Modular code automatically achieves better text classification performance
2. [x] Different text preprocessing strategies can be easily tested and optimized independently
3. [ ] Modular architecture eliminates the need for experiment tracking
4. [ ] Modular design automatically handles multiple languages
# What is the key difference between accuracy and F1-score in text classification evaluation?
1. [ ] F1-score is always higher than accuracy in spam detection tasks
2. [ ] Accuracy considers only true positives while F1-score considers all predictions
3. [x] F1-score balances precision and recall, while accuracy treats all classes equally
4. [ ] F1-score is only used for multi-class classification problems
# Why might a model trained on combined SMS and email data perform differently than domain-specific models?
1. [ ] Combined models always achieve lower performance than single-domain models
2. [x] Mixed training may improve robustness but could dilute domain-specific patterns
3. [ ] Combined datasets require different machine learning algorithms
4. [ ] Mixed training automatically eliminates the need for feature selection
# In MLflow tracking for NLP experiments, what is the most important distinction between parameters and metrics?
1. [ ] Parameters track model performance while metrics track configuration settings
2. [x] Parameters are configuration settings (strings/numbers) while metrics are numeric performance measures
3. [ ] Parameters can only be logged once while metrics can be updated during training
4. [ ] Parameters are automatically detected while metrics must be manually specified
# What is the primary challenge when applying traditional machine learning evaluation to text classification?
1. [ ] Text data requires specialized machine learning algorithms not available in scikit-learn
2. [ ] Cross-validation cannot be applied to text classification problems
3. [x] Text data often has imbalanced classes and domain-specific patterns requiring careful evaluation strategies
4. [ ] Text classification models cannot be saved or reloaded for deployment
# Why is vocabulary size (max_features in CountVectorizer) a critical hyperparameter in text classification?
1. [ ] Larger vocabularies always improve classification performance
2. [ ] Vocabulary size determines which machine learning algorithm to use
3. [x] Vocabulary size affects the balance between feature richness and noise, impacting generalization
4. [ ] Vocabulary size automatically handles class imbalance in the dataset