https://hub.imt-atlantique.fr/data-engineering-foundations/pipeline/index.html Presentation & objectives This Data Engineering course introduces students to building modern data pipelines using Azure, Python and industrial practices. It emphasizes the bronze/silver/gold approach to transform raw data into actionable insights, while integrating data quality, testing and generative AI. In particular, we will cover the following concepts: Understanding the Bronze/Silver/Gold architecture for layered data modeling Mastering ELT paradigm and modern pipeline design patterns Implementing clean code principles and Python structuration (classes, error handling) Writing unit tests and data quality tests with pytest Setting up CI/CD pipelines with GitHub Actions Implementing logging and observability (structured JSON logs) Configuring Azure monitoring and cost tracking Using generative AI tools appropriately (Copilot OK with tests) By the end of this course, you will be able to: Hugo en-us