ASkill360 · AZURE DATA ENGINEERING

Data Engineer with Azure Cloud

A 45‑hour, day‑by‑day course from fundamentals to a production real‑time pipeline — built for certification and the job interview that follows.

Enroll Now See Curriculum

45Hours

45Live Sessions

11Modules

11Assessments

2Batch Capstones

CURRICULUM MAP

11 modules. One pipeline. 45 hours.

Click a module to see every session, what it teaches, and the assessment that closes it.

Module 01 · Day 01–03 Data Engineering Fundamentals Understand the data before you move it

Day 01 What Is Data & Data Engineering? Structured, semi-structured and unstructured data, and how the data engineer role differs from analyst and scientist.
Day 02 Data Formats CSV, JSON, Parquet, Avro and ORC — and when each one is the right choice.
Day 03 The Data Engineering Lifecycle Ingestion, storage, transformation, quality and serving — and where batch pipelines fit in that chain.

Module assessment — completed independently by the student. Short written exercise — explain the data engineering lifecycle and pick the right file format for three given scenarios.

Module 02 · Day 04–08 SQL Query the data before you move it

Day 04 DDL, DML & Data Types Learn how tables are created and how rows get inserted, updated and deleted.
Day 05 Joins & Subqueries Combine multiple tables into one result set the way real reports need.
Day 06 Aggregations: GROUP BY & HAVING Turn raw rows into summaries — totals, counts and averages per group.
Day 07 Window Functions Rank, number and compare rows across partitions with RANK, ROW_NUMBER and LAG/LEAD.
Day 08 CTEs, Views & Query Tuning Write readable, reusable queries and spot what makes them slow.

Module assessment — completed independently by the student. Timed query test on a sample sales database — joins, aggregations and window functions, graded against expected output.

Module 03 · Day 09–13 Python The scripting layer behind every pipeline

Day 09 Syntax, Data Types & Control Flow Cover variables, loops and conditionals — the grammar of every script you'll write.
Day 10 Data Structures: List, Tuple, Set, Dict Store and manipulate collections of data with Python's four core built-in structures.
Day 11 Functions & Comprehensions Package logic into reusable functions and write list/dict comprehensions like a native.
Day 12 OOP: Classes & Objects Model real-world entities with classes, objects, inheritance and encapsulation.
Day 13 File & Exception Handling Read and write files safely and handle errors so pipelines don't crash silently.

Module assessment — completed independently by the student. Small coding exercise — build a class-based data structure and pass a peer code review checklist.

Module 04 · Day 14–15 Azure Storage Where every pipeline starts and ends

Day 14 Storage Accounts & ADLS Gen2 Understand Blob storage vs. Data Lake Gen2 and the hierarchical namespace that powers it.
Day 15 Access Control & Security Secure data with SAS tokens, managed identities, RBAC and container-level permissions.

Module assessment — completed independently by the student. Configure a secured ADLS Gen2 container from scratch — correct RBAC, SAS scope and access tier for a given scenario.

Module 05 · Day 16–24 PySpark Batch processing at scale

Day 16 Spark Architecture See how driver, executors and cluster manager split work across machines.
Day 17 DataFrame API Basics Select, filter and transform columns using Spark's core DataFrame operations.
Day 18 Aggregations & Joins Combine and summarise large datasets across distributed partitions.
Day 19 Spark SQL & Temp Views Query DataFrames with plain SQL by registering them as temporary views.
Day 20 UDFs & Broadcast Joins Extend Spark with custom functions and speed up joins against small tables.
Day 21 Partitioning, Shuffling & Tuning Control how data splits across the cluster and diagnose slow jobs with caching and explain plans.
Day 22 Delta Lake: ACID & Time Travel Add reliability to your data lake with transactions and the ability to query past versions.
Day 23 Delta Lake: Merge & Optimize Upsert changes with MERGE and compact small files with OPTIMIZE and VACUUM.
Day 24 Mini Project: Batch ETL Build a complete extract-transform-load pipeline end to end in PySpark.

Module assessment — completed independently by the student. Submit a working batch ETL notebook — reads, transforms and writes data through a Delta Lake table.

Module 06 · Day 25–29 Databricks The unified platform for the whole workflow

Day 25 Workspace & Clusters Get familiar with notebooks, cluster types and how a Databricks workspace is organised.
Day 26 DBFS & Unity Catalog Mount external storage and govern data access through Unity Catalog.
Day 27 Jobs & Workflows Schedule notebooks as production jobs and chain tasks into workflows.
Day 28 Storage Integration & Secrets Connect Databricks to ADLS securely using secret scopes.
Day 29 Repos & Best Practices Version-control notebooks with Repos and apply team development standards.

Module assessment — completed independently by the student. Schedule and run a multi-task Databricks workflow end to end, unaided, against a set of test data.

Module 07 · Day 30–35 Azure Data Factory Orchestrating batch pipelines across the cloud

Day 30 Pipelines, Linked Services & Datasets Learn the three building blocks every ADF pipeline is made of.
Day 31 Copy Activity & Integration Runtime Move data between sources and sinks using ADF's core copy engine.
Day 32 Mapping Data Flows Transform data visually at scale without writing Spark code by hand.
Day 33 Triggers & Parameterization Automate batch pipeline runs on a schedule or a tumbling window.
Day 34 Incremental Load & CDC Load only what changed instead of reprocessing entire datasets every run.
Day 35 Databricks Integration & Mini Project Call Databricks notebooks from ADF, monitor the run, and assemble a full end-to-end batch pipeline.

Module assessment — completed independently by the student. Build and trigger an ADF pipeline that copies and transforms data on a schedule, with no instructor input.

Module 08 · Day 36–38 Kafka & Structured Streaming One focused look at streaming's building block

Day 36 Kafka Fundamentals Topics, partitions, brokers, producers, consumers and consumer groups — how Kafka actually works.
Day 37 Practical Kafka Coding (Mentor-Led) Hands-on producer/consumer coding session taught live by a working mentor engineer.
Day 38 Spark Structured Streaming Read a live Kafka topic into Spark Structured Streaming, apply a windowed aggregation, and write the result to a Delta table.

Module assessment — completed independently by the student. Produce and consume messages on a live Kafka topic, then read that stream into Spark Structured Streaming and write a windowed aggregation to Delta — reviewed by a mentor.

Module 09 · Day 39–41 Capstone 1: Batch Pipeline on Databricks Build it entirely inside Databricks

Day 39 Architecture & Bronze Layer (Databricks) Design a batch pipeline built and orchestrated entirely inside Databricks, and land raw source files into a bronze Delta table via a scheduled Databricks Job.
Day 40 Silver & Gold Layers (Databricks) Clean and aggregate data through silver and gold Delta tables using PySpark notebooks chained in a Databricks Workflow.
Day 41 Test, Monitor & Document (Databricks) Run the full Databricks Workflow end to end, add job monitoring and alerting, and document the architecture.

Module assessment — completed independently by the student. Present the completed Databricks-orchestrated batch pipeline to a review panel, and defend the design choices made.

Module 10 · Day 42–44 Capstone 2: Batch Pipeline on Azure Data Factory Build it with ADF orchestrating every layer

Day 42 Architecture & Bronze Layer (ADF) Design a batch pipeline orchestrated by Azure Data Factory, using Copy Activity and a scheduled trigger to land raw files into a bronze layer.
Day 43 Silver & Gold Layers (ADF) Use ADF Mapping Data Flows and pipeline activities to clean and aggregate data into silver and gold layers.
Day 44 Test, Monitor & Document (ADF) Run the full ADF pipeline end to end, add pipeline monitoring and alerting, and document the architecture.

Module assessment — completed independently by the student. Present the completed ADF-orchestrated batch pipeline to a review panel, and defend the design choices made.

Module 11 · Day 45–45 Career Launch Turning skills into an offer

Day 45 Resume, LinkedIn & Negotiation Prep Turn the finished projects and skills into a resume, profile and offer you're ready to negotiate.

Module assessment — completed independently by the student. Submit a finished resume and LinkedIn profile built around the capstone projects, reviewed against a hiring checklist.

OUTCOMES

What you walk away with.

Certification-ready skills

Mapped session by session to the real industry certification exam domains.

Two working batch pipelines

Real, portfolio-ready pipelines built layer by layer — one on Databricks, one on Azure Data Factory.

Two rehearsed mock interviews

A technical scenario round and a project-walkthrough-plus-behavioral round.

ASSESSMENTS

11 modules. 11 independent checkpoints.

Every module ends with one assessment the student completes on their own.

Module	Assessment
Data Engineering Fundamentals	Short written exercise — explain the data engineering lifecycle and pick the right file format for three given scenarios.
SQL	Timed query test on a sample sales database — joins, aggregations and window functions, graded against expected output.
Python	Small coding exercise — build a class-based data structure and pass a peer code review checklist.
Azure Storage	Configure a secured ADLS Gen2 container from scratch — correct RBAC, SAS scope and access tier for a given scenario.
PySpark	Submit a working batch ETL notebook — reads, transforms and writes data through a Delta Lake table.
Databricks	Schedule and run a multi-task Databricks workflow end to end, unaided, against a set of test data.
Azure Data Factory	Build and trigger an ADF pipeline that copies and transforms data on a schedule, with no instructor input.
Kafka & Structured Streaming	Produce and consume messages on a live Kafka topic, then read that stream into Spark Structured Streaming and write a windowed aggregation to Delta — reviewed by a mentor.
Capstone 1: Batch Pipeline on Databricks	Present the completed Databricks-orchestrated batch pipeline to a review panel, and defend the design choices made.
Capstone 2: Batch Pipeline on Azure Data Factory	Present the completed ADF-orchestrated batch pipeline to a review panel, and defend the design choices made.
Career Launch	Submit a finished resume and LinkedIn profile built around the capstone projects, reviewed against a hiring checklist.

Ready to start Day 1?

The next cohort is open for enrollment now.

Enroll Now