Data Engineering

Module 1: Azure Fundamentals

• Introduction to Azure:
• Overview of Azure cloud platform
• Azure services and components

• Azure Resource Manager:
• Creating and managing resources
• Resource groups and subscriptions

• Azure Storage:
• Blob storage, file storage, table storage, queue storage

• Azure Compute:
• Virtual machines, Azure Functions, Azure App Service

Module 2: Data Ingestion and Transformation

• Azure Data Factory:
• o Creating and managing data pipelines
• Ingesting data from various sources (on-premises, cloud, social media)
• Transforming data using data transformations

• Azure Synapse Analytics:
• Serverless SQL pool
• Dedicated SQL pool
• Spark pool

• Azure Data Lake Storage Gen2:
• Storing and managing large-scale data sets

Module 3: Data Processing and Analysis

• Azure Databricks:
• Apache Spark on Azure
• Data processing and analytics
• Machine learning pipelines

• Azure HDInsight:
• Hadoop and Spark clusters on Azure
• Batch processing and data warehousing

• Azure Machine Learning:
• Building and deploying machine learning models
• Model management and governance

Module 4: Data Warehousing and Business Intelligence

• Azure Synapse Analytics:
• Data warehousing with dedicated SQL pool
• Data mart creation

• Azure Analysis Services:
• Creating and managing tabular models
• Data analysis and reporting

• Power BI:
• Integrating with Azure data sources
• Creating interactive dashboards and reports

Module 5: Data Governance and Security

• Azure Preview:
• Data catalog and governance
• Data lineage and impact analysis

• Azure Security Center:
• Protecting data and resources
• Threat detection and response

• Azure Key Vault:
• Managing secrets and certificates

Module 6: Advanced Topics

• Azure Stream Analytics:
• Real-time data processing and analysis

• Azure IoT Hub:
• Ingesting data from IoT devices

• Azure Cosmos DB:
• Globally distributed NoSQL database

• Azure Data Bricks Delta Lake:
• Scalable data lake platform

• Azure Synapse Analytics Serverless SQL Pool:
• Pay-as-you-go pricing for SQL queries

Module 7: Case Studies and Projects

• Real-world examples:
• IoT data analysis
• Retail analytics
• Fraud detection

• Hands-on projects:
• Building data pipelines
• Deploying machine learning models
• Creating data warehouses

Book Now