Azure Databricks Master Program [real time scenarios + Labs]

Real world project for Azure Data Engineers using Azure Databricks , SQL, Data Lake, Databricks, HDInsight(DP200,DP201)

What you'll learn:
You will learn how to build a real-world data pipeline in Azure Databricks and AWS Databricks
This course has been taught using real world data used to report credit data
You will acquire good Data Engineering skills in Azure using Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage Gen2, Azure SQL Database, Azure Blob Storage
You will learn how to transform data using Databricks and load into Azure Data Lake Storage Gen2 You will learn how to transform data using Databricks Notebook Activity in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2
you will learn databricks integration with multiple data sources.

Requirements
Basic understanding about cloud computing will be useful, but not necessary. Experience in Azure is not required, I will take you through everything necessary to learn this course and build the project An Azure Account is required, If you don't have one we will create a free account in the course

Description
Azure Databricks Building a solution architecture for a data engineering solution using Azure Data Engineering technologies such as Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Gen2, Azure Blob Storage, Azure SQL Database, Azure Databricks, Azure HDInsight and Microsoft PowerBI.

Integrating data from HTTP clients, Azure Blob Storage and Azure Data Lake Gen2 using Azure Databricks.

Using Parameters Databricks notebooks and passing parameters values through Azure Data Factory.

Creating Azure Databricks Workspace, Creating Databricks clusters, Mounting storage accounts, Creating Databricks notebooks, performing transformations using Databricks notebooks, Invoking Databricks notebooks from Azure Data Factory.

Creating ADF pipelines to execute Databricks Notebook activities to carry out transformations.

Implementing the Azure Data Factory Analytics monitoring tool and how to extend the capability further.

Azure Storage Solutions

Creating Azure Storage Account, Creating containers, Uploading data, Access Control (IAM), Using Azure Storage explorer to interact with the storage account

Creating Azure Data Lake Gen2, Creating containers, Uploading data, Access Control (IAM), Using Azure Storage explorer to interact with the storage account

Creating Azure SQL Database, Pricing Tiers, Creating Admin User, Creating Tables, Loading Data and Querying the database.

Integrating Snowflake with Azure Databricks

Integrating salesforce with Azure Databricks

Integrating AWS S3 with Azure Databricks

Integrating ADLS Gen2 with Azure Databricks

Integrating ADLS Gen1 with Azure Databricks

Integrating Blob with Azure Databricks

Deep dive into catalyst optimizer

Who this course is for
This course is for architects, developers and university
University students looking for a career in Data Engineering IT developers working on other disciplines trying to move to Data Engineering Data Engineers/ Data Warehouse Developers currently working on on-premises technologies, or other cloud platforms such as AWS or GCP who want to learn Azure Technologies Data Architects looking to gain an understanding about Azure Data Engineering stack Data Scientists who want extend their knowledge into data engineering