Microsoft Fabric - Apache Spark And Notebooks

Learn the Microsoft Fabric Spark environment, notebooks, dataframes, filtering, aggregating, joining dataframes and more

What you'll learn

Apache Spark and Notebook Integration into Microsoft Fabric

Overview of the Microsoft Fabric Notebooks Environment

Dataframe Basics

Common Operations with Dataframes

Filtering data using Dataframes

Aggregating data using Dataframes

Joining Dataframes

Spark Libraries

Working with Dates

SQL Statements and Spark

Spark Configuration and Tuning

Exercises and Questions for Practice

Requirements

Some working experience with Microsoft Fabric and Spark would be useful but not absolutely necessary

Description

This course takes you into the Microsoft Fabric experience of Apache Spark and Notebooks, where the power of distributed computing meets the elegance and fun of interactive coding in Python, R, Scala or SQL.In this comprehensive course, you will be introduced to Spark's pyspark library, empowering you to tackle common data challenges with remarkable ease. You will wrangle data in pandas dataframes and learn the most common dataframe operations. In short, you will acquire the skills necessary to manipulate your data in many different ways.As you move deeper into Spark, you will master the art of filtering data, joining dataframes, and manipulating dates and times with ease. You will also be exposed to using SQL in Spark, allowing you to use SQL queries to extract data from Delta Parquet files and storing them in dataframes.This course will also equip you with the knowledge you need to optimize Spark performance through effective configuration and autotuning. You will learn that there is very little you have to do to fully optimize the Spark environment in which your notebooks run. The course consists of over 30 lectures, complemented by hands-on practice exercises that will challenge your understanding of the concepts being taught and perhaps your research ability. By the end of this course, you will be a confident and proficient Spark notebook user, capable of harnessing its immense power to solve real-world data scenarios.Here are just some of the compelling benefits you will gain from this course:Master the fundamentals of Spark libraries, dataframes, and SQLLearn to how to filter, aggregate, and join data using dataframesGain expertise in manipulating dates and timesData WranglingLearn how to optimize Spark performance through configuration and autotuningSolve real-world data scenarios with Spark notebooksEnhance your resume with in-demand Fabric Spark notebook skillsPosition yourself for success in the booming field of Big DataIf you are ready to take your data skills to the next level, then enroll in this course today and learn how you can use Apache Spark and notebooks, integrated in Microsoft Fabric, in your day-to-day work.

Overview

Section 1: Introduction

Lecture 1 Introduction

Lecture 2 Important! You May Not Need to Purchase This Course!

Section 2: Introduction to Microsoft Fabric Notebooks

Lecture 3 What are they?

Lecture 4 A Quick Tour - Part 1

Lecture 5 A Quick Tour - Part 2

Section 3: Apache Spark and Notebooks

Lecture 6 Before Getting Started

Lecture 7 Introduction to Apache Spark

Lecture 8 Spark Libraries

Lecture 9 Dataframe Basics - Part 1

Lecture 10 Dataframe Basics - Part 2

Lecture 11 Dataframe Basics - Part 3

Lecture 12 Common Operations with Dataframes

Lecture 13 Filtering Data

Lecture 14 Aggregating Data

Lecture 15 Joining Dataframes

Lecture 16 Working with Dates

Lecture 17 SQL Statements and Spark

Lecture 18 Microsoft Fabric 1.1. / Spark Runtime

Lecture 19 Spark Configuration

Lecture 20 Spark Autotuning

Section 4: Data Wrangler

Lecture 21 Data Wrangler - Part 1

Lecture 22 Data Wrangler - Part 2

Lecture 23 Data Wrangler - Part 3

Section 5: Test Your Knowledge

Lecture 24 Exercise #1

Lecture 25 Exercise #2

Lecture 26 Question #1

Lecture 27 Question #2

Lecture 28 Exercise #3

Lecture 29 Exercise #4

Lecture 30 Question #3 and #4

Lecture 31 Exercise #5

Anyone wanting to learn how Apache Spark and Notebooks are integrated into Microsoft Fabric