Kafka Streams Api For Developers Using Java/Springboot

Master the Kafka Streams API to build advanced real time Kafka Streaming applications using Java and SpringBoot.

What you'll learn

Build Advanced Kafka Streams Applications using Streams API

Build Kafka Streams Application using HighLevel DSL

Build a Realtime Retail Streaming application using Streams API

Aggregate Multiple events in to aggregated events

Join Multiple Streams into one Joined Stream

Aggregate Streams into Windowed Group of Events

Build Enterprise Standard Kafka Streams Application using SpringBoot

Testing Kafka Streams using TopologyTestDriver using JUnit5

Testing Spring Kafka Streams using EmbeddedKafka and JUnit5

Build Interactive Queries to retrieve the aggregated data through RESTFUL APIs

Requirements

Java Knowledge is required

Prior experience building Kafka Applications

Prior experience working with IntelliJ or any other IDEA

Java 17 is required

Gradle or Maven Knowledge is needed

Description

Kafka Streams API is an advanced API that's part of the Kafka Ecosystem. Using Kafka Streams API, we can :Apply data transformations, Data enrichment, Branching the data into multiple data streams.Aggregating the data or joining data from multiple Kafka topics.Aggregate the Data into Window Buckets and more.The Kafka Streams API For Developers using Java/SpringBoot course is structured to give you both the theoretical and coding experience of developing Kafka Streams Applications using Streams API and also covers the techniques to use Enterprise Standard Kafka Streams Application using SpringBoot and Streams API .This is a pure hands-on oriented course where you will be learning the concepts through code. You will build a realtime Kafka Streams application by the end of this course.By the end of this course, you will have a complete understanding of these concepts:Building Kafka Streams Applications using Streams APIBuilding Kafka Streams Applications using SpringBoot & Streams APIWrite Interactive Queries to retrieve the aggregated data from a state store and expose it via RESTFUL API.Unit and Integration Testing Kafka Streams Applications using JUnit5Getting Started to Kafka StreamsIn this section, I will give you all an introduction to Kafka streams and the different terminologies that are involved in build a Kafka Streams Application.Introduction to Kafka StreamsKafka Streams Terminologies - Topology & ProcessorIntroduction to KStreams APIGreetings Kafka Streams App using KStreams APIIn this section, we will build a simple Kafka Streams App and test it in our local.Learn to build a Topology of the Greetings AppBuild the Kafka Streams Launcher Application using which we can start and stop the application.Operators in Kafka Streams using KStream APIIn this section, we will explore some of the operators in the thats part of the Kafka Streams API.Filter & FilterNotMap/MapValuesFlatMapValues/FlatMappeekmergeSerialization and Deserialization in Kafka StreamsIn this section, we will code and explore the serialization and deserialization in Kafka Streams.How Key/Value serialization and deserialization works in Kafka Streams ?Providing Default Serializer/Deserializer using Application ConfigurationBuild a Custom Serdes for Enhanced Greeting MessagesReusable Generic Serializer/Deserializer (Recommended Approach)In this section, I will show you the best approach to build a Generic Serializer and Deserializer that can be used for any type of Messages.Build a Generic Serializer/DeserializerOrder Management Kafka Streams application - A real time use caseIn this section, we will build a kafka streams application by implementing a Order Management system for a retail companyTopology, Stream and Tasks - Under the HoodIn this section, we will explore the internals of the Kafka Streams Application.Internals of Topology, Stream and TasksError/Exception Handling in Kafka StreamsIn this section, we will explore different error handlers in Kafka Streams.Failures in Kafka StreamsDefault Deserialization Error BehaviorCustom Deserialization Error HandlerDefault & Custom Processor Error HandlerCustom Production Error HandlerKTable & Global KTableIn this section, we will explore the KTable and GlobalKTable in KafkaStreams.Introduction to KTable APIBuild a topology for KTable KTable - Under the HoodGlobalKTableStateFul Operations in Kafka Streams - Aggregate, Join and Windowing EventsIn this section, I will give an introduction to stateful operators in Kafka Streams and explore the aggregation related operators in Kafka streams.StateFul Operations in Kafka StreamsHow aggregation works ? & Aggregation using "count" operatorGroup Records by using groupBy operatorAggregation using "reduce" operartorAggregation using "aggregate" operatorUsing Materialized views for count & reduce operatorStateFul Operation Results - How to access them ?In this section, I will explain about the options to retrieve the results of the aggregation.Re-Keying Kafka Records for Stateful operationsIn this section, we will code and explore the effect of null operator and the need to rekeying records during stateful operations.StateFul Operations in Kafka Streams - JoinIn this section, we will code and explore the different types of Joins in Kafka Streams Application.Join in Order Management Application - A Real Time Use CaseIn this section, we will implement join in the order management application that we have been working on so far.Introduction to Joins & Types of Joins in Kafka StreamsExplore innerJoin using "join" operator - Joining KStream and KTableExplore innerJoin using "join" operator - Joining KStream and GlobalKTableExplore innerJoin using "join" operator - Joining KTable and KTableExplore innerJoin using "join" operator - Joining KStream and KStreamJoining Kafka Streams using "leftJoin" operatorJoining Kafka Streams using "outerJoin" operatorJoin - Under the hoodCoPartitioning Requirements in Joins & Joins Under the HoodStateFul Operations in Kafka Streams - WindowingIn this section, we will explore the windowing concepts in Kafka Streams.Introduction to Windowing and time conceptsWindowing in Kafka Streams - Tumbling WindowsControl emission of windowed results using "supress" operartorWindowing in Kafka Streams - Hopping WindowsWindowing in Kafka Streams - Sliding WindowsWidowing in Order Management Application - A Real Time Use CaseIn this section, we will code and explore the new requirement to implement the windowing in the Orders Stream Application.Behavior of Records with Future & Older Timestamp in WindowingIn this section, we will explore the behavior of records with the older and future timestamp in a Kafka Streams Application.Build Kafka Streams Application using SpringBootIn this section, we will build a simple kafka streams app using SpringBoot.Introduction to SpringBoot and Kafka StreamsSetup the Project - Greeting Streams app using Spring Kafka StreamsConfiguring the Kafka Stream using application.ymlBuild the Greeting Topology Test Greeting App in LocalSpringBoot AutoConfiguration of Kafka StreamsIn this section, I will show you how spring boot auto configures Kafka Streams Application.JSON Serialization/Deserialization in Spring Kafka StreamsIn this section, we will implement the JSON Serialization/Deserialization in Kafka Streams using SpringBoot.Error Handling in Spring Kafka StreamsIn this section, I will show you error handling in Kafka Streams using SpringBoot.Handle DeSerialization Error - Approach 1Handle DeSerialization Error using Custom Error Handler - Approach 2Handle Deserialization Error - Approach 3 ( Spring Specific Approach)Handle UncaughtExceptions in the TopologyHandle Production ErrorsBuild Orders Kafka Streams Application using SpringBootIn this section, we will set up the Spring Boot Project for orders streams.Interactive Queries - Querying State Stores using RESTFUL APIsBuild a GET Endpoint to retrieve the OrderCount by OrderType - Part 1Build a GET Endpoint to retrieve the OrderCount by OrderType - Part 2Retrieve OrderCount by OrderType & LocationIdBuild a GET Endpoint to retrieve the OrderCount for All OrderTypesBuild a GET Endpoint to retrieve the Revenue by OrderTypeGlobal Error Handling for useful Client Error MessagesInteractive Queries - Querying Window State Stores using RESTFUL APIsBuild a GET Endpoint to Retrieve OrderCount by OrderTypeBuild a GET Endpoint to Retrieve the windowed OrderCount for All OrderTypesBuild a GET endpoint to retrieve the windowed OrderCount within a Time RangeBuild a GET Endpoint to retrieve the Revenue by OrderTypeTesting Kafka Streams Using TopologyTestDriver & JUnit5In this section, we will code and learn about how to write automated tests for Kafka Streams app.Testing Kafka Streams using TopologyTestDriverUnit Testing Greetings App - Writing Data to a Output TopicUnit Testing Greetings App - Testing Multiple MessagesUnit Testing Greetings App - Error ScenarioUnit Testing OrdersCount - Writing Data to a State StoreUnit Testing OrdersRevenue - Writing Data to a State StoreUnit Testing OrdersRevenue By Windows - Writing Data to a State StoreLimitations of TopologyTestDriverTesting Kafka Streams in SpringBoot Using TopologyTestDriver & JUnit5In this section, we will code and learn how to write unit tests in our Kafka Streams application that's build using SpringBoot.Integration Testing Spring KafkaStreams App using @EmbeddedKafkaIn this section, we will code and learn about writing integration tests for the Kafka Streams app using EmbeddedKafka.Introduction & SetUp Integration TestIntegration Test for OrdersCountIntegration Test for OrdersRevenueIntegration Test for OrdersRevenue By WindowsGrace Period in Kafka StreamsIn this section I will explain the concept of grace period and how it can be used in kafka streams application.Build and Package the SpringBoot App as an ExecutableIn this section, we will package the kafka streams app as an executable and launch the app.By the end of this course you will have a complete understanding of Kafka Streams API and the different kinds of applications that can be built using Kafka Streams API.

Overview

Section 1: Getting Started With the Course

Lecture 1 Course Introduction

Lecture 2 Pre-requestites

Section 2: Getting Started to Kafka Streams

Lecture 3 Introduction to Kafka Streams

Lecture 4 Kafka Streams Terminologies - Topology & Processor

Lecture 5 Introduction to KStreams API

Section 3: Greetings Kafka Streams App using KStreams API

Lecture 6 Overview of the Greetings App

Lecture 7 Setup the Greetings App

Lecture 8 Topology of the Greetings App

Lecture 9 Build the Kafka Streams Launcher Application

Lecture 10 Setting Up Kafka Environment and Test our Greeting App

Section 4: Operators in Kafka Streams using KStream API

Lecture 11 Filter & FilterNot

Lecture 12 Map/MapValues

Lecture 13 FlatMapValues/FlatMap

Lecture 14 peek

Lecture 15 merge

Section 5: Serialization and Deserialization in Kafka Streams

Lecture 16 How Key/Value serialization and deserialization works in Kafka Streams ?

Lecture 17 Providing Default Serializer/Deserializer using Application Configuration

Lecture 18 Build a Custom Serdes for Enhanced Greeting Messages

Lecture 19 Usage of CustomSerde in the Greeting App

Section 6: Reusable Generic Serializer/Deserializer (Recommended Approach)

Lecture 20 Build a Generic Serializer/Deserializer

Lecture 21 Integrate Generic Serializer/Deserializer into the Greeting App

Section 7: Order Management Kafka Streams application - A real time use case

Lecture 22 Overview of the Retail App

Lecture 23 Build the Topology for the Orders Management App

Lecture 24 Split the Restaurant/Retail Shopping Orders - Using split and branch operator

Lecture 25 Transform the Order Domain to a Revenue Domain Type

Section 8: Topology, Stream and Tasks - Under the Hood

Lecture 26 Internals of Topology, Stream and Tasks

Lecture 27 Explore the behavior of streams by modifying the stream threads

Section 9: Error/Exception Handling in Kafka Streams

Lecture 28 Failures in Kafka Streams

Lecture 29 Default Deserialization Error Behavior

Lecture 30 Custom Deserialization Error Handler

Lecture 31 Default & Custom Processor Error Handler

Lecture 32 Custom Production Error Handler

Lecture 33 Error handling when Kafka Cluster is down

Section 10: KTable & Global KTable

Lecture 34 Introduction to KTable API

Lecture 35 Build a topology for KTable

Lecture 36 KTable - Under the Hood

Lecture 37 GlobalKTable

Section 11: StateFul Operations in Kafka Streams - Aggregate, Join and Windowing Events

Lecture 38 StateFul Operations in Kafka Streams

Lecture 39 What is Aggregation & How it works ?

Lecture 40 Aggregation using "count" operator

Lecture 41 Aggregation using "reduce" operator

Lecture 42 Aggregation using "aggregate" operator

Lecture 43 Using Materialized Store for count & reduce operator

Section 12: StateFul Operation Results - How to access them ?

Lecture 44 How to access the results of Aggregation ?

Section 13: Aggregation in Order Management Application - A Real Time Use Case

Lecture 45 Total number of orders by each store using "count" operator

Lecture 46 Total Revenue by each store using "aggregate" operator

Section 14: Re-Keying Kafka Records for Stateful operations

Lecture 47 Effect of null Key in Stateful Operations & Repartition of Kafka Records

Lecture 48 Re-Keying using the "selectKey" operator

Section 15: StateFul Operations in Kafka Streams - Join

Lecture 49 Introduction to Joins & Types of Joins in Kafka Streams

Lecture 50 Explore innerJoin using "join" operator - Joining KStream and KTable

Lecture 51 Explore innerJoin using "join" operator - Joining KStream and GlobalKTable

Lecture 52 Explore innerJoin using "join" operator - Joining KTable and KTable

Lecture 53 Explore innerJoin using "join" operator - Joining KStream and KStream

Lecture 54 Joining Kafka Streams using "leftJoin" operator

Lecture 55 Joining Kafka Streams using "outerJoin" operator

Lecture 56 Join - Under the hood

Lecture 57 CoPartitioning Requirements in Joins

Section 16: Join in Order Management Application - A Real Time Use Case

Lecture 58 Join Aggregate Revenue with StoreDetails KTable

Section 17: StateFul Operations in Kafka Streams - Windowing

Lecture 59 Introduction to Windowing and time concepts

Lecture 60 Windowing in Kafka Streams - Tumbling Windows

Lecture 61 Control emission of windowed results using "supress" operartor

Lecture 62 Windowing in Kafka Streams - Hopping Windows

Lecture 63 Windowing in Kafka Streams - Sliding Windows

Section 18: Widowing in Order Management Application - A Real Time Use Case

Lecture 64 New Requirements for the Order Management Application

Lecture 65 Implementing a CustomTimeStamp Extractor

Lecture 66 Aggregate "Number of Orders" by Windows

Lecture 67 Aggregate Revenue by Windows

Lecture 68 Joins on the Windowed Data

Section 19: Behavior of Records with Future & Older Timestamp in Windowing

Lecture 69 Records with timestamps before & after the CurrentTimestamp.

Section 20: Build Kafka Streams Application using SpringBoot

Lecture 70 Introduction to SpringBoot and Kafka Streams

Lecture 71 Setup the Project - Greeting Streams app using Spring Kafka Streams

Lecture 72 Configuring the Kafka Stream using application.yml

Lecture 73 Build the Greeting Topology

Lecture 74 Test Greeting App in Local

Section 21: SpringBoot AutoConfiguration of Kafka Streams

Lecture 75 Internals of AutoConfiguring Kafka Streams in SpringBoot

Section 22: JSON Serialization/Deserialization in Spring Kafka Streams

Lecture 76 JSON Serialization/Deserialization using JsonSerde

Lecture 77 JsonSerde using custom ObjectMapper

Section 23: Error Handling in Spring Kafka Streams

Lecture 78 Handle DeSerialization Error - Approach 1

Lecture 79 Handle DeSerialization Error using Custom Error Handler - Approach 2

Lecture 80 Handle DeSerialization Errors using Spring Specific Approach- Approach 3

Lecture 81 Handle UncaughtExceptions in the Topology

Lecture 82 Handle Production Errors

Section 24: Build Orders Kafka Streams Application using SpringBoot

Lecture 83 Set up the base project for Orders Kafka Streams App

Lecture 84 Create the OrdersTopology

Section 25: Interactive Queries - Querying State Stores using RESTFUL APIs

Lecture 85 Build a GET Endpoint to retrieve the OrderCount by OrderType - Part 1

Lecture 86 Build a GET Endpoint to retrieve the OrderCount by OrderType - Part 2

Lecture 87 Retrieve OrderCount by OrderType & LocationId

Lecture 88 Build a GET Endpoint to retrieve the OrderCount for All OrderTypes

Lecture 89 Build a GET Endpoint to retrieve the Revenue by OrderType

Lecture 90 Global Error Handling for useful Client Error Messages

Section 26: Interactive Queries - Querying Window State Stores using RESTFUL APIs

Lecture 91 Build a GET Endpoint to Retrieve OrderCount by OrderType

Lecture 92 Build a GET Endpoint to Retrieve the windowed OrderCount for All OrderTypes

Lecture 93 Build a GET endpoint to retrieve the windowed OrderCount within a Time Range

Lecture 94 Build a GET Endpoint to retrieve the Revenue by OrderType

Section 27: Testing Kafka Streams Using TopologyTestDriver & JUnit5

Lecture 95 Testing Kafka Streams using TopologyTestDriver

Lecture 96 Unit Testing Greetings App - Writing Data to a Output Topic

Lecture 97 Unit Testing Greetings App - Testing Multiple Messages

Lecture 98 Unit Testing Greetings App - Error Scenario

Lecture 99 Unit Testing OrdersCount - Writing Data to a State Store

Lecture 100 Unit Testing OrdersRevenue - Writing Data to a State Store

Lecture 101 Unit Testing OrdersRevenue By Windows - Writing Data to a State Store

Lecture 102 Limitations of TopologyTestDriver

Section 28: Testing Kafka Streams in SpringBoot Using TopologyTestDriver & JUnit5

Lecture 103 UnitTest Using TopologyTestDriver in SpringBoot

Section 29: Integration Testing Spring KafkaStreams App using @EmbeddedKafka

Lecture 104 Introduction & SetUp Integration Test

Lecture 105 Integration Test for OrdersCount

Lecture 106 Integration Test for OrdersRevenue

Lecture 107 Integration Test for OrdersRevenue By Windows

Section 30: Grace Period in Kafka Streams

Lecture 108 Grace Period in Windowing

Section 31: Build and Package the SpringBoot App as an Executable

Lecture 109 Package the SpringBoot app and execute it as a Jar File

Advanced Java Developers,Kafka Developers who are curious to learn Kafka Streams API,Kafka Developers who are interested in building advanced streaming applications,Developers who wish to learn the techniques to test Kafka Streams Application using TopologyTestDriver