Handling Streaming Data with Messaging Systems

Apache Kafka, a messaging platform, is currently deployed in over a third Fortune 500 companies. This course, Deploying Kafka Cluster, will teach you the basics of Apache Kafka. You'll first learn how it can be used in modern digital platforms. Next, we'll look at Kafka's core ideas. You'll also learn how to deploy it to achieve fault tolerance. This course will equip you with the knowledge and skills to deploy Apache Kafka and other applications built around it.

Many common situations can arise when streaming platforms are used within an organization. This course, Handling Streaming Data With a Kafka Cluster, will teach you how to deal with many different situations. You'll first learn why Kafka is such a great tool for streaming data. Next, you will explore different optimizations and how to integrate with other models. Next, you will learn how to manage your data as well as perform different operations against your Kafka Cluster. You'll also learn different methods to protect your data streams. This course will equip you with the knowledge and skills to manage streaming data with Apache Kafka.

Apache Pulsar is a messaging system that uses a cool architecture to meet your needs. You can argue that it is even better if it runs on Google Kubernetes Engine. Combining both systems can offer incredible scalability, fault tolerance, and allow you to scale existing businesses or open up new opportunities for products that don't exist yet. This course, Deploying Apache Pulsar with Google Kubernetes engine, will teach you how to set up Apache Pulsar into a system that scales. You'll first learn about the advantages and disadvantages of Apache Pulsar in comparison to Apache Kafka. Next, you will learn how to install, configure, and manage Apache Pulsar using Google Kubernetes Engine. You'll also learn how to create consumers and producers that will use your Apache Pulsar system. After completing this course, you will have the knowledge and skills to run Apache Pulsar on Google Cloud. This is necessary in order to create a powerful messaging system which can be horizontally scaled around the globe.

Scaling real-time applications is difficult! They are able to access large amounts of data quickly and must route messages correctly. Apache Pulsar can be used to tackle this problem. It is highly scalable, low-latency, and high throughput pub/sub system. This course, Apache Pulsar: Handling Streaming Data, will teach you how to manage them using Apache Pulsar. You'll first learn about Pulsar Functions, which allow for serverless ETL. Next, you will learn how to connect your Pulsar deployment with Kafka and other databases using PulsarIO. You'll also learn how to migrate Kafka to Pulsar using the client wrapper. After completing this course, you will have the knowledge and skills to manage high volumes of streaming data in your applications.

The Big Data industry is evolving. In the past, processing incoming data could be done for hours or even days. You need to be able to process incoming data in minutes, or even seconds. These problems require new solutions, new architectures and new tools. In Developing Stream Processing Apps with AWS Kinesis you'll learn all about AWS Kinesis. You will first learn about how it works and how to scale it down and up, as well as how to create applications using it. You will then learn how to use various tools such as Kinesis Client Library and Kinesis Connector Library. You will also learn how to use Kinesis Firehose, a high-level Kinesis product, and how to create streaming applications using SQL queries with Kinesis Analytics.

Processing streaming data requires both low latency and high scalability. This course, Handling Streaming data with Azure Event Hub, will teach you how to use Azure Event Hubs for real-time event processing and reception.

Data warehousing, analytics, and real-time data processing are becoming more integral to companies' business models. Stream processing is now a must-have feature. This course, Architecting Stream Processing Solutions Using Google Cloud Pub/Sub will teach you how to ingest, process, and take snapshots of streaming data using the Google Cloud Platform. You can also replay messages and create replay messages. You will first learn about the basic architecture of a Publisher/Subscriber. Publishers are apps that send messages. These messages are organized into Topics. Subscriptions can be associated with Topics. Subscribers must listen to subscriptions. Each subscription has a message queue. Messages are kept in the queue until at most one subscriber has acknowledged that they have received them. Pub/Sub is a reliable messaging platform. Next you'll learn how to create topics and how to pull and push subscriptions. Push and pull subscriptions have different names. They differ in who is responsible for delivering messages to subscribers. You will also learn how to use advanced features such as Pub/Sub, including creating snapshots and searching for a specific timestamp in the future or the past. The exact semantics of creating snapshots will be explained, as well as the implications of turning off the "retain acknowledgements messages" option in a subscription. After completing this course, your knowledge and skills in Google Cloud Pub/Sub will allow you to efficiently and reliably process streaming data via the GCP.

It's possible you are wondering why "Connect" suddenly appeared next to "Kafka". Isn't Kafka a Distributed Streaming Platform? Kafka is much more than that. Kafka Connect is part of Apache Kafka, which is an ecosystem. This course, Kafka Connect Foundations, will teach you how to set up your own ETL pipelines to and from Apache Kafka. You will first learn about the ETL model and how to create your own ETL pipeline with Kafka Connect. The architecture of Kafka Connect will be next. You will also learn how to manage Kafka Connect installations in a production setting. After completing this course, your knowledge and skills in KafkaConnect will be able to create, build, and maintain your KafkaConnect installation.

It is very valuable to learn how to transform Kafka into an analytics engine. This course, Building ETL Pipelines From Streaming Data With Kafka and KSQL teaches you how to transform and shape your Kafka streaming data. This course will first show you how Kafka Streams and ksqlDB solve this problem. The next step is to learn how to transform your streams. You'll also learn how to enrich and aggregate data. After completing this course, your knowledge and skills in ksqlDB will be able to extract insights from Kafka streaming datasets.

Governance can quickly become chaotic in a data-driven world. This course, Enforcing data contracts with Kafka Schema Registry teaches you how to manage and enforce data contracts in an Apache Kafka-powered system. You'll first learn about serialization and why AVRO is such a great choice. Next, you will learn how to manage data contracts with Schema Registry. You'll also learn how to use Apache Kafka with other serialization formats. After completing this course, you will have the knowledge and skills to manage and enforce data contracts in your Apache Kafka configuration.

This course, Implementing an Events Log with Kafka will teach you how to create complex microservice architectures around immutable Kafka events. You'll first learn about the issues that can arise when migrating applications to a microsevices structure. Next, you will master Kafka basics and learn how they can help you address common problems in microservices applications. You'll also learn advanced patterns for working with Kafka data. After completing this course, you will have the knowledge and skills to create complex event-driven applications with Kafka.