
Explore kafka monitoring after the setup, and rate the course when you're ready. Learn to manage costs by stopping servers and deleting kafka and zookeeper stacks in cloudformation.
Meet Stefan Marek, the Apache Kafka course instructor with eight certifications and expertise in data analytics and AWS architecture. Connect on LinkedIn or Instagram and set a course goal.
Deploy a three-broker kafka cluster with cloudformation by uploading a template, configuring subnets and IPs, and launching the kafka resources stack to create a running cluster.
Perform a hands-on setup of a three-broker Kafka cluster, configure properties, connect to Zookeeper, create a dummy topic with replication factor three and twelve partitions, and run producer and consumer.
Learn to install and configure a visual Kafka administration suite by deploying Zoom Navigator, Kafka Manager, and Kafka Monitor on an Amazon Linux 2 instance using Docker and Docker Compose.
Set up the Kafka Manager UI with Docker Compose, configure zookeeper, and verify brokers and topics, then enable Kafka Manager as a systemd service.
Set up a Grafana Kafka dashboard by importing a prebuilt dashboard and configuring a Prometheus data source on localhost to monitor cpu, memory, and per-topic bytes in/out.
Add and duplicate Grafana panels for Kafka metrics from Prometheus, including online partitions, unclean leader election rates, under replicated partitions, and offline partitions.
Learn automated Kafka broker rolling restarts with Kafka utils, install Jolokia on broker 1, and configure Wallachia and Troika agents to monitor metrics and ensure under replicated partitions equal zero.
Install Jolokia on broker two and broker three, configure java agents, restart Kafka, and verify with curl and jq; ensure admin can query all brokers and run with Prometheus exporter.
Update Kafka configurations by applying min.in.sync.replicas=2 across all brokers using a rolling restart, validating changes with journalctl and grep, one broker at a time.
Learn how to rebalance kafka partitions manually using plain json, zookeeper and cli tools. Then verify with kafka manager and perform preferred replica election.
Learn to rebalance a Kafka cluster using LinkedIn's Kafka tools, including generating and executing partition reassignment with the signer, balancing partitions across brokers, and handling leadership.
Remove broker four by reassigning its 60 partitions to other brokers using the LinkedIn Kafka tools, then terminate the broker via CloudFormation and delete the EBS volume and IP resources.
Upgrade all kafka clients to a compatible version and update the log format to 2.0 on brokers after a rolling restart, following version-specific steps from zero 11 backward.
One of the best courses in the Apache Kafka Series!
Learn Kafka Administration, Kafka Monitoring, Kafka Operations and Kafka Upgrades in this 5+ hours course. Join hundreds of knowledge savvy students into learning some of the most important knowledges that any Kafka administrator should know and master.
----------------------------
This course is important for the following reasons:
Administration for Apache Kafka
> Setup UI tools such as Kafka Manager, ZooNavigator and Kafka Monitor to get a full view of your cluster
> Understand basic operations you can perform with these tools
Monitoring for Apache Kafka
> A Kafka that is not monitored is a ticking time-bomb.
> Setup proper monitoring for Kafka & Zookeeper
> Store your metrics in Prometheus and visualise in Grafana
Operations for Apache Kafka
> Perform the most common and difficult operations demanded in your Apache Kafka Cluster
> Roll Restart your brokers in only one command
> Update Kafka Configurations safely with no downtime
> Rebalance Partitions and Change the Replication Factor of your topics
> Add, Replace and Remove Brokers.
Upgrades for Apache Kafka
> Understand the subtlety of upgrading a Kafka Cluster
> Step by step explanation on upgrading a Cluster with no Downtime
----------------------------
PRE-REQUISITES - THIS IS AN ADVANCED COURSE
> Good Understanding of Kafka & Kafka Setup
> Basic Understanding of AWS Cloud.
> Knowledge of how to SSH into a Server
> Good Understanding of Linux & computing overall
> Strong Ability to use the Command Line
-----------------------------
Hands On Based Course
Note: This course is advanced and little time will be spent on basics. It will incur cost in AWS
This course is the first and only available Kafka Monitoring and Operations on the web. Get it now to become an Apache Kafka expert!
Section outline:
Course Introduction: Let's learn why we need Monitoring & Operations in Apache Kafka
Kafka Quick Setup in AWS: Setup Kafka using AWS CloudFormation. No Knowledge of CloudFormation is required
Kafka Administration Setup: Setup Basic Admin and Monitoring Tools on our Cluster
Kafka Monitoring Setup: Grafana + Prometheus: Setup Kafka & Zookeeper Monitoring using Prometheus and Grafana
Kafka Operations: Learn the most common and difficult Kafka operations
Kafka Cluster Upgrade: Learn the steps to properly upgrade a Kafka Cluster with no downtime
===============================
Instructor
My name is Stephane Maarek, and I'll be your instructor in this course. I teach about Apache Kafka, the Kafka ecosystem and Kafka Certifications with my focus always on helping my students improve their professional proficiencies. I am also the co-founder of Conduktor: an enterprise Apache Kafka platform & UI to help everyone use Kafka.
Throughout my career in designing and delivering these certifications and courses, I have already taught 1,000,000+ students and gotten 350,000+ reviews!
With Apache Kafka becoming much more than a buzzword out there, I've decided it's time for students to properly learn how to be a Kafka professional. So, let’s kick start the course! You are in good hands!
===============================
This Course Also Comes With:
Lifetime Access to All Future Updates
A responsive instructor in the Q&A Section
Links to interesting articles, and lots of good code to base your next applications onto
Udemy Certificate of Completion Ready for Download
This is the course that could improve your career!
Apache Kafka is a skill in high demand and there are not enough people to fulfill all the open positions. You can boost your income, take on new roles and fun challenges. Many of my students are now the Kafka experts of their companies! You can be the next!
I hope to see you inside the course!
=======================
Note: Looking for more advanced Kafka concepts? There are many volumes in the Apache Kafka Series:
Learn Kafka for Beginners v2 (great to start)
Kafka Connect Hands On Learning
Kafka Streams for Data Processing
KSQL on ksqlDB - Hands On!
Kafka Cluster Setup & Administration
Confluent Schema Registry & Kafka REST Proxy
Kafka Security (SSL SASL ACL)
Kafka Monitoring and Operations
Happy learning!