Recently I’ve encountered a bug which grinded my .NET Core application to a halt. Obviously there were no weird log messages preceding the outage so the only way for me to go was utilizing the diagnostic toolkit. …

We’ve already explored the Producer API that Confluent.Kafka provides in the previous article. As the next logical part, let’s see how to configure message consumption out of Kafka topic. Again, it is as simple as shown in the code snippet below: The whole code base could be found in…

Let’s explore APIs which Kafka provides for its producers, consumers and admins. I’m C# developer so I’ll provide examples using Confluent.Kafka library. In fact, internally it delegates calls to high-performant library librdkafka. From my experience, it’s easier to grasp new technology if it’s applied to a meaningful domain. Let’s imagine…

Once I wanted to play with Kafka, I couldn’t find a basic Docker setup to run the cluster and view topics, partitions and messages. I suspect you have the same issue right now, so let me present you a docker-compose file which will run Kafka cluster with one instance of Zookeeper and three brokers. Also we’ll add a special application to view Kafka internals — Kafdrop. Now you can you docker-compose tool to spin up the components and open http://localhost:9000/ and browse Kafka internals:

Once we’ve discussed the main concepts Kafka relies on, we can dive into components and their roles in the system. Message Message is a unit which flows throughout the system. Its typical life cycle is described as follows: Created by a producer Published to a topic by the producer Stored in…

Kafka offers a reliable, scalable and high-performant solution for implementing a Pub-Sub model with asynchronous communication between services. Let’s take a closer look at a Pub-Sub model and async communication to get the point. Pub-Sub model The Pub-Sub model is a pattern which allows us to decouple a caller and a receiver…

Fear of failures is a common reaction to new beginnings that blocks our personal and professional growth. For example, we may be terrified of new responsibilities so that we never think of a promotion. Or we may not explore new technologies/frameworks because of a possibility of not succeeding. Luckily, there…

Pattern is a general reusable solution of a common problem. Sounds like a good thing to apply immediately? Wait a minute, friend! Be aware that patterns are double-edged weapons. The first edge On the one hand, patterns give us significant benefits. Firstly, they improve our communication with other developers since we are able…