Building a Simple Kafka Client for the Web and Desktop

Kafka is an open-source platform that is used for event distribution, logging, and streaming. Distributions from Confluent and other vendors provide a web UI for broker management and client tasks, while those from Apache Kafka provide just command-line tools. Also, for most of the requirements, a simple client that publishes and consumes events or messages from the topic is sufficient. The main objectives of building such a client were:

  • Follow a web development approach 
  • Have a simple and elegant UI 
  • Build a native desktop app

Kafka client UI

Giving a Voice to Citizen Integrators

With the emergence of APIs, a simplified approach to building integrations emerged. Business users found platforms offering them a user interface that did not need them to know about protocols, adapters or other technical aspects to build integrations. The platforms that offer these capabilities are commonly known as No-code platforms and the concept is not just limited to integration but available for application development too. The business users who build such integrations are called as Citizen Integrators and the concept of opening up development is known as Democratization of Development. The platforms apply standard integration patterns to the integrations built by Citizen Integrators to ensure that there are no proliferation of interfaces or "Spaghetti architectures" springing up. 

While APIs have solidified themselves as the building block for enterprise architectures, voice based assistants have gained acceptance in the consumer space, especially with mobile and smart devices. It is now common to ask questions and get answers from the assistants, be it, the temperature, traffic, news, entertainment or virtually anything that their underlying platforms support. So, how would it be if Citizen Integrators use voice assistants to build integrations? This would make it even simpler to design integrations. There would be no need to even drag and drop or click or poke around. The No-code platform could add the voice capabilities and support various voice assistants. The Citizen Integrator would ask or dictate to the voice assistant on what data is required and the assistant would get it instantly or by asking a few additional questions. Let us look at an example where a Citizen Integrator (CI) converses with a voice assistant (VA) 

Looking at the Reactive Manifesto From an Enterprise Integration Perspective

With the adoption of Cloud and Cloud-native architectures, application integration has changed. Support for multiple devices, channels, and making the architecture responsive is one of its key tenets. APIs are the key to this. Being application-agnostic and based on web standards, APIs are invokable across channels and serve to decouple the components. An API Management layer helps in security and performance through configurable policies.

There is no longer a central middleware component that does the integration, rather the architecture is distributed and loosely coupled, helping realize the goal of it being elastic. Containerized microservices and integration components are invokable separately resulting in them being more resilient than if they were a single monolith.

Understanding Microservices Choreography Using RabbitMQ and Node.js

A colleague and I had discussed the choreography of microservices a while ago and the topic came up for discussion again recently. In order to ensure that I understood the nuances correctly, I decided to write some code to implement the approach. Here are the tools/technologies that I chose:

  • Node.js for building the microservices.
  • Restify for exposing Restful APIs.
  • CloudAMQP — a RabbitMQ as a service offering as the integration bus.
  • amqp.node — a Node.js client library for RabbitMQ.

CloudAMQP offers a free tier good enough for learning and trying things out. It also comes with an easy to use management console that gives good statistics. The tutorials and articles are useful too. There is a host of options for the Node.js RabbitMQ client libraries. I tried a few and then settled with amqp.node as the RabbitMQ site had their tutorials based on it. The library comes in two flavors, one with callbacks and the other with promises. Being comfortable with callbacks, I opted for it.