Tips, Expertise, Articles and Advice from the Pro's for Your Website or Blog to Succeed
DataStax has recently released its Astra Streaming, enabling developers to build streaming applications on top of an elastically scalable, multi-cloud messaging and event streaming platform powered by Apache Pulsar. This article will walk you through a short demo that will provide a great starting point for familiarizing yourself with this powerful new streaming service.
Here’s what you will learn:
The leading open-source event streaming platforms are Apache Kafka and Apache Pulsar. For enterprise architects and application developers, choosing the right event streaming approach is critical, as these technologies will help their apps scale up around data to support operations in production.
Everyone wants results faster. We want applications that know what we want, even before we know ourselves. We want systems that constantly check for fraud or security issues to protect our data. We want applications that are smart enough to react and change plans when faced with the unexpected. And we want those services to be continuously available.
One such technology is Apache Pulsar® — an open-source, distributed messaging and streaming platform that’s easy to deploy, simple to scale, and packed with developer-friendly APIs. So the next question is: how can you stream from Pulsar to Apache Cassandra®, the powerful NoSQL database designed to support data-heavy applications in the cloud?
Join our beginner-friendly Pulsar workshop on YouTube and learn how to connect Pulsar with Cassandra for streaming! In this post, we’ll set the scene with an introduction to Pulsar and guide you through four hands-on exercises where you’ll use these free, cloud-native technologies: Katacoda, Kesque, GitPod, and DataStax Astra DB. Each exercise will also be linked to the step-by-step instructions on the DataStax Developers GitHub wiki.
Apache ActiveMQ is an open-source message broker. It is one of the most popular Apache open source products and supports multiple industry-standard connectivity protocols. Clients can choose among multiple different languages and platforms for connectivity to ActiveMQ. Apache ActiveMQ provides client libraries in C++, Python, Java, C, etc. for developing applications. Some of the most popular protocols supported by ActiveMQ are AMQP, STOMP, MQTT, and JMS. ActiveMQ supports a wide variety of messaging use cases to meet almost all the needs of a production-grade messaging system.
AWS provided Amazon MQ is a managed service that can be used for Apache ActiveMQ or RabbitMQ. Amazon MQ service makes the management, configuration, deployment, and usage of these two message brokers easy on AWS. Amazon MQ makes the lifecycle management and operation of these services easy for an IT organization as this is a fully managed service provided by AWS. Code written to connect to on-prem Apache ActiveMQ can be transparently migrated to use ActiveMQ using AWS.
Large-scale implementations of any system, such as the event-streaming platform Apache Kafka, often involve customizations and tools and plugins developed in-house. When it’s time to transition from one system to another, the task can become complicated, drawn-out, and error-prone. Often the benefits of an alternative system (which can include significant cost savings and other efficiencies) are outweighed by the risks and costs of migration. As a result, an organization can end up locked into a suboptimal situation, footing a bigger bill than necessary and missing out on modern features that help move the business forward faster.
These risks and costs can be mitigated by making the transition process iterative, breaking off the vendor lock-in in small, manageable steps, and avoiding the "big bang" switch that often results in delayed delivery and increases the cost of running two systems in parallel for A|B testing.
The concept of message queues is fundamental in message-oriented middleware (MOM) and a constant topic of discussion for Solace users. Generally speaking, a queue is defined as a storage area where a message is stored until it is consumed by an application, or it expires. A queue provides the guarantee that the message will never be lost even if the consuming application is unavailable or if the message broker crashes.
Solace endpoints are objects created on the event broker to persist messages. There are two types of endpoints: a queue endpoint (usually just called a queue) and a topic endpoint. I detailed the differences here if you want to learn more, and in this post, I’ll explain the two consumer patterns you have to choose from, how they differ, and when you should use each.
One of the most frequent questions I get asked in the community is “What is the difference between a queue and a topic endpoint?” While both queues and topic endpoints persist messages, it’s important to understand what they are, how they’re different, and when each one should be used.
Solace endpoints are objects created on the event broker to persist messages. There are two types of endpoints: queue endpoints (usually just called queues) and topic endpoints.
In this post, I’ll explain how pub/sub messaging pattern implementation differs in Kafka and Solace PubSub+.
Publish-Subscribe, also known as pub/sub, is a popular messaging pattern that is commonly used in today’s systems to help them efficiently distribute data and scale among other things. The pub/sub messaging pattern can be easily implemented through an event broker such as Solace PubSub+, Kafka, RabbitMQ, and ActiveMQ.
Partitioning a domain is a useful way to achieve scalability of a system. The idea behind partitioning is that instead of putting everything in a single place, you divide the dataset or the work into multiple places based on some attribute, which is usually the identifier of the entity.
The division allows us to spread the storage and/or the processing to multiple machines or containers, and allows the horizontal scaling we were seeking to get.
Zato publishes/subscribe message queues and topics offer several ways to gain insight into the inner workings of all the components taking part in message delivery and this article presents an overview of the mechanisms available.
In our previous overview of the most popular messaging systems, we were talking about Apache Kafka vs. RabbitMQ. Now, let's take a look at the less powerful, but still very helpful, message brokers. We will consider the pros and cons of ActiveMQ and Redis Pub/Sub. Although these solutions aren't very suitable for processing big data, they provide a strong basis for creating small business analytics tools.
Pricing: free
HA - High Availability
SAN - Storage Area Network