Posted on

Multi-Cluster Kubernetes Management and Access

As cloud and Kubernetes have become a standard, security remains one of the top inhibitors to modern application development. To reduce security risks, organizations can’t manage access control on a cluster-by-cluster basis. And not finding a scalable approach leads to misconfigurations, vulnerabilities, and failed compliance audits.

Let us travel back in time and picture a fort. Forts were huge with massively thick walls, doors, watch towers and a moat to protect them from attacks. There were several layers of defense to keep attackers at bay. An attacker might swim across the moat but still had to climb the high walls before entering the fort. Thus, an attacker might compromise a single layer, but having several layers makes it difficult for an attacker to enter the fort.

Posted on

When To Use Apache Camel vs. Apache Kafka?

Should I use Apache Camel or Apache Kafka for my next integration project? The question is very valid and comes up regularly. This blog post explores both open-source frameworks and explains the difference between application integration and event streaming. The comparison discusses when to use Kafka or Camel when to combine them, and when not to use them at all. A decision tree shows how you can quickly qualify one for the other.

The History of Application Integration and Event Streaming

My personal history and experience in application integration and event streaming are the following. It shows my background and how I see the integration and data streaming markets.

Posted on

AWS Serverless Lambda Resiliency: Part 1

In this series of articles, we address the patterns for the resilience of cloud-native serverless systems.

This article covers an introduction to cloud-native and serverless resiliency and different scenario approach patterns, and part 2 will cover further scenario patterns.

Posted on

Leverage Docker to Produce Classic Deployment Artifacts

Traditionally, deployment artifacts were types of archives that were transferred to the target environment and installed there. They could be simple .zip archives with binaries, Java's .jar or .war files, or simply .exe executables, among others. It is worth saying that this approach implied preliminary preparation, like installing and configuring all software dependencies.

The situation changed when Docker appeared. Moreover, the principle changed: now we operate Docker images like some self-sufficient sealed units containing everything the app needs to work. But we have to admit that both principle and technology are relatively new to the industry. Many companies still rely on a VM-based approach and classic deployment artifacts. And it's quite clear why they do this: infrastructure is already bought; Ops teams are trained on how to deploy, manage and monitor applications; and so forth. In two words, this journey seems to be very long and tough. To mitigate this, we can use the well-known step-by-step method with gradual adjustments to the existing delivery pipeline.

Posted on

Extend Kubernetes Service Discovery With Stork and Quarkus

In traditional monolithic architecture, applications already knew where the backend services existed through static hostnames, IP addresses, and ports. The IT operation team maintained the static configurations for service reliability and system stability. This Day 2 operation has significantly changed since microservices began running in distributed networking systems. The change happened because microservices need to communicate with multiple backend services to improve load balancing and service resiliency.

The microservices topology became much more complex as the service applications were containerized and placed on Kubernetes. Because the application containers can be terminated and recreated anytime by Kubernetes, the applications can't know the static information in advance. The microservices don't need to be configured with the static information of the backend applications because Kubernetes handles service discovery, load balancing, and self-healing dynamically and automatically.

Posted on

Taking Your Database Beyond a Single Kubernetes Cluster

Global applications need a data layer that is as distributed as the users they serve. Apache Cassandra has risen to this challenge, handling data needs for the likes of Apple, Netflix, and Sony. Traditionally, managing data layers for a distributed application was handled with dedicated teams to manage the deployment and operations of thousands of nodes — both on-premises and in the cloud.

To alleviate much of the load felt by DevOps teams, we evolved a number of these practices and patterns in K8ssandra, leveraging the common control plane afforded by Kubernetes (K8s) There has been a catch though — running a database (or indeed any application) across multiple regions or K8s clusters is tricky without proper care and planning up front.

Posted on

Kubernetes Performance Tuning: Make the Most of Your Clusters

Five stars

Image Source

Why Is Kubernetes Performance Tuning Needed?

As Kubernetes becomes a basic infrastructure for many organizations, performance tuning for Kubernetes clusters is becoming more important. Kubernetes is a highly scalable open-source platform for orchestrating containerized workloads in server environments. It enables declarative configuration and automation of computing resources.

Posted on

Portfolio Architecture Examples: Infrastructure Collection

This article is a continuation of a series of posts about our project named Portfolio Architectures. A previous post, Portfolio Architecture Examples: Healthcare Collection, begins with a project overview, introduction, and examples of tooling and workshops available for the project.  You may want to refer back to that post to gain insight into the background of Portfolio Architectures before reading further.

Infrastructure Collection

The collection featured today is centered around infrastructure architectures. There are currently seven architectures in this collection and we'll provide a short overview of each, leaving the in-depth exploration as an exercise for the reader.

Posted on

Kafka for Real-Time Replication Between Edge and Hybrid Cloud

Not all workloads should go to the cloud! Low latency, cybersecurity, and cost-efficiency require a suitable combination of edge computing and cloud integration. This blog post explores architectures and design patterns for software and hardware considerations to deploy hybrid data streaming with Apache Kafka anywhere. A live demo shows data synchronization from the edge to the public cloud across continents with Kafka on Hivecell edge hardware and serverless Confluent Cloud.

Not Every Workload Should Go Into the Cloud

Almost every company has a cloud-first strategy in the meantime. Nevertheless, not all workloads should be deployed in the public cloud. A few reasons why IT applications still run at the edge or in a local data center:

Posted on

Not Just Buckets: Are You Aware of ALL Your Public Resources?

Public exposure to resources is a point of dread for many security practitioners. Public resources are low-hanging fruit for attackers seeking to access sensitive information or manipulate an activity -- or even deny the availability of mission-critical resources.

One well-known culprit for exposing resources is AWS built-in mechanisms. While misconfiguring them is a common and legitimate concern for security practitioners charged with protecting AWS environments, we often see another mechanism that may be even more misconfiguration-prone: resource-based policies.

Posted on

What is Data Ingestion? The Definitive Guide

What Is Data Ingestion?

Data ingestion is an essential step of any modern data stack. At its core data ingestion is the process of moving data from various data sources to an end destination where it can be stored for analytics purposes. This data can come in multiple different formats and be generated from various external sources (e.g., website data, app data, databases, SaaS tools, etc.)

Why Is Data Ingestion Important?

The data ingestion process is important because it moves data from point A to B. Without a data ingestion pipeline, data is locked in the source it originated in and this isn’t actionable. The easiest way to understand data ingestion is to think of it as a pipeline. In the same way that oil is transported from the well to the refinery, data is transported from the source to the analytics platform. Data ingestion is important because it gives business teams the ability to extract value from data that would otherwise be inaccessible.

Posted on

Why Distributed Apps Need Dependency Management

Distributed cloud applications (aka microservices) have introduced an enormous amount of complexity into the design and operation of cloud software. What used to manifest itself as complexity hidden within a single process or runtime now finds itself spread across tens or hundreds of loosely coupled services. While all of these services can use different languages and can scale independently from one another, the distributed nature can often make the app as a whole hard to navigate, hard to deploy, and hard secure.

This new complexity makes it increasingly difficult to manage and contribute to cloud-native applications and beckons questions as to how we can maintain healthy cloud software. How can we take advantage of the benefits of service-oriented design without introducing friction and cost elsewhere?

Posted on

The Internet of Things in Solutions Architecture

As internet connectivity increases, an increasing number of small devices now exist with small memory and compute capacities. These sensors connect various physical entities, such as your home alarm, thermal sensors, and car. The data from millions of these connected devices needs to be collected and analyzed. For example, weather data collected from multiple sensors can be utilized to forecast weather for wind energy and farming. There are billions of connected devices in homes, factories, oil wells, hospitals, cars, and thousands of other places that are fueling digital transformation, generating huge volumes of data and growing exponentially. 

As IoT has become very common in the manufacturing industry for handling machine data and optimizing production, the concept of Industrial IoT (IIoT) was developed. Let’s learn more about this now.

Posted on

Kafka Event Exchange Between Local and Azure

While it may not be a daunting task to set up Kafka on a local machine or within a particular network and produce/consume messages, people do face challenges when they try to make it work across the network.

Let’s consider a hybrid scenario where your software solution is distributed across two different platforms (say AWS and Azure or on-premise and Azure), and there is a need to route messages from Kafka cluster hosted on one platform to the one hosted on another. This could be a valid business scenario wherein you are trying to consolidate your solution with one cloud platform, and in the interim, you need to have this routing in place till you complete your migration. Even in the long term, there may be a need to maintain solution across multiple platforms for various business and technical reasons.

Posted on

How to Configure Apache ActiveMQ on AWS

Apache ActiveMQ is an open-source message broker. It is one of the most popular Apache open source products and supports multiple industry-standard connectivity protocols. Clients can choose among multiple different languages and platforms for connectivity to ActiveMQ. Apache ActiveMQ provides client libraries in C++, Python, Java, C, etc. for developing applications. Some of the most popular protocols supported by ActiveMQ are AMQP, STOMP, MQTT, and JMS. ActiveMQ supports a wide variety of messaging use cases to meet almost all the needs of a production-grade messaging system.

Amazon MQ

AWS provided Amazon MQ is a managed service that can be used for Apache ActiveMQ or RabbitMQ. Amazon MQ service makes the management, configuration, deployment, and usage of these two message brokers easy on AWS. Amazon MQ makes the lifecycle management and operation of these services easy for an IT organization as this is a fully managed service provided by AWS. Code written to connect to on-prem Apache ActiveMQ can be transparently migrated to use ActiveMQ using AWS.

Posted on

Introduction to Kubernetes for Application Developers

Kubernetes is a highly configurable and complex open-source container orchestration engine. Therefore, it is very easy to feel completely overwhelmed when learning it. The goal of this article is to present the very basic concepts at the core of it while keeping the focus on the development side.

Concepts

First, let’s start with some concepts we will play with in this article.

Posted on

3 Best Practices To Make Cloud Migration Easier

Cloud migration. It’s a term that comes up in most enterprise conversations at least once. While the term represents the practice of moving from on-premises infrastructure to cloud infrastructure, what is meant by “cloud migration” has evolved. Cloud migration is no longer as simple as moving from on-prem servers to AWS EC2. It could include moving to managed databases or API gateways, or maybe you need AWS for some workloads and Azure for others. Perhaps you’re a financial or public sector organization, and you need a private cloud. Or maybe you need to meet special regulatory requirements.

In this article, we’re going to look at three best practices for making cloud migration easier for your enterprise:

Posted on

Portfolio Architecture Examples: Edge Collection

This article is a continuation of a series of posts about our project named Portfolio Architectures. A previous post, Portfolio Architecture Examples: Healthcare Collection, begins with a project overview, introduction, and examples of tooling and workshops available for the project.  You may want to refer back to that post to gain insight into the background of Portfolio Architectures before reading further.  

Edge Collection

The collection featured today focuses on edge computing architectures. There are currently five architectures in this collection, and we'll provide a short overview of each, leaving the in-depth exploration as an exercise for the reader.