Posted on

OPC-UA, MQTT, and Apache Kafka: The Trinity of Data Streaming in IoT

In the IoT world, MQTT (Message Queue Telemetry Transport protocol) and OPC-UA (OPC Unified Architecture) have established themselves as open and platform-independent standards for data exchange in Industrial Internet of Things (IIoT) and Industry 4.0 use cases. Data streaming with Apache Kafka is the data hub for integrating and processing massive volumes of data at any scale in real-time. This article explores the relationship between Kafka and the IoT protocols, when to use which technology, and why sometimes HTTP/REST is the better choice. The conclusion explores real-world case studies from Audi and BMW.

Industry 4.0: Data Streaming Platforms Increase Overall Plant Effectiveness and Connect Equipment

Machine data must be transformed and made available across the enterprise as soon as it is generated to extract the most value from the data. As a result, operations can avoid critical failures and increase the effectiveness of their overall plant.

Posted on

Why You Need Both IT and OT Cybersecurity

With cyberattacks becoming more prominent and severe in today’s society, it’s no longer sufficient to focus on only information technology (IT) or operational technology (OT) to keep systems safe. While IT concerns an organization’s online infrastructure, OT extends to the connected physical devices it uses, such as smart sensors on manufacturing equipment.

You need cybersecurity for both to maintain resiliency across an organization. Here’s a closer look at why both are necessary today and for the foreseeable future.

Posted on

Kafka for Condition Monitoring and Predictive Maintenance in Industrial IoT

The manufacturing industry is moving away from just selling machinery, devices, and other hardware. Software and services increase revenue and margins. A former cost center becomes a profit center for innovation. Equipment-as-a-Service (EaaS) even outsources the maintenance to the vendor. This paradigm shift is only possible with reliable and scalable real-time data processing leveraging an event streaming platform such as Apache Kafka. 

This post explores how the next generation of software for Condition Monitoring and Predictive Maintenance can help build new innovative products and improve the OEE for customers.

Posted on

Apache Kafka in the Public Sector — Part 4: Energy and Utilities

The public sector includes many different areas. Some groups leverage cutting-edge technology, like military leverage. Others like the public administration are years or even decades behind. This blog series explores how the public sector leverages data in motion powered by Apache Kafka to add value for innovative new applications and modernize legacy IT infrastructures. This is part 4: Use cases and architectures for energy, utilities, and smart grid infrastructure.

Blog series: Apache Kafka in the Public Sector and Government

This blog series explores why many governments and public infrastructure sectors leverage event streaming for various use cases. Learn about real-world deployments and different architectures for Kafka in the public sector:

Posted on

Data Management for Industrial IoT

Advances in technology continue to drive change in industrial operations. As businesses seek to leverage these advances, it’s important to understand how different technologies impact their operations. Data generated from sensors and applications have the potential to dramatically affect industrial processes; they generate a lot of data. Businesses need to understand the characteristics and shape of that data, as well as how to effectively analyze and apply it to drive improvements. This Refcard dives into data management for industrial IoT applications, including industry 4.0, time-series data, and more.
Posted on

How to Accelerate Hyper-Automation With Industrial IoT

Many enterprises have already adopted business process automation (BPA) to improve efficiency and reduce human error. However, by and large, industrial automation is fragmented – it applies to specific business aspects but is not used across the entire organization yet. The key to expanding automation across the company is hyper-automation.

In 2020, Gartner named hyper-automation the # 1 technology trend of the year. So what is it, and how can large and small businesses benefit from it? This article will walk you through specific areas where industrial IoT integration can help accelerate your pace of hyper-automation.

Posted on

Apache Kafka for Industrial IoT and Manufacturing 4.0

This post explores use cases and architectures for processing data in motion with Apache Kafka in Industrial IoT (IIoT) across verticals such as automotive, energy, steel manufacturing, oil&gas, cybersecurity, shipping, logistics. Use cases include predictive maintenance, quality assurance, track and track, real-time locating system (RTLS), asset tracking, customer 360, and more. Examples include BMW, Bosch, Baader, Intel, Porsche, and Devon.

Why Kafka Is a Key Piece of the Evolution for Industrial IoT and Manufacturing

Industrial IoT was a mess of monolithic and proprietary technologies in the last decades. Modbus, Siemens S7, SCADA, and similar "concepts" controlled the industry. Vendors locked in enterprises by intentionally building incompatible products without open interfaces. These systems still run on Windows XP or similar non-supported outdated operating systems and without security in mind.

Posted on

Apache Kafka for the Connected World: Vehicles, Factories, Cities

The digital transformation enables a connected world. People, vehicles, factories, cities, digital services, and other "things" communicate with each other in real-time to provide a safe environment, efficient processes, and a fantastic user experience. This scenario only works well with data processing in real-time at scale. This blog post shares a presentation that explains why Apache Kafka plays a key role in these industries and use cases but also to connect the different stakeholders.

Software is Changing and Connecting the World

Event Streaming with Apache Kafka plays a massive role in processing massive volumes of data in real-time in a reliable, scalable, and flexible way integrating with various legacy and modern data sources and sinks.

Posted on

Smart Factory with Apache Kafka and 5G Campus Networks

The Fourth Industrial Revolution (also known as Industry 4.0) is the ongoing automation of traditional manufacturing and industrial practices using modern smart technology. Event Streaming with Apache Kafka plays a key role in processing massive volumes of data in real-time in a reliable, scalable, and flexible way of integrating with various legacy and modern data sources and sinks. This blog post explores Apache Kafka's relationship to modern telco infrastructures that leverage private 5G campus networks for Industrial IoT (IIoT) and edge computing.

Event Streaming with Kafka at the Disconnected Edge

Apache Kafka is the new black at the edge.

Posted on

Apache Kafka Is NOT Real-Time!

Is Apache Kafka really real-time? This is a question I get asked every week. Real-time is a great marketing term to describe how businesses can add value by processing data as fast as possible. Most software and product vendors use it these days, including messages frameworks (e.g., IBM MQ, RabbitMQ), event streaming platforms (e.g., Apache Kafka, Confluent), data warehouse/analytics vendors (e.g., Spark, Snowflake, Elasticsearch), and security / SIEM products (e.g., Splunk). This blog post explores what "real-time" really means and how Apache Kafka and other messaging frameworks accomplish the mission of providing real-time data processing.

Definition: What Is Real-Time?

The term "real-time" is not easily defined. However, it is essential to define it before you start any discussion about this topic.

Posted on

How to Connect Azure IoT Hub and CrateDB Cloud to Ingest IoT Sensor Data

This article will describe how to launch a CrateDB cluster on Azure, connect it to Azure IoT Hub, and test it by ingesting simulated sensor data using an Azure IoT Solution Accelerator.

Step #1: Simulating Sensor Data

It’s necessary to first understand the type of data your IoT application will produce in order to accurately mimic this information for testing purposes. Smart factories, for example, will have myriad sensors that collect data in a variety of structures. CrateDB makes it possible to model different data structures in a single table through the use of dynamic objects, which can be queried to an arbitrary depth (this is not a recommended practice for production, but is helpful within the simple confines of this demonstration).

Posted on

Train Your Dragons: 3 Quick Tips for Harnessing Industrial IoT Value

How to train your [IIoT] dragons...

While industrial IoT projects have many stakeholders across the enterprise, three key roles are most responsible for decisions impacting the system's long-term financial potential. Each plays a part in training the dragons blocking their organization's path to leadership in a connected world. How can these digital project leaders navigate the real-world challenges both inside and outside their enterprise and bring about successful industrial evolution?

You may also like: Top Three Industrial IoT Implementation Challenges

First, there's the engineering manager. Their task is to deliver a functional industrial IoT system. Sometimes, their teams include enterprise software developers, though are more commonly made up of talented embedded systems and controls engineers with little cloud experience.

Posted on

5 Greatest TED Talks on IoT

Here are our favorite IoT TED Talks. While they are a few years old, a lot of the topics are still relevant today!

We've gathered five of our favorite TED (Technology, Entertainment, Design) talks highlighting various aspects of the Internet of Things. The speakers below cover a wide range of topics including how to design clothing and architecture for IoT, exploring IoT's potential applications while addressing security threats, and providing insight into the exponential advancement of technology and what we can expect from the future of the Internet.

You may also like: DZone's Guide to IoT: Applications, Protocols, and Best Practices

Dr. John Barrett: The Internet of Things

Both the Head of Academic Studies at the Nimbus Centre for Embedded Systems Research at Cork Institute of Technology (CIT) and Group Director of the Centre's Smart Systems Integration Research Group, Dr. John Barrett's research is focused on integrating smart systems in the physical world. With nearly 30 years of knowledge in packaging and systems integration, plus over 100 publications in topics related to his field, he gives the definitive, all-inclusive TED Talk on the Internet of Things.

Posted on

Writing About IoT [Prompts]

Ever struggle with what to write? No worries, we've got you covered. Here's a list of IoT prompts and article ideas to help cure even the worst cases of writer's block. So, take a moment, check out the prompts below, pick one (or two!), and get to writing.

Also, please feel free to comment on this post to bounce around ideas, ask questions, or share which prompt(s) you're working on. 

Posted on

Apache Kafka, KSQL, and Apache PLC4X for Industrial IoT and Automation

Learn more about IIoT automation with Apache Kafka, KSQL, and Apache PLC4X

Data integration and processing is a huge challenge in Industrial IoT (IIoT, aka Industry 4.0 or Automation Industry) due to monolithic systems and proprietary protocols. Apache Kafka, its ecosystem (Kafka Connect, KSQL), and Apache PLC4X are a great open-source choice to implement this IIoT integration end-to-end in a scalable, reliable, and flexible way.

This blog post covers a high-level overview of the challenges and good, flexible architecture to solve the problems. In the end, I share a video recording and the corresponding slide deck. These provide many more details and insights.

Posted on

What Is High Cardinality?

If you are working with a database, especially with time series data, then you have likely faced the challenge of handling high cardinality data.

In particular, time series high cardinality is a common problem in industrial IoT (e.g., manufacturing, oil and gas, utilities, etc.), as well as some monitoring and event data workloads.

Posted on

Prepare Your Enterprise for Industrial IoT Success

Throughout our ten years of working with equipment manufacturers to connect, collect, and integrate operational data with enterprise systems, we’ve seen many trends impacting industrial IoT project success. The world has finally moved beyond most of the technological limitations for building innovative solutions. All the necessary tools exist to create connected product systems that perform as expected. They work. Now there’s a new trend, and it’s not a good one. We’re seeing business teams at equipment manufacturers telling engineering managers and IT leaders to evaluate and choose an IoT platform for the company. Run an online demo. Read API documentation. Build a proof of concept. Compare prices. Most of these projects never see the commercial light of day. They get stuck. Why? They get stuck because this approach to digital transformation is completely backward.

Evaluating the IoT Platform Problem

We’ve said technology isn’t the problem. Here’s the reality. The right technology for your system is available today. When used correctly by experienced teams, it will produce your desired outcomes. This is a well-charted territory. You can have remote monitoring with predictive maintenance and integrate machine data with your business workflows. These are solved challenges.

Posted on

A Deep Dive Inside Industrial IoT (IIoT)

Image titleJaded sensors and real-time analytics are revolutionizing construction, transportation, and logistics, from asset safety to increased condition monitoring and proactive deployment of maintenance crews before equipment fails.

From the manpower efforts to the digital technologies, Internet of Things (IoT) has changed the way we interact with the world around us, bringing new opportunities from every challenge and new risks to the most personal areas of our lives. But the real revolution among various industries is going on, from heavy industry and agriculture to city infrastructure and medical care to the healthcare industry.