The ever-increasing scale of IoT device connections and deployments requires IoT messaging platforms to be massively scalable and robust at scale. To stress-test the scalability of our open-source MQTT messaging broker, EMQX, we established 100 million MQTT connections to a 23-node EMQX cluster.
In this test, each MQTT client subscribed to a unique non-wildcard topic. When publishing, we chose a 1-to-1 publisher/subscriber topology for a low, constant publishing rate of 90k messages per second, then provoked a burst of messages and reached 1M messages processed per second at peak. We also compared how the increasing cluster size affected the maximum subscription rate using two different database backends: running in RLOG mode and plain Mnesia. Here we will detail our setup and some of the challenges we faced along the way.