Creating a Scalable MQTT Environment


MQTT or Message Queue Telemetry Transport is a messaging protocol for the internet of things or IoT and is lightweight enough to be supported by the smallest devices. However, the protocol is robust enough to ensure that important messages get to their destinations every time. With MQTT devices such as smart energy meters, connected cars, trains, satellite receivers, and personal health care devices can talk to each other as well as with other systems or applications.

MQTT is a light-weight IoT messaging protocol over TCP/IP designed for large-scale telemetry environments with remote locations where a small code footprint is required and/or network bandwidth is limited. It operates as a producer-consumer paradigm, where many “Publishers” (for example, sensors) send out messages to a small set of “Subscribers” (for example, client applications), which consume the messages. This Publish-Subscribe arrangement is mediated by a message broker. The broker is responsible for distributing messages from the publishers to interested subscribers based on the topic of each message. The client application can subscribe to specific topics from certain devices. They can use the messages for monitoring or managing the devices. Each subscriber can receive messages from many sensors. For purposes of configuration and management, a subscriber can also act as a publisher and vice versa.

There are many sensors available from a variety of vendors. Each of them has its own identity, type, address etc. Each sends out messages on different topic and at a different frequency. The broker receives messages from all connected sensors and sends those to the right client application(s) depending on the topic.

MQTT Guiding Principles

To ensure correct functioning, both broker and Subscriber applications need to be tested for scalability and performance before the deployment in an organization. They need to be tested against the expected number of sensors sending a range of messages at different frequency. How can an organization be certain that its new broker or client application will live up to performance requirements? This can be very difficult to achieve. With the number of devices available, it is expensive to try to maintain a lab with even an adequate sampling of these devices. Also, if a sensor is supposed to send out critical information, it is very important that receiving and processing of those alarms is tested thoroughly. In real-time situations reaction time is critical, for example if temperature reaches a boiling point, or pressure is above the admissible level. Now imagine a disastrous situation where thousands of sensors are publishing these types of alarms. A comprehensive disaster preparedness and recovery plan is required. The application should be tested to handle those correctly. It is also necessary for staff training, to ensure that the proper process is in place to handle and to recover from a disaster in a timely manner with least impact on production.

The Right Approach

It is not cost effective to have a large scalable lab just for testing. The best alternative is the use of a MQTT simulator. A simulator, like, MIMIC Simulator, can create a real-world test lab with thousands of IoT sensors and devices. It allows application vendors to design, develop and test their Brokers, Load balancers and client applications using secured connections, in a virtual and scalable network environment. Using a simulator, they can assure their customers that their applications will be able to handle the scales of connections, topics and varieties of messages to work properly when deployed across heterogeneous environments. The organizations deploying those applications can also try variety of pathological scenarios before deploying them.

MIMIC Simulator can help create a virtual IoT Smart City in their test lab. Each developer, tester or staff member can have their own private city simulation. They are able to simulate many thousands of MQTT based Publishers and Subscriber, each with their own IP addresses Port, Client ID and Authentication. Each Publisher can publish multiple, unique topics. Each Subscriber can subscribe to multiple topics, including wild-card topics.

Realistic testing of the application against the simulation early on significantly reduces problems found in later stages of testing. In addition, it is easy to do the future planning by creating a lab with current and new device simulations, instead of waiting for a new device purchase. It is easy to configure sensors to generate the type of alarms at any needed frequency. All these can be done without running around the lab and worrying if all the devices are connected and configured properly. Imagine the time this would save in the testing procedure! With a simulator, application developers can enhance the scope of regression testing. Testing proceeds faster and companies can have higher confidence in the level of software testing, and software products can be released to customers more quickly. And in this way companies can win the time-to-market battle with the competition while assuring a high level of quality!


By : Pankaj Shah

Pankaj Shah is the CEO of Gambit Communications. The company is a leader in network and IoT simulation tools that enhance the productivity of management software developers and enterprise users while lowering their costs. MIMIC Simulator is a modular family of simulators used by leading networking vendors for use in applications from development and testing to operator training and disaster simulations.


Leave a Reply

Your email address will not be published. Required fields are marked *