Welcome to A Comprehensive Guide to MQTT, your ultimate resource for mastering the Message Queuing Telemetry Transport (MQTT) protocol.

This story is free for everyone: Free Link

MQTT is a lightweight and efficient messaging protocol designed for constrained devices and low-bandwidth, high-latency, or unreliable networks. It follows a publish-subscribe architecture, making it an ideal choice for IoT (Internet of Things), industrial automation, home automation, and real-time messaging applications.

This guide is structured into 7 modules covering fundamental to advanced MQTT concepts, hands-on implementations, and real-world applications.

In this first section, we’ll dive into the fundamentals of MQTT, starting with the most basic question: What is MQTT?

Pre-requisites:

1. Basic networks knowledge (stuff like TCP/UDP, Ports etc.)
2. Programming basics (any language will do)

Table of contents:

1. What is MQTT?
2. Why MQTT?
3. Comparison with HTTP, CoAP and AQMP
4. MQTT architecture overview
5. MQTT protocol versions
6. Hands-on-001: Setting up MQTT

What is MQTT?

MQTT, which stands for Message Queuing Telemetry Transport, is a lightweight, open, and simple messaging protocol designed for constrained devices and low-bandwidth, high-latency, or unreliable networks. It was originally developed in 1999 by Dr. Andy Stanford-Clark of IBM and Arlen Nipper of Arcom (now Cirrus Link) to monitor oil pipelines over satellite connections. Since then, MQTT has evolved into one of the most widely used protocols in the Internet of Things (IoT) ecosystem.

At its core, MQTT is a publish-subscribe (pub/sub) protocol that enables efficient communication between devices. It allows devices (called clients) to send (publish) or receive (subscribe to) messages (called topics) through a central server known as the broker. This decoupled architecture makes MQTT highly scalable and flexible, ideal for IoT applications where thousands or even millions of devices need to communicate seamlessly.

Why MQTT?

MQTT was designed with specific goals in mind, making it ideal for IoT and other constrained environments. Here’s why MQTT stands out:

1. Lightweight and Efficient:
   MQTT’s small message headers and minimal overhead make it perfect for low-bandwidth, high-latency networks. It’s designed to work on devices with limited processing power and memory.
2. Low Power Consumption:
   MQTT’s efficient communication model reduces power usage, making it suitable for battery-operated devices like sensors and wearables.
3. Reliable Message Delivery:
   With three levels of Quality of Service (QoS), MQTT ensures messages are delivered reliably, even in unstable network conditions.
4. Scalability:
   MQTT’s publish-subscribe architecture allows it to handle thousands or even millions of devices simultaneously, making it ideal for large-scale IoT deployments.
5. Real-Time Communication:
   MQTT enables real-time, bi-directional communication, allowing devices to send and receive data instantly.
6. Decoupled Architecture:
   Publishers and subscribers don’t need to know each other’s existence. This decoupling makes the system flexible and easy to scale.
7. Support for Offline Messaging:
   MQTT supports retained messages and persistent sessions, ensuring that messages are delivered even if a client goes offline temporarily.

Comparison with HTTP, CoAP, and AMQP

HTTP is designed for the web and follows a request-response model, which is inefficient for IoT scenarios where devices need to send data frequently. MQTT’s publish-subscribe model and low overhead make it far more efficient for real-time, bi-directional communication in constrained environments.

CoAP is also designed for constrained devices and uses UDP for faster communication. However, MQTT’s TCP-based protocol and built-in reliability mechanisms (QoS levels) make it more suitable for scenarios where message delivery is critical. CoAP is better suited for small, frequent messages in low-power networks.

AMQP is a powerful protocol designed for enterprise messaging systems, offering features like message queuing, routing, and transactions. However, its complexity and higher overhead make it less suitable for IoT devices with limited resources. MQTT’s simplicity and efficiency make it a better fit for IoT applications.

MQTT Architecture Overview

MQTT follows a publish-subscribe (pub/sub) architecture, which is fundamentally different from traditional client-server models like HTTP. In the pub/sub model, communication is decoupled, meaning that the sender (publisher) and receiver (subscriber) don’t need to know each other. Instead, they interact through a central intermediary called the Broker.

This architecture is highly scalable and flexible, making it ideal for IoT applications where thousands or even millions of devices need to communicate seamlessly. Let’s break down the four key components of MQTT architecture:

Broker

What is a Broker?

The Broker is the central hub of the MQTT architecture. It receives messages from publishers and routes them to the appropriate subscribers based on their subscribed topics. Think of the broker as a post office that ensures messages are delivered to the right recipients.

Responsibilities of the Broker:

• Message Routing: The broker matches incoming messages from publishers with the topics subscribed to by subscribers.
• Session Management: The broker manages client connections, subscriptions, and persistent sessions.
• Quality of Service (QoS): The broker ensures messages are delivered according to the specified QoS level (0, 1, or 2). We will learn more about this later.
• Retained Messages: The broker can store the last message on a topic and deliver it to new subscribers.
• Security: The broker handles authentication and authorization of clients.

Popular MQTT Brokers:

• Mosquitto: Lightweight and open-source.
• EMQX: High-performance and scalable.
• HiveMQ: Enterprise-grade with advanced features.

Topics

Topics are hierarchical strings that define the routing structure for messages in the MQTT network. They act as a “label” or “address” for the messages, guiding the broker in determining which subscribers should receive them.

How Topics Work

When a publisher sends a message, it assigns a topic to that message. The broker then uses this topic to route the message to all the subscribers who are interested in that topic.

For example:

• A publisher sends a temperature reading with the topic sensors/temperature/room1.
• A subscriber who is interested in temperature data from room1 will subscribe to the topic sensors/temperature/room1 to receive updates.

Topic Hierarchy

Topics in MQTT are structured hierarchically, similar to file paths. Each level of the hierarchy is separated by a slash (/). For instance, the topic sensors/temperature/room1 can be broken down into:

• sensors: The general category of the topic.
• temperature: A sub-category, indicating the type of data being sent.
• room1: The specific location from where the data is coming.

Topic Wildcards

MQTT supports wildcards in topics, which provide a flexible way for subscribers to receive messages from multiple topics that match a pattern. There are two types of wildcards:

• Single-level wildcard (+): This matches exactly one level in the topic hierarchy. For example, sensors/+/room1 will match both sensors/temperature/room1 and sensors/humidity/room1, but not sensors/temperature/room2.
• Multi-level wildcard (#): This matches any number of levels in the topic hierarchy. For example, sensors/# will match all topics starting with sensors/, such as sensors/temperature/room1, sensors/humidity/room2, etc.

Publisher

What is a Publisher?

A Publisher is an MQTT client that sends (publishes) messages to a specific topic. Publishers are typically sensors, devices, or applications that generate data. For example:

• A temperature sensor publishing temperature readings to the topic sensors/temperature/room1.
• A smart home app publishing a command to the topic home/living-room/light/switch.

Publishers don’t need to know who the subscribers are. They simply send messages to the broker, which handles the rest. A single publisher can send messages to multiple subscribers indirectly through the broker. Messages are published to specific topics, which are hierarchical strings (e.g., sensors/temperature/room1).

Subscriber

What is a Subscriber?

A Subscriber is an MQTT client that receives (subscribes to) messages from specific topics. Subscribers are typically applications, dashboards, or devices that need to process or display data. For example:

• A mobile app subscribing to sensors/temperature/room1 to display real-time temperature data.
• A cloud service subscribing to home/living-room/light/switch to log light switch events.

Subscribers specify the topics they are interested in (e.g., sensors/temperature/room1). Subscribers also don’t need to know who the publishers are. They simply receive messages from the broker. A single subscriber can receive messages from multiple publishers indirectly through the broker.

MQTT Protocol Versions (3.1, 3.1.1, 5.0)

Since 1999, MQTT has three major versions released:

1. MQTT 3.1 (2010): The first standardized version of MQTT.
2. MQTT 3.1.1 (2014): A minor update with clarifications and improvements.
3. MQTT 5.0 (2019): A major update with new features and enhancements.

You can find a detailed comparison of all the three versions here. We will be using MQTT 5 for all our example, since it is the newer one.

Hands-On-001:

Setting Up an MQTT Broker and Publishing/Subscribing:

Now, it’s time to get hands-on! In this hands-on, we’ll walk through:

1. Setting Up an MQTT Broker (EMQX).
2. Publishing and Subscribing using MQTT clients like MQTT Explorer or Mosquitto CLI or Postman.

By the end of this hands-on, you’ll have a fully functional MQTT environment where you can publish and subscribe to topics. Let’s get started!

Setting Up an MQTT Broker

The first step is setting up an MQTT broker is the central server that handles communication between publishers and subscribers.

Here, we’ll use EMQX. You can also check out Mosquitto and HiveMQ if interested.

EMQX is a high-performance, scalable MQTT broker designed for large-scale IoT deployments.

Step 1: Install EMQX

• On Linux:

wget https://www.emqx.io/downloads/broker/v5.0.0/emqx-ubuntu20.04-v5.0.0-amd64.deb
sudo dpkg -i emqx-ubuntu20.04-v5.0.0-amd64.deb

• On macOS:

brew install emqx

• On Windows:
  Download the installer from the EMQX website. Download the open source version.

You can also use docker or docker-compose or kubernetes to setup EMQX. Just make sure to open ports 1883 and 18083.

Step 2: Start EMQX

• Start the broker:

emqx start

Access the EMQX dashboard at http://localhost:18083 (default credentials: admin/public).

Testing the Broker

Use an MQTT client (e.g., Mosquitto CLI or MQTT Explorer or the good old Postman client) to publish and subscribe to topics.

Simply, open Postman and click new request and select ‘MQTT’ as the type. Postman has a fantastic blog on getting started with MQTT. Follow these steps and you should be able to see your connections in the dashboard.

Postman MQTT Blog

Use localhost:1883 as your host URL in postman