Protocols 20 min read • Updated: March 2025

MQTT Protocol: Complete Guide for IoT

Master the MQTT protocol for IoT. Learn about topics, QoS levels, retained messages, wildcards, and best practices with practical examples.

1. Introduction to MQTT
2. MQTT Architecture
3. MQTT Topics Explained
4. Quality of Service (QoS) Levels
5. Retained Messages
6. Last Will and Testament
7. Clean Session & Persistent Sessions
8. MQTT Security
9. Practical Examples
10. Best Practices
11. Troubleshooting

1. Introduction to MQTT

MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe messaging protocol designed for constrained devices and low-bandwidth, high-latency, or unreliable networks. It's the de facto standard for IoT communication.

📚 History:

MQTT was invented in 1999 by Dr. Andy Stanford-Clark of IBM and Arlen Nipper of Eurotech for monitoring oil pipelines via satellite. It became an OASIS standard in 2013 and ISO standard in 2016.

Why MQTT for IoT?

Lightweight: Minimal code footprint (as small as 10KB)
Low Power: Designed for battery-powered devices
Low Bandwidth: Efficient for cellular/satellite connections
Unreliable Networks: Handles disconnections gracefully
Many-to-Many: One-to-many message distribution
Decoupled: Publishers and subscribers don't know each other

MQTT vs HTTP for IoT

Feature	MQTT	HTTP
Protocol Type	Publish-Subscribe	Request-Response
Header Size	2 bytes minimum	~800 bytes
Power Consumption	Low	High
Network Usage	Low	High
Message Push	Yes (server to client)	No (client polling required)
Connection	Persistent	Per request

2. MQTT Architecture

MQTT uses a client-server architecture with three key components:

┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Publisher │ │ Broker │ │ Subscriber │ │ (Device) │────▶ │ (Server) │────▶ │ (Client) │ │ │ │ │ │ │ │ - Sensors │ │ - Routes │ │ - Dashboard │ │ - ESP32 │ │ Messages │ │ - Mobile │ │ - Arduino │ │ - Filters │ │ - Backend │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ └───────────────────────┼───────────────────────┘ │ ┌──────────┴──────────┐ │ Multiple Clients │ │ Can Connect │ └─────────────────────┘

Components Explained

Publisher

Any device that sends data to the MQTT broker. Examples:

Temperature sensors
ESP32/Arduino boards
Smart home devices
Industrial machines

Broker

The central server that receives and routes messages. Popular brokers:

Mosquitto: Open-source, lightweight (recommended for beginners)
HiveMQ: Enterprise-grade, scalable
EMQX: High-performance, open-source
AWS IoT Core: Managed cloud service
Azure IoT Hub: Microsoft's cloud IoT platform

Subscriber

Clients that receive messages from the broker. Examples:

Mobile applications
Web dashboards
Backend servers
Other IoT devices

3. MQTT Topics Explained

Topics are UTF-8 strings that the broker uses to filter messages for each connected client. They represent the "channel" or "subject" of the message.

Topic Structure

home/livingroom/temperature
factory/line1/machine2/vibration
city/traffic/intersection5/cars
agricultural/field1/soil-moisture

Topic Best Practices

💡 Topic Design Tips:

Use hierarchical structure (level/level/level)
Keep topics descriptive but concise
Avoid special characters (use hyphens or underscores)
Don't start with / (creates empty level)
Use consistent naming conventions

Topic Wildcards

Subscribers can use wildcards to match multiple topics:

Single-Level Wildcard (+)

# Subscribe to all rooms' temperature
home/+/temperature

# Matches:
✅ home/livingroom/temperature
✅ home/bedroom/temperature
✅ home/kitchen/temperature

# Does NOT match:
❌ home/livingroom/humidity  (different last level)
❌ home/livingroom/sensor/temperature  (extra level)

Multi-Level Wildcard (#)

# Subscribe to everything under home
home/#

# Matches:
✅ home/livingroom/temperature
✅ home/livingroom/humidity
✅ home/livingroom/sensor/battery
✅ home/any/number/of/levels

# Must be at the end:
✅ home/#
❌ home/#/temperature  (INVALID!)

Topic Examples by Use Case

Use Case	Topic Structure	Example
Smart Home	home/{room}/{device}/{metric}	home/kitchen/light1/state
Industrial	factory/{line}/{machine}/{sensor}	factory/line3/motor5/temp
Agriculture	farm/{field}/{sensor}/{type}	farm/north/soil1/moisture
Fleet Tracking	fleet/{vehicle}/{data}	fleet/truck42/gps

4. Quality of Service (QoS) Levels

MQTT defines three QoS levels that determine the delivery guarantee of messages. Choose based on your application's needs.

QoS 0: At Most Once (Fire and Forget)

┌──────────┐         ┌──────────┐
│Publisher │         │ Subscriber│
│          │──MSG──▶ │          │
│          │         │          │
└──────────┘         └──────────┘
No acknowledgment

Delivery: Not guaranteed
Overhead: Minimal (1 packet)
Use Cases:
- High-frequency sensor data (losing one reading is OK)
- Non-critical updates
- Bandwidth-constrained environments

QoS 1: At Least Once (Acknowledged Delivery)

┌──────────┐         ┌──────────┐
│Publisher │         │ Subscriber│
│          │──MSG──▶ │          │
│          │◀─PUBACK─│          │
└──────────┘         └──────────┘
Acknowledged, may duplicate

Delivery: Guaranteed, may have duplicates
Overhead: Medium (2 packets)
Use Cases:
- Important sensor readings
- Command messages
- Most common QoS level

QoS 2: Exactly Once (Assured Delivery)

┌──────────┐         ┌──────────┐
│Publisher │         │ Subscriber│
│          │──PUBLISH─▶│         │
│          │◀─PUBREC──│          │
│          │──PUBREL─▶│          │
│          │◀─PUBCOMP─│          │
└──────────┘         └──────────┘
Guaranteed, no duplicates

Delivery: Guaranteed, no duplicates
Overhead: Highest (4 packets)
Use Cases:
- Critical commands (door unlock, alarm disable)
- Billing/payment messages
- State synchronization

⚠️ QoS Considerations:

Higher QoS = more bandwidth and battery usage
Both publisher and subscriber QoS matter
Effective QoS is the minimum of both
QoS 2 is rarely needed in IoT

5. Retained Messages

Retained messages are stored by the broker and sent to new subscribers immediately upon subscription. Perfect for "last known good" values.

How Retained Messages Work

┌─────────────┐      ┌──────────┐      ┌─────────────┐
│ Publisher   │      │  Broker  │      │ Subscriber  │
│             │      │          │      │             │
│ Publish     │      │          │      │             │
│ retain=true │─────▶│ Store    │      │             │
│             │      │          │      │             │
│             │      │          │      │ Subscribe   │
│             │      │          │◀─────│             │
│             │      │          │      │             │
│             │      │ Send     │─────▶│ Receive     │
│             │      │ retained │      │ immediately │
└─────────────┘      └──────────┘      └─────────────┘

Use Cases

Device Status: Last known online/offline state
Sensor Values: Latest temperature reading
Configuration: Current device settings
Discovery: Device availability information

Example: Device Status

// Publish device online status with retain
client.publish("home/sensor1/status", "online", { retain: true });

// When device goes offline
client.publish("home/sensor1/status", "offline", { retain: true });

// New subscriber immediately knows device status
subscriber.subscribe("home/sensor1/status");
// Receives: "offline" (instantly, without waiting for next publish)

Clearing Retained Messages

To remove a retained message, publish an empty payload with retain=true:

client.publish("home/sensor1/status", "", { retain: true });

6. Last Will and Testament

The Last Will and Testament (LWT) is a message that the broker publishes on behalf of a client if the client disconnects unexpectedly.

How LWT Works

┌──────────┐      ┌──────────┐      ┌──────────┐
│  Client  │      │  Broker  │      │Subscriber│
│          │      │          │      │          │
│ Connect  │      │          │      │          │
│ + LWT    │─────▶│ Store    │      │          │
│ config   │      │ LWT      │      │          │
│          │      │          │      │          │
│          │      │          │      │ Subscribe│
│          │      │          │◀─────│ to LWT   │
│          │      │          │      │ topic    │
│ 💥 Crash │      │          │      │          │
│          │      │ Publish  │      │          │
│          │      │ LWT      │─────▶│ Receive  │
│          │      │ message  │      │ alert    │
└──────────┘      └──────────┘      └──────────┘

LWT Configuration Example

// ESP32/Arduino with PubSubClient
const char* willTopic = "home/sensor1/status";
const char* willMessage = "offline";

// Connect with LWT
if (client.connect("sensor1", willTopic, 1, true, willMessage)) {
  // Publish online status
  client.publish("home/sensor1/status", "online", true);
}

// If client crashes or loses power:
// Broker automatically publishes "offline" to home/sensor1/status

Use Cases

Device Monitoring: Alert when devices go offline
Safety Systems: Trigger alarms on communication loss
Auto-recovery: Initiate backup systems

7. Clean Session & Persistent Sessions

Clean Session (MQTT 3.1)

Clean Session = true: Broker doesn't store subscriptions or missed messages
Clean Session = false: Broker maintains session state

Session Expiry (MQTT 5.0)

MQTT 5.0 introduces session expiry for better control:

// MQTT 5.0 example
connectOptions.setSessionExpiryInterval(3600); // 1 hour

When to Use Each

Scenario	Recommended Setting
Mobile app (intermittent)	Clean Session = true
Always-on sensor	Clean Session = false
Critical monitoring	Clean Session = false + QoS 1
Temporary diagnostics	Clean Session = true

8. MQTT Security

Authentication Methods

Username/Password

// Basic authentication
const char* username = "myUser";
const char* password = "mySecurePassword123";

client.connect("clientID", username, password);

Client Certificates (TLS/SSL)

// Secure connection with certificates
WiFiClientSecure net;
net.setCACert(caCert);
net.setCertificate(clientCert);
net.setPrivateKey(privateKey);

PubSubClient client(net);
client.setServer("mqtt.example.com", 8883);

Encryption (TLS/SSL)

Port 1883: Unencrypted MQTT
Port 8883: MQTT over TLS (recommended)
Port 80/443: MQTT over WebSocket

🔒 Security Best Practices:

Always use TLS in production
Never hardcode credentials in firmware
Use unique credentials per device
Implement access control lists (ACL)
Regularly rotate passwords/certificates
Monitor for unusual activity

9. Practical Examples

Example 1: ESP32 Temperature Sensor

#include <WiFi.h>
#include <PubSubClient.h>
#include <DHT.h>

#define DHTPIN 4
#define DHTTYPE DHT22

const char* ssid = "YOUR_WIFI";
const char* password = "YOUR_PASSWORD";
const char* mqtt_server = "broker.hivemq.com";

DHT dht(DHTPIN, DHTTYPE);
WiFiClient espClient;
PubSubClient client(espClient);

void reconnect() {
  while (!client.connected()) {
    String clientId = "ESP32Client-" + String(random(0xffff), HEX);
    if (client.connect(clientId.c_str())) {
      client.publish("home/livingroom/status", "online", true);
    } else {
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(115200);
  dht.begin();
  WiFi.begin(ssid, password);
  client.setServer(mqtt_server, 1883);
}

void loop() {
  if (!client.connected()) reconnect();
  client.loop();

  float temp = dht.readTemperature();
  float hum = dht.readHumidity();
  
  char tempStr[8];
  char humStr[8];
  dtostrf(temp, 1, 2, tempStr);
  dtostrf(hum, 1, 2, humStr);
  
  client.publish("home/livingroom/temperature", tempStr);
  client.publish("home/livingroom/humidity", humStr);
  
  delay(10000);
}

Example 2: Node-RED Subscriber

// Node-RED flow (JSON)
[
  {
    "id": "mqtt-in",
    "type": "mqtt in",
    "name": "Temperature",
    "topic": "home/+/temperature",
    "qos": "1",
    "broker": "mqtt-broker"
  },
  {
    "id": "debug",
    "type": "debug",
    "name": "Log Temperature"
  }
]

Example 3: Python Backend Subscriber

import paho.mqtt.client as mqtt

def on_connect(client, userdata, flags, rc):
    print(f"Connected with result code {rc}")
    client.subscribe("home/#")

def on_message(client, userdata, msg):
    print(f"{msg.topic}: {msg.payload.decode()}")

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect("broker.hivemq.com", 1883, 60)
client.loop_forever()

10. Best Practices

Topic Design

✅ Use consistent naming: home/livingroom/temp
✅ Keep it hierarchical for easy filtering
❌ Avoid: home/temp1, home/temp2 (not scalable)

Payload Format

✅ Use JSON for complex data: {"temp": 23.5, "hum": 60}
✅ Use plain text/numbers for simple values: 23.5
✅ Consider Protocol Buffers for efficiency

Connection Management

✅ Implement automatic reconnection
✅ Use LWT for offline detection
✅ Set appropriate keepalive (60-120 seconds)
✅ Handle connection errors gracefully

Resource Optimization

✅ Use QoS 0 for high-frequency, non-critical data
✅ Batch sensor readings when possible
✅ Use retained messages for state
✅ Implement sleep modes for battery devices

11. Troubleshooting

Problem: Client Won't Connect

Check broker address and port (1883 or 8883)
Verify network connectivity (ping broker)
Check firewall rules
Validate credentials (username/password)
Review broker logs for error codes

Problem: Messages Not Received

Verify topic spelling (case-sensitive!)
Check QoS levels match
Ensure subscription is active
Test with MQTT client (MQTTX, MQTT Explorer)
Check broker logs

Problem: High Battery Drain

Reduce publish frequency
Lower QoS level (use QoS 0 when possible)
Increase keepalive interval
Implement deep sleep between publishes
Use WiFi power saving mode

Problem: Connection Drops

Increase keepalive interval
Check network stability
Implement exponential backoff for reconnection
Use persistent sessions (clean session = false)

Debug Tools

MQTTX: Cross-platform MQTT client
MQTT Explorer: Visual MQTT browser
mosquitto_sub: Command-line subscriber
Broker logs: Check server-side errors

Next Steps

Now that you understand MQTT, try these projects:

Build a temperature monitoring system with ESP32
Set up your own Mosquitto broker
Create a Node-RED dashboard
Integrate with Home Assistant
Explore MQTT 5.0 features

Useful Tools:
MQTT Message Generator | MQTT Topic Tester

Table of Contents

1. Introduction to MQTT

Why MQTT for IoT?

MQTT vs HTTP for IoT

2. MQTT Architecture

Components Explained

Publisher

Broker

Subscriber

3. MQTT Topics Explained

Topic Structure

Topic Best Practices

Topic Wildcards

Single-Level Wildcard (+)

Multi-Level Wildcard (#)

Topic Examples by Use Case

4. Quality of Service (QoS) Levels

QoS 0: At Most Once (Fire and Forget)

QoS 1: At Least Once (Acknowledged Delivery)

QoS 2: Exactly Once (Assured Delivery)

5. Retained Messages

How Retained Messages Work

Use Cases

Example: Device Status

Clearing Retained Messages

6. Last Will and Testament

How LWT Works

LWT Configuration Example

Use Cases

7. Clean Session & Persistent Sessions

Clean Session (MQTT 3.1)

Session Expiry (MQTT 5.0)

When to Use Each

8. MQTT Security

Authentication Methods

Username/Password

Client Certificates (TLS/SSL)

Encryption (TLS/SSL)

9. Practical Examples

Example 1: ESP32 Temperature Sensor

Example 2: Node-RED Subscriber

Example 3: Python Backend Subscriber

10. Best Practices

Topic Design

Payload Format

Connection Management

Resource Optimization

11. Troubleshooting

Problem: Client Won't Connect

Problem: Messages Not Received

Problem: High Battery Drain

Problem: Connection Drops

Debug Tools

Next Steps