Publish/Subscribe is a messaging pattern that decouples message producers (publishers) from consumers (subscribers). The publisher sends messages to a broker under a topic or channel, and subscribers receive messages from the broker by subscribing to specific topics.

Free Link
Part 8: Link
Part 10: Link

Unlike direct messaging (e.g., HTTP requests), Pub/Sub allows multiple subscribers to receive the same message and enables asynchronous communication.

  • Publishers send messages without knowing who will receive them.
  • Subscribers receive messages they are interested in without knowing who sent them.
  • Messages are routed based on topics or channels.
  • The broker is the intermediary that manages message distribution between publishers and subscribers. It receives messages from publishers, determines which subscribers are interested in those messages based on topic subscriptions, and forwards the messages accordingly.

This decoupling allows systems to scale efficiently and makes them more maintainable. Pub/Sub is widely used in microservices, event-driven architectures, chat applications, notifications, and more.

For example, when you have millions of IOT devices each sending a message every 10 second, Pub/Sub is a great choice compared to HTTP. The devices can just send the messages to the broker and forget about it. They don’t need to keep waiting for a response from the server. The servers can subscribe to some topic in the broker and recieve the messsage the devices send.

There are many Pub/Sub protocols out there. Some of them include Redis Pub/Sub, NATS, MQTT, GCP has its own Pub/Sub and even Kafka can be used as a Pub/Sub broker.

For this course, we’ll stick to MQTT due to its simplicity and popularity in small-scale and IoT applications.

Using MQTT in Go

We’ll use the Eclipse Paho MQTT client for Go. It provides methods to connect to a broker, subscribe to topics, and publish messages. We’ll use the EMQX public broker for demonstration, but the same code works with any MQTT broker. For production, it’s better to host your own broker or use a broker provided by the cloud services like GCP.

go get github.com/eclipse/paho.mqtt.golang

Subscriber Program

The subscriber listens for messages on a given topic (/demo in this case). It connects to the broker and waits for messages from any publisher on that topic.

package main

import (
 "fmt"
 mqtt "github.com/eclipse/paho.mqtt.golang"
)

func main() {
 opts := mqtt.NewClientOptions().
  AddBroker("tcp://broker.emqx.io:1883").
  SetClientID("go_subscriber")

 client := mqtt.NewClient(opts)
 if token := client.Connect(); token.Wait() && token.Error() != nil {
  panic(token.Error())
 }
 defer client.Disconnect(250)

 topic := "/demo"
 client.Subscribe(topic, 0, func(_ mqtt.Client, msg mqtt.Message) {
  fmt.Printf("Received message: %s\n", msg.Payload())
 })

 select {} // block forever
}

Client Options:
We specify the broker address and a unique client ID. The broker is where publishers send messages.

Connect:
client.Connect() opens a TCP connection to the broker.

Subscribe:
client.Subscribe(topic, qos, callback) registers a function to handle incoming messages on the topic. The qos (Quality of Service) 0 means “at most once” delivery.

Keep Running:
select {} is a simple way to prevent the program from exiting so it can keep listening indefinitely.

Publisher Program

The publisher sends messages to the same topic that the subscriber is listening to. Each message will be received by all subscribers of that topic.

package main

import (
 "fmt"
 "time"
 mqtt "github.com/eclipse/paho.mqtt.golang"
)

func main() {
 opts := mqtt.NewClientOptions().
  AddBroker("tcp://broker.emqx.io:1883").
  SetClientID("go_publisher")

 client := mqtt.NewClient(opts)
 if token := client.Connect(); token.Wait() && token.Error() != nil {
  panic(token.Error())
 }
 defer client.Disconnect(250)

 for i := 1; i <= 3; i++ {
  msg := fmt.Sprintf("Hello %d", i)
  client.Publish("/demo", 0, false, msg)
  fmt.Println("Published:", msg)
  time.Sleep(time.Second)
 }
}

Client Options and Connect:
Same as the subscriber, but we use a different client ID. Multiple clients can connect to the same broker simultaneously.

Publish Messages:
client.Publish(topic, qos, retained, payload) sends a message to the broker. Any subscribers to the topic will receive it.

Sleep Between Messages:
We wait 1 second between messages so the subscriber has time to process each one.

Disconnect:
Gracefully disconnects from the broker.

Now, If you run the subscriber first and then the publisher, the subscriber will print every message the publisher sends:

Received message: Hello 1
Received message: Hello 2
Received message: Hello 3
...

Extending MQTT with HTTP: Callbacks and Webhooks

In many systems we don’t just want to fire a message and forget. We also want to know whether the message was actually received and processed. Pub/Sub systems like MQTT guarantee that a message reaches the broker, because well the broker can always send a response back, but they do not guarantee that the subscriber device actually got the message and acted on it.

In fact, there may not even be a subscriber online when a message is published. In those cases, we might want to know that and retry the publishing.

Imagine a server that manages thousands of smart bulbs in people’s houses. The server might publish a message like “turn on” to a topic for a particular bulb. MQTT guarantees that the message reaches the broker, but not that the bulb itself actually received and executed the command. Maybe the bulb is offline, maybe the Wi-Fi dropped. From the server’s perspective, it has no way to know.

A common thought is to make the bulb publish back an acknowledgment to another MQTT topic. But then we face the same question again: how does the bulb know the server got that acknowledgment? If the server then sends another acknowledgment to confirm receipt, it can easily spiral into an endless loop of acknowledgments bouncing back and forth.

This is where callbacks and webhooks solve the problem:

  • A callback is a mechanism where one system tells another, “let me know when you’re done.” Instead of polling or waiting, the caller provides a return path and moves on; the callee calls back once the work is finished.
  • A webhook is a type of callback that happens over HTTP. One system exposes a URL (an HTTP endpoint), and the other system makes an HTTP POST request to that URL when the event occurs. The receiving system can then respond with 200 OK to confirm receipt.

Instead of relying only on MQTT acknowledgments, the server can embed a HTTP callback URL inside the MQTT message payload. For example, the server could publish a message like this:

{
  "command": "turn_on",
  "callback_url": "https://smarthome-server.com/callback/bulb123/operation456"
}

When the smart bulb receives the command and successfully turns itself on, it makes an HTTP POST request to the provided callback URL:

The server is running a simple HTTP endpoint that handles this callback. When it responds with a 200 OK, the bulb knows its acknowledgment has been received. If the server is down or unreachable, the bulb retries the POST call until the callback succeeds.

This approach combines the strengths of both HTTP and Pub/Sub. MQTT remains the fast, lightweight way to distribute commands at scale to thousands of devices, while callbacks provide reliable confirmation that a device actually processed the command.

It’s the same idea used in many industries when two systems need to talk asynchronously. One system performs the work, and when it’s done, it calls back the other through a Webhook.

Here’s how we can extend your publisher code into a combined MQTT publisher + HTTP callback server:

package main

import (
 "encoding/json"
 "fmt"
 "net/http"
 "time"

 mqtt "github.com/eclipse/paho.mqtt.golang"
)

type CallbackPayload struct {
 Device    string `json:"device"`
 Status    string `json:"status"`
 Timestamp string `json:"timestamp"`
}

func main() {
 go startHTTPServer()

 opts := mqtt.NewClientOptions().
  AddBroker("tcp://broker.emqx.io:1883").
  SetClientID("go_publisher")

 client := mqtt.NewClient(opts)
 if token := client.Connect(); token.Wait() && token.Error() != nil {
  panic(token.Error())
 }
 defer client.Disconnect(250)

 topic := "/demo"
 for i := 1; i <= 3; i++ {
  msg := fmt.Sprintf(`{"command":"turn_on","device":"bulb123","callback_url":"http://localhost:8080/callback/bulb123/op%d"}`, i)
  client.Publish(topic, 0, false, msg)
  fmt.Println("Published:", msg)
  time.Sleep(2 * time.Second)
 }

 select {} // keep server alive
}

func startHTTPServer() {
 http.HandleFunc("/callback/", func(w http.ResponseWriter, r *http.Request) {
  var p CallbackPayload
  if err := json.NewDecoder(r.Body).Decode(&p); err != nil {
   http.Error(w, "bad request", http.StatusBadRequest)
   return
  }
  fmt.Printf("Callback received: %+v\n", p)
  w.Write([]byte("acknowledged"))
 })

 fmt.Println("HTTP callback server running on :8080")
 http.ListenAndServe(":8080", nil)
}
  1. The server publishes MQTT messages with a callback_url pointing back to itself (http://localhost:8080/callback/...).
  2. When a device receives the command and finishes, it makes an HTTP POST to that callback URL with its status.
  3. The same Go server listens on port 8080, parses the JSON payload, prints it, and responds with 200 OK.
  4. If the device retries (say the server was down for a bit), it will succeed once the server is back and serving callbacks.

Till Next Time

Pub/Sub lets you decouple producers from consumers and move past the limitations of direct request/response messaging. In practice this means systems can stay responsive, scale cleanly, and tolerate failure without grinding to a halt. Once you start thinking in terms of events instead of calls, you unlock an entirely different way of designing distributed applications.

You see this pattern everywhere: financial systems streaming trades to multiple risk engines, ride-sharing platforms pushing driver locations to nearby passengers, multiplayer games syncing state across clients, and IoT fleets reporting telemetry to cloud services. The details vary, but the principle stays the same. Events flow through a broker, and multiple independent subscribers can react in real time without the producer even knowing they exist.

References for further reading: