freshidea - Fotolia

Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Non-real-time data interchange IoT protocols: Consider your options

Near-real-time data protocols are often better suited for IoT environments, but choosing one without careful consideration can bring unwanted long-term consequences.

As most Internet of Things data does not need to be sent in real time, a decoupled mechanism is often used. Sure, a jet engine can stream IoT data continuously to Apache Storm, but an air compressor or gas pipeline valve might only burst a few kilobytes -- and do that only once per hour.

For this reason, most IoT data interchanges use non-real-time protocols. But with all the options, which should be leveraged?

In order to discuss IoT protocols, we first need a solid definition. By protocol, we mean the application layer of the OSI Networking Model, which is a way to classify network traffic and pieces of that, for example, the software (e.g., TCP), the arrangement (e.g., packets) and the medium (e.g., Ethernet).

IoT protocols describe a process for transmitting data -- usually divided into packets -- on an IP network. Contrast that with the layer below the protocol -- the transmission layer -- which could be Z-Wave, ZigBee, NFC, Bluetooth 4 (aka BLE), BACnet, Wi-Fi or cellular. Each fits different needs such as transmitting data from a few millimeters, a few meters or across the Internet, or operating using a low-cost chip without requiring a more expensive modem and cell phone number or network interface.

HTTP is not what we usually use

HTTP is a network application layer protocol that can be used for IoT. You can try that yourself; go buy a $25 BeagleBone computing card, plug in a $5 temperature and humidity sensor, and then use its onboard Web server running on Linux to check the temperature in your house over the Internet while you are at the office.

However, HTTP is in many instances not ideal for IoT because it doesn't offer predictable latency and because it tends to rely on polling to detect state changes. Most of the IoT protocols in wide use leverage a publish-and-subscribe, message-queuing mechanism to send data to a messaging server. That can allow offline, decoupled operation.

Those familiar with IBM's MQ Server and its many open source alternatives, such as OpenMQ, are likely already familiar with message queues. MQTT, one of the most important IoT protocols, is, in fact, based on the IBM MQ model, which is suggested by its name.

What IoT protocols are not

Here we are not talking about IoT protocol programming frameworks or IoT clouds, which are different entities. A protocol framework is programming code to make working with the protocol easier. It relieves the programmer from having to understand low-level details.

IoT cloud frameworks, in contrast, are designed to work with a specific IoT vendor's clouds. For example, one of its APIs would be used to register a device on the IoT cloud and perhaps provision it with a cellular carrier.

Working in confined spaces

What makes IoT protocols different than say FTP, MQ, JMS or SFTP network protocols is that they are designed to work in a confined or constrained environment. While the IoT message is usually expanded into a wordy XML or JSON representation -- and encryption can be added, which is even more wordy -- the protocol itself is designed to be small so it works over slow network connections and simple, low-cost computers chips such as those fastened to a simple IoT device, like a solar-powered road sign.

To understand why one might need to work in a confined space, remember that an IoT device is typically nothing more than an Intel, Raspberry Pi or BeagleBone $25 card, or an even smaller chip with $1 to $25 sensors and a Bluetooth, Wi-Fi, cellular or Ethernet connection. It would not be cost-efficient to put a cellular modem on each of those or to provide a router close to each. Nor is the goal always to phone up the wide area network; some IoT protocols are designed to talk to other IoT devices or reach out only a few meters and then let the messaging server ferry the data to the IoT cloud.

The last IoT step is the collection of all the data packets that get assembled at the receiving end to do something useful. Most commonly, this is to use REST to call a Web server over the HTTP layer. The IoT device is programmed to do that, but the programmer does not need to know what protocol is used. The Web service then uploads the data it received from the Web service, such as temperature, vibration or motion. Depending on the application, this transmission might work in the other directions too, as the IoT application can send instructions to the IoT device telling it to take some action.

Let's briefly take a look at the dominant IoT protocols: MQTT, CoAP, DSS and XMPP.


MQTT is a graduation of the MQ message queue to the IoT space. So that they are not accused of having invented anything new, MQTT developers make clear that they have moved far away from the MQ series model. Message queues are designed to work when the recipient is offline, which is why it's called a queue.

MQTT is based on the publisher-subscriber model that includes the notion of topics. For example, a network-connected device could publish on the topic of temperature, such as the temperature of a particular machine or the ambient temperature. Whatever application is interested in receiving messages about the temperature topic would subscribe to it. MQTT devices can send these topics to each other or to a messaging server for delivery to the IoT cloud.

Each IoT protocol has its own terminology. For MQTT these are quite clever and even poetic; for example, news that a device has gone offline sent by another device broadcasts a somber "Last Will and Testament." With MQTT, quality of service does not mean attenuation or the percent of dropped packets; it instead indicates how much effort is put forth to guarantee the delivery of a message, be it "at most once, at least once, exactly once." Shakespeare said "brevity is the soul of wit" -- that is brief and absolutely clear.

This small code snippet offers an idea of how all of this fits together. It uses the MQTT.js framework to connect to a public MQTT server so that you prototype your ideas. This simple JavaScript code publishes and a retrieves a temperature reading and prints it to stdout (the console). To run it, you need to install Node.js, which is a JavaScript server that does not need a browser.

var mqtt = require('mqtt');
var client = mqtt.connect('mqtt://');

client.publish('temperature', '120C');

client.on('message', function (topic, message) {



In order to cram as much data into the smallest space possible, the CoAP IoT protocol has binary fields in the payload and a 4 byte header. CoAP runs over UDP and can work on SMS.

CoAP is designed to work on low-cost, battery-powered, low-network-speed devices. It would work, for example, for a smart light bulb that the manufacturer does not want to cost too much due to all of the IoT gadgets attached to it. CoAP can work with as little as 10 KB of RAM and 100 KB of memory into which to load the code.

CoAP is not an offline message queue protocol -- it uses GET, PUT, POST and DELETE to send and receive messages, like REST does -- so it requires a persistent connection. If the connection goes offline, the data is lost. But that often does not matter as sometimes it is better to send no data at all than send stale data.

To see it in action, order a Particle IoT prototyping computer card for $19, plug in a USB cable and download its Android or IoT app, and create an account on its cloud. The LED on the card lights up based on data sent from the Particle IoT app, thus showing a working IoT app. While not too thrilling, it demonstrates that it works. You could then add a humidity and temperature sensor, look at its code examples, and build an app using its CoAP framework and IoT cloud.


The Data Distribution Service standard is mainly used for industrial applications requiring device-to-device communications. Fostered by the Object Management Group, it's a machine-to-machine middleware standard that, like MQTT, uses a publish-and-subscribe model. But while it runs at a much higher speed than MQTT, it loses data when the target device goes down. It is often referred to as a bus, suggesting something fast like a hardware interface.

DDS, like CoAP, does automatic discovery, meaning it can find other devices without requiring someone to key them into some kind of topology.


XMPP is better known as Jabber, a once-popular instant messenger. XMPP was initially designed for real-time communications but has morphed into a publish/subscribe model.

XMPP is based on XML and does not use a central messaging server like CoAP and MQTT. Because it uses the email address format, it is easy to route XMPP to its destination.

Next Steps

The next big IoT challenge is getting the protocols right

Do Google's practices hinder IoT development?

Ten legacy giants making their mark on IoT

Dig Deeper on Internet of Things (IoT) Standards and Certifications