A sensor transmits its measured value to a broker via MQTT every second. Under normal operating conditions, this produces an apparently complete measurement series. After a brief network interruption, however, individual values may be missing – or the same data record may suddenly appear twice in the database. Neither necessarily indicates a fault in the sensor or MQTT broker. One of the decisive factors is which Quality of Service level is used for transmission and how the downstream data processing handles lost or retransmitted messages.
MQTT distinguishes between QoS 0 “At most once”, QoS 1 “At least once” and QoS 2 “Exactly once”. A higher QoS level provides greater delivery assurance but requires additional communication steps. For industrial measurement data, QoS 2 is therefore not automatically the best choice. A missing intermediate value from a temperature measurement updated once per second may be insignificant, for example, whereas losing a quality data record generated only once may leave the subsequent documentation incomplete.
In addition, the QoS level does not solve every problem associated with industrial data transmission. During longer offline periods, it must be clarified whether measured values are buffered locally. For data transmitted retrospectively, the original measurement time is important. With QoS 1, an application must be able to identify possible duplicate messages. And in large IIoT installations, transmission intervals, the number of sensors, broker load and data aggregation are just as important as the actual QoS setting.
The correct MQTT QoS level should therefore not be selected according to the principle “the higher, the better”. What matters is the significance of each individual data record, how much information loss can be tolerated and how the complete data chain from the sensor to the database is designed.
What does MQTT Quality of Service mean?
MQTT Quality of Service describes the delivery logic of a message. It has nothing to do with the measurement accuracy of the sensor. A temperature value of 82.4 °C does not become more accurate with QoS 2 than with QoS 0. The QoS level merely determines how the message is transmitted and acknowledged between the MQTT components involved.
The significance of the individual message is therefore particularly important when selecting the appropriate level. If a process value is updated every second, the next measured value may already be available shortly after one message is lost. A data record generated only once, for example for a completed production cycle, may not have a second opportunity for transmission. This difference in the significance of the data must be evaluated before selecting the QoS level.
MQTT QoS 0: At most once
QoS 0 operates according to the “At most once” principle. The publisher sends the MQTT message without waiting for confirmed delivery at this QoS level. If the message is not successfully transmitted during a communication failure, QoS 0 does not trigger a retransmission. Individual values may therefore be lost.
For continuously updated measurement values, this does not necessarily have to be a problem. If, for example, a pressure value is transmitted every second for a live dashboard and one data point is lost during a brief interruption, another current value is already available one second later. If a complete historical record of every individual intermediate value is not required, QoS 0 can therefore be a technically appropriate solution. Its advantage is the comparatively low communication and processing overhead.
MQTT QoS 1: At least once
QoS 1 operates according to the “At least once” principle. Receipt of a message is acknowledged. If the expected acknowledgement is not received, the sender can transmit the message again. Compared with QoS 0, this significantly reduces the risk of an unnoticed message loss.
However, this method has an important consequence: the same data record can arrive at the receiver more than once. This can happen, for example, when the original message arrives successfully but the acknowledgement is lost on the return path. The sender can then no longer determine with certainty whether the first transmission was successful and sends the message again. For measurement data applications, this means that QoS 1 can be particularly suitable when losing a data record would be more critical than receiving a duplicate – provided that the downstream software can identify repeatedly received data records.
MQTT QoS 2: Exactly once
QoS 2 uses a more extensive message exchange to ensure that a message is delivered exactly once within the MQTT communication. This provides stronger protection against both message loss and duplicate MQTT delivery. However, this additional protection requires more protocol steps and therefore increases communication and processing overhead.
With only a few devices and low data rates, this additional overhead is often insignificant. In installations with hundreds or thousands of data sources and short transmission intervals, however, it can become relevant. In addition, “Exactly once” must not be interpreted as a guarantee that a data record will automatically be processed exactly once throughout the entire IT system. After MQTT delivery, further layers such as applications, databases or analytics processes may follow and require their own mechanisms for reliable processing.
QoS 0, 1 and 2 compared
| Criterion | QoS 0 | QoS 1 | QoS 2 |
|---|---|---|---|
| Principle | At most once | At least once | Exactly once |
| Confirmed delivery | No | Yes | Extended message exchange |
| Loss of individual messages | Possible | More strongly protected | More strongly protected |
| Duplicate MQTT delivery | Not caused by QoS retransmission | Possible | Prevented by the QoS 2 procedure |
| Communication overhead | Low | Medium | High |
| Typical use | Rapidly updated, loss-tolerant data | Measurement data with high requirements for completeness | Messages with high requirements for single MQTT delivery |
The table shows why a blanket recommendation such as “always use QoS 2 for industrial applications” is not appropriate. A continuously updated measured value has different requirements from a one-time test data record. The selection should therefore be based on the information value of the message rather than solely on the maximum available delivery assurance.
Which QoS level is suitable for measurement data?
For measurement data, the first question should be what happens if exactly one data record is missing. With a live value, the loss of information may be very small because the next current value is transmitted shortly afterwards. In a historical process record, however, a gap can make subsequent analysis more difficult. The loss of a one-time test or quality data record can be even more critical.
| Data type | Typical requirement | Possible QoS tendency |
|---|---|---|
| Live display of a process value | Current value is more important than every intermediate value | QoS 0 may be sufficient |
| Historical process recording | As few data gaps as possible | QoS 1 may be appropriate |
| One-time test or quality data record | Data record should not be lost | Consider QoS 1 or QoS 2 |
| Event or alarm message | Individual message may be highly important | Higher delivery assurance may be appropriate |
| Many rapidly transmitting sensors | Limit broker and network load | Balance QoS against data rate |
This means that a combination of different QoS levels may be appropriate even within the same machine. A continuously transmitted pressure value could, for example, be sent with QoS 0, while a one-time fault message or quality data record is transmitted with QoS 1. The QoS level is therefore often more a characteristic of the data type than of the device as a whole.
Why can duplicate messages occur with QoS 1?
A duplicate message with QoS 1 does not necessarily indicate a communication error. It can be a direct consequence of the intended delivery principle. Assume that a sensor successfully sends a measured value to the broker. The broker acknowledges receipt, but this acknowledgement is lost during a brief network interruption. The sender can no longer determine with certainty whether the original message arrived and may therefore retransmit it.
If both messages are subsequently stored in a database without any further checks, two apparently separate measurements are created. For measurement data, the time of receipt alone should therefore ideally not be used to identify a data record. An original measurement timestamp, a sequence number or a unique data record identifier can allow the downstream application to identify repetitions. A data record could, for example, logically consist of Sensor ID + timestamp + sequence number + measured value. If exactly the same combination is received again, the application can treat it as a retransmission.
QoS 1 works particularly well when the application is designed according to the principle: a duplicate can be handled, but a lost relevant data record cannot.
Understanding the difference between QoS and offline periods
A higher QoS level does not replace complete offline data storage. If, for example, the network connection of an IIoT gateway fails for 30 minutes, the first question is where the measured values recorded during this period are stored. QoS describes the delivery of MQTT messages, but it does not automatically provide storage for an unlimited number of measured values that cannot yet be transmitted.
| Situation | Role of QoS | Additional requirement |
|---|---|---|
| Brief interruption during transmission | QoS influences acknowledgement and retransmission | Select the appropriate QoS level |
| Gateway without a connection for an extended period | QoS alone is not sufficient | Consider local buffering |
| Broker unavailable for an extended period | QoS is only part of the overall behavior | Define publisher, session and storage strategy |
| Historical measurement series must remain complete | Delivery reliability is important | Consider timestamps, buffering and retransmission |
If historical measurement data must remain complete even during longer communication failures, a store-and-forward strategy is therefore often required. The gateway stores the data locally and transmits it once the connection has been restored. It is important to determine how much local storage is available, how long it will last at the intended data rate and how the system behaves when a large number of buffered data records are transmitted within a short period after a longer outage.
Timestamps and the chronological assignment of measurement data
For measured values transmitted retrospectively, the original measurement time is particularly important and can be more relevant than the later reception time. A sensor may, for example, take a measurement at 10:15:03, while the message does not reach the higher-level system until 10:27:40 because of an interrupted connection. If only the reception time were stored, the measured value would appear more than twelve minutes too late in the historical time series.
The original timestamp should therefore ideally be part of the measurement data record. This is particularly important when data from several machines or gateways is correlated, when measured values are transmitted in batches after an offline period, or when events must later be reconstructed precisely in time. This in turn requires appropriate time synchronization of the components involved.
QoS, data rate and broker load
In small applications, the additional communication overhead associated with higher QoS levels is often barely noticeable. The situation changes, however, as the number of devices and the transmission frequency increase. Just 100 sensors sending one message every second already generate 8,640,000 measurement messages per day. With 1,000 sensors under the same conditions, the figure would be 86,400,000 measurement messages per day. Additional acknowledgements and processing steps can add up significantly at this scale.
For this reason, the payload size should not be the only factor considered. The number of publishers, number of subscribers, transmission interval, topic structure, QoS level and the performance of the broker, network and downstream database are also relevant. An unnecessarily high QoS level can be just as inefficient as an unnecessarily high transmission rate.
It is also important to distinguish between measurement rate and transmission rate. A sensor may internally perform 100 measurements per second, while only one measured value per second may be required for central plant monitoring. Depending on the application, averages, minimum and maximum values, change thresholds or event-based transmissions can reduce the volume of data. However, such aggregation must not remove information that may be required later – for example, short pressure peaks or fast vibration events.
Practical example: measurement data from a production machine
A production machine records pressure, temperature, flow and other process variables. The values are transmitted via an IIoT gateway to a central MQTT broker. The pressure value is mainly used for a live display of the current machine status and is transmitted every second. During a brief communication interruption, one measurement point is missing. Since the next current value is already available one second later and a complete second-by-second history is not required for this display, QoS 0 may be sufficient in this case.
After each completed production cycle, the same machine also generates a quality data record. This contains, for example, the cycle number, maximum process pressure, temperature and test result. This data record is generated only once per cycle. If the message is lost, the documentation for that specific production process is missing. QoS 1 may be appropriate here if the receiving application can identify possible duplicates using the cycle number or a unique data record identifier.
If the network connection fails completely for 15 minutes, however, even the higher QoS level alone is not sufficient. If the documentation must remain complete, the gateway must store the data records generated during this period locally and retransmit them later. The original measurement times must be retained so that the data can subsequently be placed correctly back into the process history.
This example shows that the entire machine does not necessarily require a single QoS level. Continuous process values, quality data records and event messages can have different requirements for delivery reliability, duplicate detection and offline storage.
Systematically selecting the QoS level
- First define which measured values, status information and events are to be transmitted and determine the significance of each individual data record.
- Evaluate the consequences of losing a measured value and whether a later current value effectively replaces the missing information.
- For historical or one-time data records, determine how complete the transmission must be and whether QoS 1 or QoS 2 is required.
- With QoS 1, provide a unique means of identifying measurement data records so that possible duplicates can be detected.
- Consider measurement rate and transmission rate separately and determine whether local data aggregation is appropriate.
- Include the number of publishers, subscribers, message rate and broker performance in the system design.
- For longer communication failures, determine whether local buffering and store-and-forward functionality are required.
- Retain the original measurement time in the data record if delayed transmission is possible.
- Deliberately test network interruptions, broker restarts and reconnections and then check for missing or duplicate data records.
Common mistakes
- Using QoS 2 for all data as a matter of principle: The highest level of delivery assurance is not automatically the most efficient solution. Rapidly updated, non-critical measurement values may be transmitted appropriately using QoS 0.
- Confusing QoS 1 with “exactly once”: At least once explicitly means that a message may arrive at the receiver more than once.
- Failing to consider duplicates in the data structure: Without a timestamp, sequence number or unique data record identifier, retransmitted messages are difficult to distinguish from genuinely new measurements.
- Equating QoS with offline storage: Longer network outages may additionally require local data buffering.
- Storing only the reception time: Retrospectively transmitted measured values may then appear at the wrong position in the historical time series.
- Treating measurement rate and transmission rate as identical: Not every sensor value recorded internally necessarily has to be transmitted to a central system via MQTT.
- Treating all data types identically: A live process value, an alarm message and a quality data record can have completely different delivery requirements.
- Failing to test offline scenarios: Problems with retransmission, delayed transmission or data loss often only become apparent during a real communication failure.
IIoT solutions for industrial measurement data
For reliable industrial measurement data transmission, the entire chain must be considered: sensors, data acquisition, edge or gateway components, network, MQTT broker and downstream data platform. Simply supporting the MQTT protocol does not determine whether a solution is suitable for the particular measurement task.
Depending on the application, local data preprocessing, timestamping, data aggregation, buffering during communication failures and controlled retransmission may be particularly relevant. An edge gateway can, for example, process high-speed raw data locally and transmit only the information actually required for operation, analysis or documentation to the broker. This can reduce network load, broker load and long-term storage requirements.
You can find suitable components and systems for acquiring, processing and transmitting industrial measurement and process data under IIoT solutions at ICS Schneider.
Conclusion
MQTT QoS 0, 1 and 2 address different requirements. QoS 0 provides low communication overhead but accepts the possibility of message loss. QoS 1 increases delivery reliability but can result in messages being received more than once. QoS 2 prevents duplicate MQTT delivery at the cost of additional protocol overhead.
For industrial measurement data, the significance of the individual data record should therefore be evaluated first. A live value updated once per second can be treated differently from a one-time quality data record or fault message. The way in which longer offline periods are handled is equally important. If historical data must remain complete, local buffering, original timestamps and controlled retransmission are just as much part of the overall concept as the QoS level.
The data volume should also not be underestimated. In large IIoT systems, even small differences in transmission intervals or the number of communication steps can have a significant impact on the broker, network and database. Measurement rate, transmission rate, data aggregation and delivery reliability should therefore be considered together.
For reliable MQTT measurement data transmission, the key principle is therefore: do not simply select the highest possible QoS level. Instead, define for each data type how much information loss is acceptable, how duplicates are handled and how the overall system should behave under real communication failures.
FAQ: MQTT QoS 0, 1 and 2 for measurement data
What is the difference between MQTT QoS 0, QoS 1 and QoS 2?
QoS 0 operates according to “At most once”, QoS 1 according to “At least once” and QoS 2 according to “Exactly once”. As the QoS level increases, the assurance of MQTT delivery increases, but so does the communication and processing overhead.
Is MQTT QoS 2 always better than QoS 1?
No. QoS 2 requires a more extensive message exchange. If possible duplicates can be reliably detected, QoS 1 can provide a useful compromise between delivery reliability and overhead for many measurement data applications.
Why can duplicate messages occur with MQTT QoS 1?
If the original message arrives but the associated acknowledgement is lost, the sender may retransmit the message. The receiver may therefore receive the same data record more than once.
How can duplicate MQTT measurement values be detected?
Measurement data records can, for example, contain an original timestamp, a sequence number and a unique device or data record identifier. This makes it possible to determine whether a data record is genuinely new or has merely been retransmitted.
Is QoS 0 suitable for industrial sensor data?
Yes, if the loss of an individual intermediate value can be tolerated. This can be the case, for example, with frequently updated live process values where the next current measured value is already available a short time later.
Which QoS level is suitable for historical measurement data?
If as few data records as possible should be missing, QoS 1 can be appropriate provided that possible duplicates can be detected. For particularly demanding requirements, QoS 2 can be considered. For longer offline periods, local storage and subsequent retransmission must also be taken into account.
Does QoS 2 prevent data loss during a longer network outage?
No. QoS 2 does not replace local data storage. If measured values continue to be generated during a longer offline period, it must be ensured separately that they are stored and can be retransmitted later.
Why should an MQTT measured value contain its own timestamp?
With delayed transmission, the reception time no longer corresponds to the actual measurement time. A dedicated timestamp allows the measured value to be positioned correctly within the historical measurement series.
Does MQTT QoS affect measurement accuracy?
No. The QoS level affects message transmission, not the metrological accuracy of the sensor or the transmitted measured value.
Does a higher MQTT QoS level increase communication load?
Yes. Higher QoS levels require additional communication steps and acknowledgements. With only a few devices, the difference is often insignificant, but with many sensors and short transmission intervals it can add up considerably.
Should all data from a sensor be transmitted using the same QoS level?
Not necessarily. A continuously updated process value may have different requirements from a one-time alarm or quality message. Different topics or data types can therefore use different QoS levels.
Which is more important: MQTT QoS or local data buffering?
They serve different purposes. QoS determines the delivery logic of an MQTT message. Local buffering, on the other hand, protects measurement data that cannot yet be transmitted at all during a longer communication interruption.
How do you select the correct MQTT QoS level for measurement data?
The first step is to evaluate the consequences of a lost or duplicate data record. The transmission rate, broker load, offline behavior, timestamping and the ability to detect duplicates are then considered. Only then can the appropriate QoS level for the respective data type be selected.