In our first post in this IoT series, we discussed how to design an Internet of Things (IoT) platform and that it is valuable only if it covers the end-to-end chain, the whole being greater than the sum of the parts. In this second article, Umesh Puranik and I look at using open-source components to build an IoT platform; we describe the logical platform architecture and its components. The following diagram shows this logical architecture.
A quick description of the components in the above diagram and their significance in the overall scheme follows. Note that certain functionality could be made available either locally (components 6-9) or remotely (i.e., cloud-based: components 10-13), or potentially as a hybrid. Similarly, it is possible to integrate different logical components (e.g., 4-5 or 1-2-3-4) into a single physical component. These implementation decisions depend on the actual use case scenario.
1. Sensors and Actuators
These are the endpoints in an IoT system. Sensors initiate the chain by providing the data, while actuators complete it by executing some action. Of course both of these are not necessary in every IoT system: a weather monitoring system, for instance, may not contain any actuators. In such a case, a visualization tool or a reporting engine may serve as the endpoint of the chain.
Sensors can be of different technologies (e.g., electrical, electronic, electro-mechanical, electro-chemical). Their output signal can be analog (e.g., temperature sensors, light level sensors) or digital (e.g., limit switches), and this output is typically provided on a wire interface. An IoT system often has both analog and digital sensors. Likewise, actuators might be analog or digital.
2. Sensor/Actuator Interface
The sensor data is read by interfacing devices like microcontrollers over wire interfaces using either analog or digital I/O pins. Analog sensor output can be digitized using a built-in Analog to Digital Converter (ADC) in the microcontroller. Digital sensor output is typically read using the general purpose input/output (GPIO) pins of the microcontroller. Some sensors use various digital interface mechanisms like I2C, SPI, etc. and provide output in digitally encoded formats (e.g., LTC2990 – quad temperature, voltage and current monitor with I2C interface, LM74 – temperature sensor with SPI interface).
3. Interface Device/Controller
The Interface Device/Controller is a microcontroller or a PC that communicates with the sensor interfaces using wired (UART – RS232, RS485, RS422, GPIO,USB) or wireless (Wi-Fi (802.11), Bluetooth, ZigBee (802.15.4), X10) protocols. Wired interfaces are simple and cheap, but can be used only over a small physical distance. Wireless interfaces are more complex and expensive, but offer much larger area coverage and easier deployment. They also offer the advantage of a large fan-in, i.e., one Interface Device/Controller could communicate with many sensor interfaces. The sensor interface can be implemented as an integral block of the Interface Device/Controller.
The interface device can be leveraged for edge computing if it has sufficient processing power and resources. It can carry out some local processing of the data, draw inferences, and initiate actions locally. This is helpful in low-bandwidth, high-latency networks. It can also provide fast responses to important events or keep running critical functions even if the network connectivity is lost.
4. Gateway
The components described so far constitute a ‘sensor platform’. The gateway is the interface of the sensor platform with the rest of the IoT system. It provides various functions like abstracting and encapsulating the sensor platform, aggregating data from diverse sensor platforms, sending (routing) data to appropriate places in the IoT pipeline, etc. It can be implemented on a single hardware-software platform (microcontroller or PC) along with the Interface Device.
The gateway uses protocols like HTTP, REST, MQTT, CoAP, DDS, AMQP, XMPP, etc. It communicates with the Interface Device on one side and the rest of the IoT pipeline on the other side. For the communication between the Interface Device and the Gateway, protocols like MQTT and CoAP are popular because of their ability to work with the resource-constrained devices. For a Publish-Subscribe messaging protocol like MQTT, the gateway acts as a message broker and the Interface Device acts as a client.
For large IoT systems, the gateway can itself be a client to another messaging system like RabbitMQ or Apache Kafka that can provide scalability, buffering and queuing of data, protocol translation, etc. The gateway/router message broker, in turn, can itself be a client to another messaging mechanism that would take the data sent by it to the other components in the analytics pipeline.
6. Storage (Local/Cloud)
Data management is a challenge in an IoT system because of the three Vs of the data – Variety, Velocity, and Volume. Individual sensors and devices are generating data in different (even unrelated) formats, often at high velocity (e.g., in industrial environments, traffic management, etc.). The overall volume of the data generated is huge because of the large number of sensors and devices in the IoT system. (As an aside, the fourth V of the data – Veracity – needs to be handled at the data capture point such as a sensor interface.)
To be effective, this data needs to be processed at (near) real-time speeds. In addition to this, it may also need to be stored at different stages of processing, like raw data that comes from the sensors for historic reasons, preprocessed data, results of analytics, etc.
All these requirements need to be considered while selecting a database for an IoT system. Often, a combination of different databases is used in order to get the required functionality with satisfactory performance, e.g., NoSQL databases for fast data ingestion (to handle the velocity) and HDFS/Hadoop for handling the large volume processing in an offline manner. Such an approach increases the complexity of the system.
7. Analytics (Local/Cloud)
A key component in the pipeline is the analytics engine, which, like the databases, needs to be able to handle the three Vs of data, in real-time or near-real-time. It should have the ability to process continuous streams of data and calculate rolling metrics, aggregate statistics, event correlation, trend analysis, predictive analysis, etc. There are various open-source alternatives to choose from (e.g., Apache Storm, Apache Spark).
8. User Interface (UI) (Local/Cloud)
All this data won’t do any good if people can’t easily visualize and then act on it. So the user interface is a must. It provides the ability to peek into the happenings, set rules for actions, etc.
The user interface can be implemented either as web applications or as apps for mobile or desktop platforms (Android, iOS, Windows).
9. Applications (Local/Cloud)
Applications use the data and insights extracted by the analytics engine to realize the true value of the end-to-end IoT pipeline. The significance lies in the actions executed by the application. For example, an application for industrial IoT can help improve operational efficiency, or avoid downtime by predicting failures and suggesting preventive maintenance in advance. An application for smart city traffic management could alert authorities about congestions, adjust traffic signal timings dynamically to smooth out the traffic flow, and inform commuters to avoid congested roads.
Such applications might be stand-alone applications specifically designed for desktop or mobile platforms, or they could be tightly coupled with the given IoT environment for specific cases, like in industrial monitoring and control systems.
Now that we are familiar with the individual components and the end-to-end IoT system, the next article in this series will describe a messaging-based approach along with some implementation choices.
Image Credits: DAILY ILLUMINATION | Jeremy Berg
Dr. Siddhartha Chatterjee is Chief Technology Officer at Persistent Systems. Umesh Puranik is a Principal Architect at Persistent Systems.
