Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Engineering IoT data to be agile, smart and valuable

As more industries explore the boundless business opportunities inherent in IoT connectivity, new use cases are being developed every day. From smart thermostats and health-monitoring wearables to connected cars, innovative devices and systems are poised to generate revenue throughout the IoT ecosystem. But there is something even more significant that IoT will generate in abundance, and that’s data.

The cost of data systems, as a percentage of IT spend, is already growing. If you consider that more than 20 billion devices are expected to be interconnected by the year 2020 — and given the fact that all of them will collect and share data — it’s easy to imagine that we will be awash in data… nearly endless data! And although big data can be a valuable commodity for big business, we already have a great deal of data that is being wasted. In fact, Forrester Research estimates that enterprises are only making use of between 20-50% of their structured data for business intelligence.

IoT data deluge

Organizations of all sizes have an increasingly urgent need to organize disparate data that can be used to increase operational efficiency, improve customer service and glean valuable business insights. This requires a holistic view of all data, no matter the source and in whatever form it takes — whether operational data, customer-related SaaS data, high-volume social media and machine data, or unstructured data such as contacts, emails and web logs.

The rising cost and complexity of storing and maintaining data is driving demand for new, innovative technologies. But it’s more than just a problem of storage and organization. Interconnected devices, machines and things create high-transaction data transmitted to and from centralized gateways, much of which requires intelligent computing for probability and analysis. Today’s businesses require advanced data engineering capabilities to make their data smarter and more valuable.

For example, consider connected cars in IoT. Each connected vehicle on the road will be transmitting and receiving a variety of analytical data — system diagnostics, location information, traffic navigation and so forth — that could be used for everything from maintenance scheduling and roadside assistance service to autonomous driving. This scenario creates the challenge of receiving, managing and computing various types of high-transaction data.

Driving business insights

Communications solutions provider Comtech Telecommunications develops location-based service systems for connected cars, providing the in-car user interface and navigation maps, as well as managing a data aggregation platform. Analytical data sent to the aggregation gateway ranges from critical data that requires real-time action, such as location and in-route navigation, to lower priority data, such as reminders to schedule routine maintenance.

Of course, for any data to be of use it needs to be aggregated, analyzed, sorted and shared, whether that means internally within an enterprise or in partnership with third parties. And performing all these tasks with large volumes of transactional data would be nearly impossible without the help of computing resources. Data engineering facilitates and enables the capabilities needed to perform data computing functions, such as probability and analysis.

Good behavior modeling

Let’s say, for instance, that a motorist has engaged with a company like AAA or Urgent.ly for roadside assistance service. Location information being used for the car’s navigation system can also be shared with the roadside assistance company. However, let’s assume that the car breaks down and is no longer transmitting its location. Predictive algorithms can help the roadside service pinpoint the current location of the driver in distress by analyzing the last known location of the vehicle in combination with historical data related to the motorist’s daily driving behavior.

Likewise, some insurance companies are using connected car data as well, offering drivers the chance to opt-in to a program that tracks driving behavior in exchange for safe driving discounts. Some consumers may have concerns about sharing this level of personal data, so data security can become an issue — as is the case with many other types of IoT data.

Questions about data security are just one aspect of the data engineering process required to design robust data aggregation and analysis platforms for the IoT. I work with data engineers who build complex models of data integration and analysis, testing them for very high rates of data acquisition. Imagine hundreds of thousands of vehicles on the road, each aggregating data every second of every day! This vast scale requires testing programs that can simulate millions of connected devices.

Beyond data science, data engineering enables design and development of an integrated system that uses not only data analysis, but also predictive analytics and machine learning, modeling your data and augmenting it with real-time data such as traffic conditions. As a result, the agility and accuracy of your IoT data can grow over time — along with the business value. Smart enterprises that operationalize their disparate data have a lot to gain from simplifying and streamlining data systems, thereby reducing complexity and cost of ownership, while sparking innovative new business models and unlocking valuable business insights that drive competitive advantage.

All IoT Agenda network contributors are responsible for the content and accuracy of their posts. Opinions are of the writers and do not necessarily convey the thoughts of IoT Agenda.