Big data has been such big news that one important point has been lost amid all the hype: Machine-generated and sensor-based data have been around for decades. Outside the comparatively new worlds of e-commerce and ERP lies a massive amount of machine technology. From robots and machine vision systems on the shop floor to enormous assets like gas turbines, windmills and jet engines, machines drive essential business processes and create huge amounts of complex, real-time data.
And such data sources are suddenly pushing the so-called Internet of Things -- which refers to connecting all devices to the Internet -- into the limelight. The C-level suite has long ignored the value of machine data and relegated its use to the plants and field assets such as pipelines and heavy vehicles where the data originated. But execs are finally waking up, realizing that there's analytical gold buried deep in machine data -- and to mine it they'll need to mix traditional e-commerce and ERP-based big data sources.
So how can companies put the Internet of Things into operation, blending disparate data sources to create the hybrid analytics of the future?
The people factor
The challenge is made even more complex by the massive technological divide between big data in the Internet of Things and big data on the enterprise software side. The standards, uses and tools are different, as are the frequency and quantity of data.
For more on the Internet of Things
GE, Amazon look to the Internet of Things
The Internet of Things: A revolution of sorts
From Internet of Things into business benefits
There is a cultural barrier, too. In most companies, particularly in manufacturing, the assembly workers, mechanics and electrical engineers who work with shop floor and Internet of Things data never talk to enterprise app users or share data with them. They don't work together, they don't read the same publications and they don't attend the same conferences.
That makes it harder to take full advantage of a pan-enterprise big data opportunity. For most companies, it will take a change in business culture and in technology to make good on the promise that comes from blending two disconnected environments. And it's anyone's guess which realm will be the hardest to reform.
Putting it all together
The technology side itself is daunting enough. The ocean of data generated by "things that spin," to use a term coined by industrial data pioneer GE Software, are massive -- a single aircraft engine can churn out a terabyte of data in a single flight. The uses for this data, and therefore the sampling rates, data integration challenges and analytical models, are also variable. Some of the data must be analyzed in real time at the point of origin -- for example, an anomaly in the engine that signals an impending fire must prompt an immediate alert -- while other data might be used to build a complex predictive model that analyzes a subset of the engine’s sensor data over a month. Creating a platform for collecting, integrating and analyzing heaps of unstructured machine data is a colossal undertaking. To date, the body of standards needed is still more dream than reality. Companies like GE Software, Cisco and Texas Instruments are in hot pursuit, but it won't be easy.
A new platform has to make the most of the opportunity that comes from blending industrial data with back-office ERP and e-commerce data. If a company sees data that says it needs to repair expensive assets like jet engines and uses that information to order parts in its ERP system and schedule maintenance in its human resources system, it can reach whole new levels of efficiency and cost-effectiveness.
Seizing such opportunities requires blending corporate culture as well. Pioneering companies are already bringing the two sides together to strategize on what the next-generation analytics and predictive models they'll be able to build should look like. These conversations are taking place well ahead of laying the necessary technological foundations -- and for all the right reasons.
Getting to the point where the Internet of Things blends seamlessly with enterprise big data will be an arduous journey, but it's one that should start now. The good news is part of it will be easy: Existing business intelligence tools will adapt readily, building the new reports that this hybrid data environment will require. Bridging cultural and data infrastructures will take more time and investment in people and technology. The results will be worth waiting for.