Manage Learn to apply best practices and optimize your operations.

IoT and testing: The new rules of engagement

The internet of things market continues to expand rapidly with the installed base of IoT devices now forecasted to grow to a staggering 31 billion by 2020. Traditional testing approaches are simply not going to be able to test this new massively interconnected world of different devices, created by different companies, using different technologies, all communicating to deliver digital user experiences. A fundamentally new approach to testing is needed.

Today, most software teams only test what they build, i.e., they only test the software components that their developers have written. All other software components are “out of scope” of testing, even software components that are critical to the product and the user experience (such as the network the product uses to communicate). In the IoT world, this approach simply isn’t good enough. All products in the IoT world involve multiple products from different vendors interacting to deliver a digital user experience and a specific value proposition, and vendors need to start taking responsibility for — and testing! — the whole value proposition, not only the small piece of the pie that they developed. This requires the established rules of software testing to be ripped up and replaced in a hyperconnected world.

IoT is all about collaboration — and that includes how to approach testing. In a hyperconnected world, you need to be able to test other people’s technology to assure the experience — and this requires a huge mind shift. Take the connected fridge for example. The consumer’s only concern is that the fridge is constantly stocked with milk, nothing else matters to them. This requires the entire ecosystem that is part of fulfilling this need to work flawlessly together: From the RFID chip on the milk bottle to the supermarket to the payment platform to the fridge manufacturer. To achieve this end goal, it requires you to share test assets and environments between all involved parties in meeting the user expectation.

The ubiquity of virtualization and now containerization make this possible. For example, the fridge manufacturer can create a test environment in a container and pass that to the milk producer who has his own milk-testing container. He can then plug the containers together and test the system as a whole. Each container requires a clean interface as to how it interacts with the other test containers.

The sheer scale of IoT has killed manual testing, and the need to collaborate has killed code-level testing. You can’t expect to have access to other people’s code. For example, the milk manufacturer is not going to share its source code, but it will share a container. Put simply, IoT is the final nail in the coffin for code-level testing.

The new rules of engagement

So what does testing look like in this brave new world? In a hyperconnected world of digital user experiences, you must adopt a user centric approach to testing.

I believe there are five keys to success:

  1. Test through the eyes of the user and APIs
  2. Test all aspects of the user experience
  3. Monitoring is now testing
  4. Extend automation and analytics beyond test execution
  5. Report test status in terms of the user experience

With IoT, the days of testing what you build are over. You need to focus on the value proposition and test everything that is part of the user experience — test everything that you touch. For IoT to realize its dreams it must adopt a user-centric approach to testing.

All IoT Agenda network contributors are responsible for the content and accuracy of their posts. Opinions are of the writers and do not necessarily convey the thoughts of IoT Agenda.