Get started Bring yourself up to speed with our introductory content.

IoT testing 1, 2, 3... Is this algorithm working?

As momentum builds toward a ubiquitous, connected internet of things world, an array of new smart devices are marching to market, like so many Galactic Stormtroopers. Wi-Fi thermostats and video doorbells; Amazon Echo and Google Home; even remote-controlled window shades… all manner of nifty gadgets to delight today’s tech-savvy consumer. Yet, while smart devices or “things” like these are certainly the focal point for interconnected systems, they are only the tip of the iceberg.

The nature of IoT has created a new reality in which physical devices and virtual interfaces coexist in a tangled web. How well a solution works — or fails to work — is dependent on a multitude of factors: back-end software, hardware, algorithms, security, interoperability and connectivity. Testing the performance of a smart device is only one piece of the puzzle. As a result, IoT requires a paradigm shift in the way that products and solutions are developed and tested.

Seek the truth

Just suppose that your innovative new solution is nearly ready for market after years of design work. The end-user device has undergone product reliability testing, and your mobile app has been put through its paces. The dream is coming to fruition.

But what happens if your algorithm calculations are off, even by just a little bit? How do you know for sure that your new solution will perform as expected within the broader ecosystem? The interaction of software, hardware and environmental conditions creates many unknown variables.

After you introduce a new IoT system is not the time to discover broken algorithms, interoperability issues or a buggy user interface. If you have to go back to the drawing board and start the whole development process all over again, you’ll lose valuable time, money and your competitive advantage.

The complex nature of IoT use cases means that connected products and solutions require rigorous and reproducible development and testing of software, hardware, sensors and algorithms before going to market. This can only be done under real-life conditions, or in a simulated real-world environment, where precise reproduction of deployment conditions enables identification and comparison of various parameters. In other words, you can achieve “ground truth” validation of reliable performance and accurate algorithms.

Lab-driven product development allows all the components to be tested and optimized collectively, ensuring appropriate reactions to inputs like touch, voice and motion tracking. This method also enables a wide range of communications and security protocols to be effectively tested, such as Wi-Fi, 4G LTE, 5G, Bluetooth, ZigBee, etc.

Moreover, beyond testing performance and interoperability, the ability to reproduce nearly any scenario in a lab setting also can aid in validating the business case and fine-tuning competitive differentiation. This enables device manufacturers, solution providers and enterprises to deliver initiatives to market more efficiently and with greater overall success.

Watch your step

So what are the most critical steps to take when testing IoT? In general terms, the broad testing and QA categories include data testing using firmware; testing data comprehensiveness; validating data accuracy; simulation-based testing; interruption testing; connectivity and interoperability.

However, to simplify the process, IoT testing can be broken down into three steps:

  1. Sensor: Algorithm validation
  2. Device: Connectivity/compatibility
  3. Use case: Functionality, performance and usability

In the more established realm of software development, a considerable amount of manual testing and QA processes are automated through continuous integration. With IoT, the interdependencies between software, hardware and the ecosystem make testing more complicated, but there are still some aspects that can be automated for rapid integration, speeding time to market without sacrificing quality.

For example, third-party APIs, GUIs and other system components can be virtualized to isolate performance issues and speed up testing cycles. Likewise, test management frameworks are helpful for unifying testing of apps across web and smart devices. And testing for compliance with regulatory requirements can be partially automated using static analysis tools.

There’s no question that the greenfield IoT market presents considerable opportunities for business success. But bringing a new use case to market also requires a significant investment in time and money. Before making a leap of faith, give yourself a head start over the competition by ensuring compatibility and end-to-end platform performance at all stages of the development lifecycle with ground truth validation and a smarter testing protocol.

All IoT Agenda network contributors are responsible for the content and accuracy of their posts. Opinions are of the writers and do not necessarily convey the thoughts of IoT Agenda.