I’ve been doing this software engineering thing for nearly 25 years now. During all but maybe the last five years, I’ve used just a few common design patterns and building blocks to do most of my work. I’m oversimplifying a bit, but I have to admit that the building blocks were pretty similar across a variety of projects. Improvements in hardware capabilities — released about every 18 months or so — were the true driving force behind software advancement. Software design didn’t have to change too much because the hardware enabled it to simply “work better” or “do more.” As a result, I’d say I gained a reasonable sense of mastery building similar solutions over and over. I honed and improved with each iteration while the design and toolchain remained relatively stable.
That time has come to an abrupt end.
The explosion of cloud
The last five years have felt like a different world altogether. Cloud technology has ushered in an era of combined software and hardware innovation at a remarkably fast pace. The explosion of cloud is happening so fast and it’s so wide-reaching that it’s difficult to appreciate. Cloud innovation is enabling developers to do things that simply were not practical or even possible just a few short years ago. It’s amazingly empowering yet unnerving at the same time. Just when you start to gain a sense of mastery, something supersedes it, leaving you with a shiny new learning curve and a tough choice. The choice to use the “latest and greatest” could be your keystone or leave you with a pile of technical debt.
The fact is that I’m spending less time, almost none, futzing with the physical stuff. This allows me to spend nearly all of my time innovating at the application level. That aspect is fantastically liberating. Less time setting up means more time focusing on core application value with quicker iterations between releases. This focus on core application value is happening across several layers of the stack. Everyone is wasting less time with setup, leaving more time to think and truly innovate. This has generated an enormous wave of new tools, services and capabilities which are, in fact, fueling each other.
The reality of compounding innovation and microservices
Building innovation on top of innovation has enabled leaps of progress in a short period of time. The net effect is a layering of new applications on top of new services, on top of new hardware, often pulled together with new connectivity. That’s a great deal of “new.” Significant improvements in process have helped minimize risk. However, it’s worth noting that nobody has much experience managing all of these blocks working in concert over time.
The whole point here is that this change is happening quickly. It doesn’t leave much time to gain mastery or learn from mistakes before it changes or is replaced entirely. That churn can equate to outdated components left scattered throughout a distributed solution. Updating something in production may be something like a scary Jenga tower of technical debt. That’s where engineers pick the least risky component to update so they don’t bring down the production system.
Handling scale is only part of an IoT technology. It’s equally as important to maintain and improve that technology over time. It’s especially so in the case of IoT, where a device has a lifespan which will likely span several generations of the supporting technology. The pilot phase of an IoT system often deprioritizes lifecycle management. Don’t let that tower get too tall before considering how you will manage change.
The churn of cloud IoT development
If you feel like your development skills are falling behind despite the fact that your skill set has broadened, you’re not alone. The current pace of development is unprecedented during my time. I’m hearing similar perspectives from colleagues doing cloud development. There’s this constant buzz that you’ll get left behind if you’re not using the latest “cool tool”. Cloud tools and services used in IoT systems are changing so fast that it’s exceedingly difficult to gain mastery and stay current at the same time.
We should expect this wave of innovation to continue and get even faster. The overwhelming advice is simply to get good at designing for change. The reality is that we have to embrace and build for it. It seems this realization of change is what has fueled the microservices buzz, which, in part, helps isolate the impact of frequent updates or “churn” of underlying components. Simply put, it’s easier to manage change for a small service. Designing smaller, functional components using skill set-proficient teams can help mitigate lack of mastery for something new. Limiting the scope of the service or component helps those teams “get deep” fast without having to understand all of the details of a big complex technology at once.
The age-old guidance to take the time and consideration to build a strong foundation still holds true. Just make sure that foundation is on castors so you can swap it out or move it without tearing down and rebuilding everything on top of it.
All IoT Agenda network contributors are responsible for the content and accuracy of their posts. Opinions are of the writers and do not necessarily convey the thoughts of IoT Agenda.