The fine line between IoT data collection and privacy

When the Lifelog architecture development and analytics team at Sony Mobile Communications set up a program for capturing and analyzing data from wearable devices linked to the activity tracking app, their biggest challenge wasn't technical in nature. It was getting clearance from the company's legal department on the proposed data collection and privacy practices.

Jonas Sellergren, who manages the team, said at Strata + Hadoop World 2015 in New York City that it took some doing to balance "this opportunity and the risks a company exposes itself to with potentially sensitive data." And there's a lot of data involved: currently 5 terabytes per month, which Sony processes and analyzes in a Spark system running in the Amazon Web Services cloud.

To help ease the legal concerns, Hakan Jonsson, a data scientist on the Lifelog team, said they adopted an "obvious-data-usage principle" specifying they should only collect data that can benefit the app's users. That wasn't an academic exercise. In one case, he said, "we decided not to collect a piece of data that was useful to us [because] we couldn't find a use for it from the user point of view."

Data collection and privacy on the Internet of Things is a particularly big deal in consumer applications. During a Strata session at the conference, Charles Givre, a data scientist at consultancy Booz Allen Hamilton, detailed his use of a connected-car device sold by Automatic Labs Inc. If the data collected isn't protected properly, he said, "you can really start to get a scary level of insight into someone's daily life."

In an interview before the conference, Automatic Labs' vice president of software engineering, Rob Ferguson, emphasized that privacy is a top priority, even inside his company. "Everyone on my staff has training and clear guidelines that they're not supposed to look up information without a driver's consent," he said.