Privacy Integrated Queries

robk — Thu, 06 Mar 2008 21:57:00 GMT

Data abounds in the digital age. Bits by the billions are collected on a daily basis from a variety of sources: Web services, financial programs, governmental agencies. Those who specialize in data mining and analysis could spend a hundred lifetimes sifting through such data, looking for patterns and clues that could help explain and fine-tune 21st-century life.

But they can't.

Much of that never-ending stream of data is privacy-protected--as it should be. Nobody wants their personal information accessible to any and all. Privacy is the cornerstone of the Internet era. Without it, society would devolve into digital chaos. But those same privacy protections deny experts access to all that massive, tantalizing data.

Frank McSherry aims to change all that. McSherry, a researcher at Microsoft Research Silicon Valley, is demonstrating, along with colleague Cynthia Dwork, a principal researcher at the same lab, a project called Privacy Integrated Queries, designed to enable the mining of huge data collections while not putting individuals' private information at risk.

"We're looking to put together a privacy-preserving data-mining platform," McSherry says, "tools that analysts can use, even without privacy training, to interact with and mine data from sensitive data sets that they wouldn't otherwise have access to.

"Cynthia Dwork and I have had a lot of prior experience with this privacy-preserving data analysis over the last few years. There's a lot of really formal mathematics behind it, but every time we do something new, we start from scratch in some sense: prove theorems, write papers--this is the model for convincing people things are private. We thought it would be smart to try to factor out the common technology we've been using in each of these results and package it in a framework that people could use to put together their own analyses."

One scenario in which such a platform could play a useful role relates to recent troubles in the financial sector.

"Some folks," McSherry says, "are really excited about finding what went wrong in the subprime collapse. Unfortunately, all that data is locked up--all the mortgage information, who bought what, at what rates--sealed up for privacy reasons. People can't sort out where the next collapse will be and how to counteract it, and that's unfortunate, that privacy is getting in the way, in some sense, of a real common-good happening. They didn't want to know who had what mortgage, but what parts of the country are most at risk."

Perhaps that episode would have developed differently if data analysts had access to the Privacy Integrated
Queries technology devised by McSherry and Dwork.

"We put together something that looks a lot like LINQ, Language Integrated Query, a sort of SQL-style, programmatic data access," McSherry explains. "To the user, it's basically indistinguishable. But under the covers, the privacy thing is going on, instincting about what you're asking for and communicating back with the data center, trying to determine if this is OK and pushing a lot of formal mathematics around, making sure that, at the end of the day, you haven't compromised privacy.

"The goal was to try to make it transparent to the users, so they didn't have to worry about what weird, funny machinations underneath are going on. They could just program against it as if it were LINQ."

But this latest project has one significant distinction that sets it apart from LINQ.

"Unlike LINQ," McSherry says, "you don't get to just enumerate a data set. You have to stay one step removed. You explain what you'd like to do with the data set and how you'd like it to be aggregated, and the results come back to you, perturbed a little bit. You get a little bit of noise, and the noise introduces uncertainty about the answer, which turns into this formal notion of privacy."

Privacy Integrated Queries could prove beneficial in a number of settings. The medical field, for example, could benefit greatly if the data could be safely aggregated without disclosing the associated personal information. Another potential usage could find takers within Microsoft.

"We have all sorts of data on who searched for what," McSherry notes. "It's good stuff. Microsoft would love to collaborate with external researchers, people interested in Web research who don't have access to the scale of data that we have, but we're really concerned about privacy. That sort of stalls research, to some extent, for Web researchers who don't have anywhere to go. They can't start up their own Web-search engine.

"With this sort of technology, we can really start the ball rolling on this type of work, work that people just haven't been able to do before."

Frank McSherry and Cynthia Dwork of Microsoft Research Silicon Valley in front of their TechFest poster on Privacy Integrated Queries.

TechFest Live! : privacy

Privacy Integrated Queries