TechFest Live! : Silicon Valley

The Value of TechFest

robk — Wed, 25 Feb 2009 01:57:00 GMT

I just ran into Roy Levin, Microsoft distinguished engineer and managing director of Microsoft Research Silicon Valley, in the Rainier/St. Helens Room at the Microsoft Conference Center. We found ourselves marveling at Commute UX, one of the TechFest 2009 demos I'll be posting about soon, and I asked him about what TechFest means for him and his lab.

"The value of TechFest," Levin said, "is for people to get a feel for the breadth and the depth of the innovation that goes on in Microsoft Research. In the room we're in right now, there is a sampler, really, of the larger event, and just wandering around this room, you see stuff from so many different areas of computer science and their applications to so many different areas of real life. For me, that's where the real impact of this event is: just feeling-- in a very direct way, a very visceral way--how the work that we do can make the world better through computing."

Levin's Silicon Valley lab specializes in research on distributed computing, and that means TechFest plays a particularly valuable role.

"There are a couple of demos here, in this room, from our lab that focus on understanding and modifying the way that complicated distributed systems work, in data centers in particular," he said. "The benefit, from my point of view, from participating in this event is that people get a glimpse at the problems, the very difficult problems, that we face in trying to harness the world of parallel computing and make it useful, in the form of cloud services, trying to operate these very large services and provide things on a global scale."

Finally, Levin noted, there's a special quality to the event that makes it loom large on researchers' annual calendars.

"There's a special place in my heart for TechFest," he stated, "because, obviously, it's a hometown event, compared to going to a conference or some other meeting. It also is 'hometown' in a different way, because members of Microsoft Research come in from all over the world. We get to meet with people that we don't get to see the rest of the year. That's really great."

Roy Levin (left) and Xian-Sheng Hua of Microsoft Research Asia chat during TechFest.

Improving Data-Center Performance

robk — Tue, 24 Feb 2009 21:50:00 GMT

Moises Goldszmidt (above), principal researcher at Microsoft Research Silicon Valley, is showing a pair of demos, in conjunction with lab colleague Mihai Budiu, that examines performance in data centers.

"The challenge," Goldszmidt says, "is: How do I summarize thousands of machines and hundreds of metrics and find the key elements over that huge space that's giving us surprises, such that I can let it retrieve that fingerprint? How do I do that automatically?"

The demo is called Predicting Problems in the Data Center.

"We are using very sophisticated machine-learning techniques," Goldszmidt states, "that build automated models that are able to extract the main characteristics of each one of these crises."

The value of such work is readily apparent.

"Eighty percent of the time, we're predicting one hour in advance a set of actions we need to do to mitigate a problem," he says, resulting in "less downtime, less latency for our clients using our services. Our services are more efficient to run, because we don't have to have that many people look at the problem."

The second demo in the booth is named Profiling the Performance of Distributed Systems. It features a colorful analytics engine that enables the monitoring of vast data centers.

"Once you have a large cluster that you can run your services and applications on, it's very hard to understand, if something goes wrong, what's wrong," explains Budiu (left). "It could be a hardware problem. It could be your application has partitioned the data wrongly. It could be something in between, such as the network being down.

"This is a tool which pulls a lot of metrics from the machines in your cluster and allows you to easily visualize the data and find correlations in the data. You can assign colors to metrics and drag and drop the colors into other windows to see how the metrics correlate with other metrics.You can assign colors according to how many CPU cycles are utilized and immediately drop the color to see where the high CPU cycles are being used, in which machine."

In both demos, Goldszmidt declares, the objective is the same.

"Reduce the time to understand what the heck is going on," he says. "Build a better service. That's the final goal."

Privacy Integrated Queries

robk — Thu, 06 Mar 2008 21:57:00 GMT

Data abounds in the digital age. Bits by the billions are collected on a daily basis from a variety of sources: Web services, financial programs, governmental agencies. Those who specialize in data mining and analysis could spend a hundred lifetimes sifting through such data, looking for patterns and clues that could help explain and fine-tune 21st-century life.

But they can't.

Much of that never-ending stream of data is privacy-protected--as it should be. Nobody wants their personal information accessible to any and all. Privacy is the cornerstone of the Internet era. Without it, society would devolve into digital chaos. But those same privacy protections deny experts access to all that massive, tantalizing data.

Frank McSherry aims to change all that. McSherry, a researcher at Microsoft Research Silicon Valley, is demonstrating, along with colleague Cynthia Dwork, a principal researcher at the same lab, a project called Privacy Integrated Queries, designed to enable the mining of huge data collections while not putting individuals' private information at risk.

"We're looking to put together a privacy-preserving data-mining platform," McSherry says, "tools that analysts can use, even without privacy training, to interact with and mine data from sensitive data sets that they wouldn't otherwise have access to.

"Cynthia Dwork and I have had a lot of prior experience with this privacy-preserving data analysis over the last few years. There's a lot of really formal mathematics behind it, but every time we do something new, we start from scratch in some sense: prove theorems, write papers--this is the model for convincing people things are private. We thought it would be smart to try to factor out the common technology we've been using in each of these results and package it in a framework that people could use to put together their own analyses."

One scenario in which such a platform could play a useful role relates to recent troubles in the financial sector.

"Some folks," McSherry says, "are really excited about finding what went wrong in the subprime collapse. Unfortunately, all that data is locked up--all the mortgage information, who bought what, at what rates--sealed up for privacy reasons. People can't sort out where the next collapse will be and how to counteract it, and that's unfortunate, that privacy is getting in the way, in some sense, of a real common-good happening. They didn't want to know who had what mortgage, but what parts of the country are most at risk."

Perhaps that episode would have developed differently if data analysts had access to the Privacy Integrated
Queries technology devised by McSherry and Dwork.

"We put together something that looks a lot like LINQ, Language Integrated Query, a sort of SQL-style, programmatic data access," McSherry explains. "To the user, it's basically indistinguishable. But under the covers, the privacy thing is going on, instincting about what you're asking for and communicating back with the data center, trying to determine if this is OK and pushing a lot of formal mathematics around, making sure that, at the end of the day, you haven't compromised privacy.

"The goal was to try to make it transparent to the users, so they didn't have to worry about what weird, funny machinations underneath are going on. They could just program against it as if it were LINQ."

But this latest project has one significant distinction that sets it apart from LINQ.

"Unlike LINQ," McSherry says, "you don't get to just enumerate a data set. You have to stay one step removed. You explain what you'd like to do with the data set and how you'd like it to be aggregated, and the results come back to you, perturbed a little bit. You get a little bit of noise, and the noise introduces uncertainty about the answer, which turns into this formal notion of privacy."

Privacy Integrated Queries could prove beneficial in a number of settings. The medical field, for example, could benefit greatly if the data could be safely aggregated without disclosing the associated personal information. Another potential usage could find takers within Microsoft.

"We have all sorts of data on who searched for what," McSherry notes. "It's good stuff. Microsoft would love to collaborate with external researchers, people interested in Web research who don't have access to the scale of data that we have, but we're really concerned about privacy. That sort of stalls research, to some extent, for Web researchers who don't have anywhere to go. They can't start up their own Web-search engine.

"With this sort of technology, we can really start the ball rolling on this type of work, work that people just haven't been able to do before."

Frank McSherry and Cynthia Dwork of Microsoft Research Silicon Valley in front of their TechFest poster on Privacy Integrated Queries.