Search results for 'app:weblogs' matching tag 'Microsoft'

Beautiful video about Microsoft External Research

Anonymous — Mon, 10 Aug 2009 05:00:00 GMT

I just noticed a beautiful video about Microsoft’s External Research (my previous group). A great view at very high-level of what the team is doing. I had the honor/pleasure of working with some of the folks in the video.

Although I have stayed in touch with everyone in External Research, I am really looking forward to finding an opportunity to working more closely with them again.

SciScope available (including source code)

Anonymous — Fri, 24 Jul 2009 05:00:00 GMT

This is another offering from my previous team and my good friend Bora (who I think is an upcoming star in Microsoft).

SciScope (see it live) is a prototype web application that allows data discovery from across multiple distributed heterogeneous data repositories. It leverages Bing Maps (formerly Microsoft Virtual Earth) and Microsoft SQL Server 2008 to support queries involving spatial, temporal and thematic constraints over an index of sensors operated by agencies such as USGS, EPA and NOAA as well as user provided data. SciScope leverages taxonomies stored as triples in SQL Server to provide search suggestions and for dealing with semantic heterogeneity between different data repositories.

This CodePlex release includes some desktop tools to simplify data publishing and content crawling for SciScope namely Catalog Publisher and Catalog Updater.

SciScope - Home

Wow... amazing photos submitted for Bing

Anonymous — Thu, 23 Jul 2009 05:00:00 GMT

I submitted few of my photos for the internal-to-Microsoft version of this photograph competition, few months ago. No surprise, mine didn't have a chance when they can get such gorgeous photographs!

Microsoft and GPLv2 - I bet you weren’t expecting this :-)

Anonymous — Mon, 20 Jul 2009 05:00:00 GMT

I didn’t know about it until last Friday when Alex Wade told me. Today, Microsoft announced the submission of GPLv2 code into the Linux kernel!!! Unbelievable. As Jim Zemlin characteristically said: “Hell has frozen over, the seas have parted”. :-))

You can read all about it in this NetworkWorld article (and I am sure all over the tech news).

Digital Past to Digital Presence

robk — Thu, 26 Feb 2009 06:00:00 GMT

Of the 150-odd demos in TechFest 2009, the best-named is Digital Past to Digital Presence, a collection of concepts from the Socio-Digital Systems group at Microsoft Research Cambridge. David Kirk, a post-doctoral researcher in the group, explains what ties the technologies together.

"Both bits of the booth are connecting with something's that's absent, somebody you can't be with, bringing things closer to together," he says. "With Digital Past, it's bringing you closer to your past, in two different ways. With Digital Presence, it's connecting with people you can't physically be with. In both cases, they're connecting with bits of your life that are distant in some way and bringing them together."

The demo features four discrete attempts to bridge these physical and temporal gaps: Family Archive, Time Card, CellFrame, and Wayve.

Family Archive: a table-like device with a screen built into the table top and a standard USB connection for peripherals, used for media management in the home.

"At least that's where we started," Kirk says. "It's always on, touch-interactive, multitouch display. We've built three of them and deployed them in family homes for a month at a time. We envision a world where you walk up, put a pen drive in, and content spills out into it. Once you've got digital content in, you can create containers for it.

"One of the things we're particularly interested in doing is to allow people to combine not only already-digital stuff, but to give them interesting ways of digitizing physical stuff that otherwise might not get integrated with those digital things. It's all about creating rich collections of media."

To that end, the Family Archive includes a camera stand on one side of the table that can photograph physical objects placed on the display and capture an image that can be added to a themed digital container, from, say, a vacation. The objects are rendered on a blank background to establish them as separate entities from other things that might be on the screen when the image is taken.

"You can have this integrated, easily digitized collection of all the photos you've taken, ticket stubs, postcards you might have bought," Kirk says, "and have a much richer archive of memories."

Further explorations of the concept could extend to 3-D capture of physical objects.

Time Card: "We're basically looking at ways of creating timelines," says Richard Banks, an interaction designer at the Cambridge lab, "to either represent your own life or lives of others. We have two key scenarios. One is the creation of timelines as a form of memorial for somebody's who's passed away. In this case, this is my grandfather, who left me a suitcase full of photos. I'm going through those photos and scanning them and adding metadata in order to create a timeline so I can better understand what it was he did during his life, as well as, to some extent, to honor him. It's like making a photo album about him.

"The second scenario we're interested in is about creating timelines on the fly about your own online activities. Our long-term goal is to ask: If you record these things about what you did, how will they impact you in 20 or 30 years? Wouldn't it be interesting, for example, to record the Twitter feeds you're creating now, so that in 20 years' time you can look back on them and reflect on what interested you. One of my goals with Time Card will be to create an object that for 40 years was recording my life and the things I was doing online, and at the end of my life, it could literally be unplugged and given to my daughter as an heirloom."

Currently, the Time Card focus is on archiving images and text. But, as Banks says, "that doesn't mean we're just talking about photos. Some of the content that I have in here are physical objects, like my grandfather's ribbon from when he was a pilot in the Second World War. Here's a letter about his deployment, and here's an insignia from the squadron that he was in. Although it's about imagery, it's about different kinds of imagery that come together to tell a story."

In incorporating images of physical objects, Time Card resembles Family Archive. That, Banks says, is deliberate.

"This general theme we're interested in," he says, "we're calling technology heirlooms. It's about just looking at technology generally and saying: 'What about 30 years' time? Where will this be? Who will care about it? What will people want to do with it?' "

CellFrame: a small, standalone, wireless display and communication device to bring the benefits of social networking to those not actively participating in it.

"There are certain people who might have access to the Internet and certainly might not have a wireless network at home," says Sian Lindley, currently a vendor working with the Socio-Digital Systems group. "but whom you might want to include in social networking."

The device consists of a phone in a frame. Those not inclined to adopt the latest technological trends could put such a device in their home and receive content sent to them. The scenario shown during TechFest displays a family social network, with channels for family members. By using touch input, family members could share content by dragging it across the display and dropping it onto another channel. Additional functionality enables a user to respond to received content via a simple user interface.

"It's just a very simple way," Lindley says, "of seeing who it is, not using the Internet, but still getting some of that information you might otherwise not get."

Wayve: another device, resembling a digital picture frame but incorporating technology that lets people connect playfully and creatively.

"This is like a digital sticky note," Lindley says. "I can create a message to be on display for my family, and I might leave this device somewhere like the kitchen where everyone's going to see it at a glance. It's connected wirelessly to the Internet, so I can also send and receive messages."

Wayve can send a photo to an e-mail address or a photo message to a mobile phone--or to another Wayve.

"What we expected," Lindley says, "is that this might be useful for household messaging, like 'I'm going to be late,' or to ask what was for dinner. But what we found when we deployed these was that people used it in creative and expressive ways. You can do things like play tic-tac-toe. People would do things like take pictures of their dogs and send them to their friends, and the friends would draw hats on the dogs and send them back.

"Just kind of playful, simple, expressive things as a way of connecting families."

Specification Inference for Security

robk — Thu, 26 Feb 2009 06:00:00 GMT

Aditya Nori, a researcher in the Rigorous Software Engineering team at Microsoft Research India, just gave me a brief overview of his demo, entitled Specification Inference for Security, and as he made repeated references to the poster in his TechFest booth, I though it would be instructive to share:

A higher-resolution version is available that enables you to enlarge the poster to read through everything, but the key to the project--a new algorithm that automatically infers explicit information-flow security specifications from program code--is located under Specification on the left-hand side of the poster, which defines the classification of nodes in a data-flow graph of program code as "sources," "sinks," or "sanitizers." A source is a node that returns tainted, bad data. A sink receives that bad data, and a sanitizer cleans the data, so that even if it receives tainted data, it does not pass it along. It's a case of garbage in/no garbage out.

"This project is about improving the quality of existing static-analysis tools for security," Nori explains. "Most automated tools for security rely on specifications, because they really need to know what they're searching for. Our job here is to go through the programs and automatically infer specifications in the program."

He points to the "Information flow vulnerabilities" portion of the poster and to the node named ReadData1.

"If you look at this code fragment," Nori continues, "here's a data-flow graph of the response to this code fragment. Information flows from this method or function call into this function call [Prop1]. If I just went though this graph and asked if there was something wrong, if there's a bug over here, it's hard to say, because you need more context.

"That's exactly what a static-analysis tool is going to say: You need to provide more information, and you need to say what the problem is. On the other hand, if I consider this a source, a producer of taint, and that one a sink, a consumer of taint, then the static-analysis tool will be able to say that the first part is a bad part. There could be a malicious user sending some data that screws up your database. If this [Cleanse] was a cleanser, which actually checks that whatever is being passed on to the database is safe data, then this part is OK. What the static-analysis tool needs to know, in addition to your program, is the role of every function in your program: Here's a function of source, here's a function of sink, here's a function of sanitizer."

In the real world of software development, though, that's an improbable scenario.

"It's really unreasonable to assume that if somebody is presented with 1 million lines of code, they're actually, mindfully going to go through the code and annotate every method as a source, sink, or sanitizer," Nori stipulates. "That's unacceptable. No developer is going to do it. So what we do is go through the code and automatically analyze the code and annotate every function as a source, sink, or sanitizer."

He turns to the Architecture section of the poster.

"Here's a high-level overview," he says. "We take the program, do static analysis, and convert it into a data-flow graph. The data-flow graph includes a bunch of constraints. For technical reasons, it's helpful to look upon these constraints as probabilistic constraints We take the data-flow graph, convert it into this probabilistic model, and feed that to a constraint solver. The solution to the set of constraints precisely tells us which methods in our program correspond to sources, sinks, and sanitizers."

And in a real-life test, the technique worked wonders.

"We ran our tool on 10 critical Microsoft business applications," Nori reports, "and we discovered 67 new sources, 25 new sanitizers, and 75 new sinks. The next step for us was to assess the quality of these specifications. Did these specifications really improve the quality of an existing static-analysis tool? We took a static-analysis tool that is being developer by Microsoft for security and ran that tool on these applications. The tool discovered 89 vulnerabilities, of which 20 were false positives. Then we ran the tool again with our new specifications, and we discovered 335 vulnerabilities. We were very excited about that. Another nice thing about our specifications is that they eliminated 13 of the 20 false positives from this set."

Such numbers grab attention within Microsoft, and there is a strong possibility that the specification-inference algorithm might be included in an upcoming product release. Successful technology transfer makes researchers smile, but Nori knows there is much left to be done.

"My collaborators believe this is a new way of analyzing programs, combining program analysis with statistical analysis," he says. "We are a long ways from applying this to other domains and program analysis. That's the future of this project. We look at this project as the starting point for combining program analysis and statistical analysis."

Language-Agnostic Search

robk — Thu, 26 Feb 2009 06:00:00 GMT

Some of the demos featured in TechFest 2009 were submitted by the Cairo Microsoft Innovation Center, and i got a chance to speak with a couple of Cairo researchers, Kareem Darwish and Motaz El-Saban, about their work.

"We're trying to enable multilingual search," Darwish said, "in the space of text documents and in the space of printed documents. In the case of printed documents, this is the OCRLess."

Then El-Saban took over.

"OCRLess is about language-independent technology," he said, "that allows you to search within scanned document images without the use of OCR (optical character recognition). Traditionally, if you have a document image, you would need to convert it into text using OCR, and then you can search. What we do as an alternative approach is take the text query and transform it into an image for rendering. Then we match it against the image document. It's based on image matching and indexing, and what we're showing here is five languages--English, Arabic, Chinese, Hebrew, and hieroglyphics."

Hieroglyphics ready to be searched.

It's an ingenious approach, upon which El-Saban expanded.

"The first step is to segment it," he explained. "The segment can be a character, a part of a character, or a word. in English, it's a character. We take all these segments and cluster similar shapes together in a completely unsupervised manner, then assign an ID to each cluster. Now, every page is presented, instead of by the characters, by a set of IDs, and we index this set of IDs into a regular search engine. With a query, we do the same thing. We render it into an image using the font of the book, then we segment it into pieces, and we actually assign, for each piece, the closest cluster ID.The query is a string of IDs, basically, and we go search for the string in the book. This makes it effective, because we're using image matching, and efficient, becomes we're using an underlying basic search engine.When we actually match the image query to an image inside the book, we're using template matching, an array of pixels."

So where might this work lead?

"There are a number of possibilities," El-Saban said. "Libraries could use something like this, or whatever entities sit on large volumes of documents, possibly written in many languages, for which you don't have an OCR. Another area of potential is handwriting search. You write your own notes by hand, and then, without even having to recognize your handwriting, you can still search them. I'm trying to sit with different contacts in Microsoft product groups to see if there is an interest to take this project in a specific direction."

Back to Darwish, whose project is called Trans-Bulletization.

"We're trying to enable people to search using English queries against documents in many, many different languages," he says, "and then present the results, not in the original language of the documents, but in a bulletized list, a summary that removes superfluous words from the English translation and puts the information into bulletized form. The user then can very quickly learn what the document is about and consequently make a decision whether they want to invest more time reading the full translation or the original document."

He went on to discuss how entire documents are boiled down to bulleted lists.

"The key technology is all the documents we are going to search, we translate into English first, and then deploy the bulletized technique," Darwish said.."Actually, it's a sentence-reduction technique. To reduce the sentences, we use a dependency parser that recognizes the main verb, the subject of this verb, and the object. Given that these are the core components of the sentence, then we find all the other pieces--prepositional phrases, modifiers, and so forth--then make a judgment about the information content of these pieces. If they have low information content, then they're candidates for removal, but before we remove them, we have to make sure that they don't break anything in the sentence. We won't remove a noun phrase unless that, if we remove it, it won't break the flow as measured by language models."

Such help could provide a boon to many in this information-saturated age.

"For people who work in an organization that requires sifting through a lot of documents in many, many languages, this would be really useful," Darwish concludes. "They're getting all the information content in a shortened version, so they can scan lots of documents very, very quickly. A typical user might be a reporter who wants to see how people look at a particular issue across different countries. As the person enters a query across the countries he's interested in, he gets articles from Japan, the Middle East, from China and Europe, and so forth, and then he can see all the different views at the same time, in bulleted lists. He can do this very, very efficiently and very, very quickly."

Motaz El-Saban (left) and Kareem Darwish in their TechFest booth.

Mining Personal Relationships

robk — Thu, 26 Feb 2009 06:00:00 GMT

Zaiqing Nie of the Web Search & Mining Group at Microsoft Research Asia is demonstrating a project called EntityCube, an English-language version of a wildly popular Chinese project called Renlifang.

In Mandarin, "renlifang" means "three people." The nomenclature is intentional, because the project, and its EntityCube manifestation, is all about demarcating the relationships among a group of people.

"EntityCube automatically summarizes relevant information about people," Nie explains. "For example, here, we've summarized the search results for Bill Gates. You can see that Steve Ballmer is here, IT leaders, celebrities. [He clicks on the Ballmer icon.] You can also have a Web page just for Steve Ballmer. We find all related people, the type of related people, news about him.

"The important part is that we automatically generate this. It's not manual. You can see much of this information on the Web, but human beings do that work. Our input is Web pages. The output is this summarization page and this social graph."

The EntityCube project site offers a good explanation of the technology:

"The need for collecting and understanding Web information about a real-world entity (such as a person or a product) currently is fulfilled manually through search engines. But the information about a single entity might appear in thousands of Web pages. Even if a search engine could find all the relevant Web pages about an entity, the user would need to sift through all the pages to get a complete view of the entity. EntityCube is an entity search and summarization system that efficiently generates summaries of Web entities from billions of crawled Web pages. The summarized information is used to build an object-level search engine about people, locations, and organizations and explore their relationships."

The value in such an approach is confirmed by its embrace by the Chinese public. The project page goes on to note: "Renlifang has been well received by Chinese Internet users and mainstream media in China, with positive comments and millions of page-views during peak days."

Such a reception validates the extensive effort the project has required.

"We've been working on this technology, object-level search, for five years," Nie says. "We got very positive feedback from the Chinese version. Renlifang is very popular right now. People like this idea of mining these results and integrating the search. Renlifang and EntityCube are the first time these mining techniques have been combined with search, and as far as scale and functionality, we are the best."

And the technology is not limited to the human domain. Other entities can be searched and summarized, as well.

"You can have other queries besides name queries," Nie says. "For example, the topic 'climate change.' You can find people who are connected with climate change.

"We're showing EntityCube People, but we are going to produce other ones, like EntityCube Academic, which will be the next version of Libra, and EntityCube Products. Eventually, EntityCube.com will be the portal to other object-level search verticals. That's the long-term goal of our research vision."

The next step for Nie and colleagues is to continue collecting feedback and refining the approach. The interest spawned by the TechFest demo can only help.

"We are trying to release research surveys that invite people to use it and provide feedback," Nie says. "From our feedback today and yesterday, we've found that English speakers also like this idea a lot."

Avatar Revealed!

robk — Wed, 25 Feb 2009 06:00:00 GMT

To be honest, it wasn't too difficult to track down the woman who served as the model for the virtual receptionist in the Situated Interaction demo. Her name is Andrea, and she works as a group assistant in Building 99, the headquarters of Microsoft Research Redmond.

Andrea isn't actually a receptionist; she just plays one on computer screens.

But she used to be. That's how she encountered Eric Horvitz.

"They originally told me they were going to do it when I was still a receptionist," she recalls. "I think I was talking to Eric. They had a different avatar originally, and they wanted to update it because it was an older version. They were excited about the project and didn't want to revamp it, essentially. So we just started talking about it, and they said, 'Hey, how would you feel about that?' and I said, 'I think it would be fun.' So I did it."

Horvitz says Andrea, whom he calls "my favorite admin," seemed a natural.

"I thought she’d be a great person," he says. "She was willing to help. She’s friendly and smart. She’s expressive. We had to catch her expressions. We have a whole folder, a day of video, where she went through all the sequences: awe, surprise, disgust, all the phonemes. We just started rendering them, but we have her on tape, so we can go deep."

Happiness, sorrow, surprise, amusement: Andrea's range of emotions bring the virtual receptionist to life.

Andrea agreed to sit for a couple of photo shoots, which ranged from the glamorous to the peculiar.

"For the first photo shoot," she recalls, "I came in, and they did all my makeup, which was fun! They had me sit in the big room at Microsoft Studios, on this stool, and they would get really close and take pictures all the way around, and I would move a millimeter, and they would say, 'Your posture! You're slumping!' The first one, I had to do a lot of expressions, which was really hard. [laughs] I don't know how to make a sad face! Was I supposed to be overly expressive? I had to do the visemes, I would start a word and then stop, so they could catch the shape of my mouth. Then they had me do a live recording. People would come up, and I would react the way I normally would at the reception desk. They videotaped several of those interactions.

"The second one was just primarily pictures and trying to get more angles--and really close up to my teeth. I had to hold my mouth open, and they got really, really close to my teeth. I think they wanted to get the texture. That was funny."

Andrea first got a glimpse of her avatar last September, right after the second photo shoot and right before Microsoft's annual company meeting.

"It was crazy," she says, "because it looked so much like me. I was really surprised, because Eric was saying: 'Now, you know, it's really at the bottom level. We haven't really refined it yet.' But I was surprised at how good it was, actually."

Having been "discovered" at the Building 99 receptionist desk, Andrea, naturally, had a few suggestions to offer to the researchers.

"When we were shooting the live footage," she says, "and I was reacting as I would as a receptionist, yeah, there were times when I said, 'No, I probably would do it this way or I would respond this way.' Some of the scripts they had me change a little bit, but not too much. I guess that would be the little added bit. They asked some questions about certain things, but it wasn't too much."

The virtualization experience has given Andrea a taste of being a celebrity of sorts in Microsoft circles.

"Probably like once a week," she smiles, "somebody says, 'Oh, so you ...' and I'm like, 'Yeah, it's me.' [laughs] But most people don't recognize me unless they knew me before. It's usually people I know who recognize me. They'll come back and say, 'A few months ago, I saw this thing ...' 'Yeah, it was me.' "

Sounds like an enjoyable adventure.

"It's pretty good," she says. "It's fun. I like it. It allows me to bond with people in a different way. They recognize me, so it's almost like an icebreaker sometimes.That's cool."

But there was one tiny drawback, pertaining to a trivia-game research project in which her avatar was used.

"I took the trivia test, because they had it in the kitchenette to gather data," she says, "and my virtual self beat myself. My virtual self was smarter than my actual self!

"That was a humbling experience."

Dan Bohus (left), Eric Horvitz, and the two Andreas

The Virtual Receptionist

robk — Wed, 25 Feb 2009 06:00:00 GMT

One of the more intriguing demos on display during TechFest 2009 is called Situated Interaction, a project by Eric Horvitz and Dan Bohus of Microsoft Research Redmond that aims to enable a new generation of interactive systems that can reason about their surroundings and provide an engaging, appropriate set of responses.

The project includes a number of different examples of the technology's potential, the most notable of which presents a virtual receptionist, or situated conversational agent, that can act as a front-desk receptionist, with a lifelike avatar interacting with visitors, making shuttle reservations or welcoming and registering guests. Horvitz, a principal researcher and research-area manager of the Adaptive Systems and Interaction group, explains:

"What we're doing is about understanding some general principles about how you integrate computation into the flow of tasks in an everyday manner," he says. "The receptionist domain was just the first one we tried out, because we had a couple of interesting issues there: many people being handled by a receptionist,:recognizing who's in the same group and requiring the same task, such as taking a shuttle. How long are people waiting? Are they getting frustrated? It's about managing people's attention."

Below is a peek at the virtual receptionist at work. At top left are visualizations of the resources being utilized by the technology. Below that, the receptionist avatar is presented, alongside options for the task at hand. At bottom left is a side view of the persons being recognized--in this case, Horvitz (left) and Bohus. On the right, the scene is being analyzed by the technology: faces, clothing, affiliation, level of engagement, group, level of attention, last speaker, active engagement, goal, time.

That's just one potential manifestation of the work. Another, Bohus explains, concerns a video trivia game.

"We created a trivia-game system," says Bohus, a researcher in the Adaptive Systems and Interaction group. "We put it on the third floor of Microsoft Research, and we're getting data because people are engaging with it after they grab their coffee, for a little break."

Horvitz elaborates:

"The trivia game was set up mostly as a way of exploring engagement. We actually rolled out the receptionist into the hallway, and she just sat there, watching people, making eye contact, People would come over, and she'd say: 'Oh, are you interested? Come over and play.' The trivia game is just a way to explore engagement and predict when somebody is interested in engaging versus not. It'll play the game with you, but if you don't do very well, it'll basically move away from you and say to somebody else: 'You want to help this guy? He's not doing very well,' and get somebody in to help you. It's very cute."

Then there's a third proof of concept in the works.

"The other task we're working on right now," Horvitz adds, "about which we're very excited, is Personal Concierge, which we're not showing today but we showed [Monday] to [Microsoft Chief Research Officer and Strategy Officer] Craig Mundie in a private session. He's very supportive. This lady, Laura, is stationed by my door, my office, and she handles my whole schedule and the statistics of my comings and goings, People come to my door, and she says: 'He's busy right now. Hang on a minute," or "He's five minutes late, he went off the network a minute ago.' She knows my schedule and knows how to negotiate with people."

It's entertaining, interacting in human fashion with a machine, but behind the scenes, a whole host of artificial-intelligence forces are at play: speech recognition, detection and tracking of persons and groups, intention recognition, attention and engagement modeling, and natural-language processing.

"This whole project is about weaving together lots of components into a bigger whole," Horvitz says, "from natural language to computer vision, the acoustical microphone--weaving it together to see if we can get a bigger whole than the sum of its parts.

"I think you'll see a lot more [human-computer interaction] based in dealing with a presence like this, in a very natural way, with gestures and expressions, eye contact. The technology itself could be used for everything from bank receptionists to personal secretaries to productivity assistants to teaching kids in a very hot way, not like a cold, intelligent tutoring system, but as a very hot way for engaging many students in a classroom."

He then offers one more possible implementation of his and Bohus' work:

"Something we will see is technology going into elevators," Horvitz states. "I would say there are three generations of elevators: man on a chair, simple buttons you press, and then the elevator that understands when you put your hand in the door. It waits for you while you're talking. It has a camera on the inside and a camera on the outside, and it monitors facts and conversations to know when it should hold the elevator or come and take you somewhere. It might even be overlaid into other invisible technologies, like doors that open as you approach but don't open when you're walking past them, that understands trajectories."

All this is suitably impressive, of course, but there is that other question: Who was the person whose real-life presence was pressed into service as that receptionist avatar? We're on the case.