Microsoft demonstrates touch screen for any surface [video]

By Tom Warren, on 18th Oct 11 3:34 pm with 30 Comments

Microsoft Research OmniTouch

Microsoft is developing a new technology in its research labs that will allow users to touch any surface as if it were a touch screen.

The technology is called OmniTouch and was born in Microsoft’s Research labs. The software giant has created a prototype unit that is wearable and projects images onto various surfaces. The interactions can take place on the go with no calibration required. Imagine holding up your hand and dialling a number in the palm of your hand, that’s how OmniTouch works. ”We wanted to capitalize on the tremendous surface area the real world provides,” explains Hrvoje Benko, of the Natural Interaction Research group. “The surface area of one hand alone exceeds that of typical smart phones. Tables are an order of magnitude larger than a tablet computer. If we could appropriate these ad hoc surfaces in an on-demand way, we could deliver all of the benefits of mobility while expanding the user’s interactive capability.”

The custom camera works in a similar way to Microsoft’s Kinect system but has been modified to work at short range. A user simply carries the camera on their person and the system will project user interfaces onto various surfaces. “We wanted the ability to use any surface,” Benko says. “Let the user define the area of where they want the interface to be, and have the system do its best to track it frame to frame. This creates a highly flexible, on-demand user interface. You can tap on your hand or drag your interface out to specify the top left and bottom right border. All this stems from the main idea that if everything around you is a potential interface, then the first action has to be defining an interface area.”

Microsoft is demonstrating the technology as part of its presentation during UIST 2011, the Association for Computing Machinery’s 24th Symposium on User Interface Software and Technology, being held Oct. 16-19 in Santa Barbara, California.

  • Siddarth Malhotra

    The Sixth Sense by Pranav Mistry, anyone?

    • deathmore34

      I forgot the name, but yes, I was about to make a comment on that and thank you for naming the product for me.

    • Eingoluq

      That’s what I was thinking.

    • trashoner

      yes but this definetely takes it to a new level with more touch control than sixth sense. It also doesn’t implement barcode and object scanning or other things that come with sixth sense. I like this aproach better. It’s simpler and it seems to be far more accurate with the kinect camera. I like what sixth sense is doing but they’re making the device way too complex for the masses. 
      here is sixth sense doing something they better patent before MS does, that is if they haven’t already.

    • Benjamin Ostanciaux

      In fact that’s the same since Pranav Mistry and Microsoft are partners now. So OmniTouch = Sixth Sense + Microsoft’s Kinect Technology.

    • Olisaebuka Maduka

      Pranav has a history with Microsoft, as a UX researcher around 2005 and  then with Microsoft Research as a research intern according to his linkdin page, watch and you’ll Microsoft implementation of some of his great ideas into their future vision of office. You could also watch Pranav Mistry at TED

  • Anonymous

    That’s awesome over about 10 years it’s normal. 

  • Anonymous

    Nope, is there a video about it?

  • Anonymous


  • Guest

    I hope they will patent the technology so that Apple won’t steal it again and sell is their innovation.

    • Guest

      Well at least Apple will put it into a product that does something for you. MS will get a patent and then wait until Apple or someone else has a huge success with it. Then they’ll say “but we had a research project on that before you”.

    • Tom W

      Yup, it’s frustrating to see that. Microsoft does some of the best research work in the tech industry and yet Kinect is the only major product recently that has taken the consumer market by storm.

    • Guest

      And Kinect isn’t a pure MS R&D play. The base technology comes from PrimeSense, which MS licensed. In fact if Apple hadn’t passed on PrimseSense earlier, Kinect would never have happened. And even then it only came about because the product group fell in love with the technology and approached MS’s R&D group to help them productize it. In other words, it was an aberration of the normal process in many respects.

    • trashoner

      im pretty sure Sixth Sense already owns most of those patents right now.

    • Guest

      How can you be “pretty sure” about that, when you have no idea how long MS has been working on any of this or what patents they may have already been granted?

    • Bassettjb

      I remmeber reading that the guy that developed Sixth Sense was speaking to Microsoft when Sixth Sense was demostrated at TED.

    • Anonymous

      He is involved with the project at MS.

    • Gamer

      Sadly, there’s cross patent between Microsoft and apple.

  • McAkins Online

    I am so sorry for Microsoft, for pioneering technologies that are too advanced for their time only to have Apple steal in the future and claim the awesomeness.

    • Guest

      The main objective of pioneering technology is to allow you to leapfrog the competition and earn higher profits as a result. It’s not to gather a patent arsenal that can later earn some relatively minor royalty revenue by patent trolling those who succeeded in making those market breakthroughs.

  • John Lueders

    I like this one better than the pockettouch video/post.  This looks promising.  A few years back, Bill Gates talked about every surface being a display (one of his CES talks I believe) and this brings us closer to that reality.

  • Jinge

    That is a nice tech, but i think it already exists some virtual keyboards like that (but on flat surface) where you project your keyboard.
    So MS please show us some nice things!

  • Kazi

    It seems like, Pranav Mistry Sixth Sense is stolen, I wish he should have the sole patent of this technology. Sad example from microsoft!!!

    • Guest

      Fo troll.

    • BigChiefSmokem

      How is it stolen if you have the guy at TED saying he is open-sourcing it.

      If Microsoft monetizes it its bad but if Apple does… give me a break. Same sob story with the Kinect technology.

  • Dhaoracle

    This would be so awesome with some glasses that would project an image to touch and a hologram. Cant wait Microsoft for  you to perfect this..

  • Owais_503

    this is what apple will do with this research and apple might do other things with it to.

  • Lord Jem

    it has potential.. all they need now is a more practical and time saving implementation of this technology 

  • Heishun

    other patent design seem more advance & user friendly than it, Microsoft can consider to buy it.