Microsoft’s future Kinect 2 device will reportedly lip read and measure emotions

By Tom Warren, on 28th Nov 11 2:38 pm with 46 Comments

Xbox Kinect

Microsoft’s next-generation Kinect will feature lip reading support, according to reports.

The rumored support will allow developers to make use of an overhauled motion sensor and voice recognition unit. Euogamer reported on Monday that one development source indicates that the Kinect 2 will be powerful enough to lip read, detect when gamers are angry and even determine which direction they are facing. Kinect 2 is also expected to support the tracking of pitch and volume of player voices and facial characteristics to better measure their emotions.

The current Kinect unit has been updated several times to improve its camera tracking. Recently Microsoft launched its Avatar Kinect technology which allows the sensor to track mouth and eyebrow movements. There’s been a number of rumours suggesting the company is also working to build in finger tracking to the next-generation of Kinect. Eurogamer reports that the current version isn’t powerful due to its use of USB technology. Future versions will utilise a faster technology to provide more data to the Xbox.

Microsoft’s Xbox console allows Kinect users to control TV shows using their voice or by gestures. The software maker is currently on the verge of delivering a massive overhaul to its dashboard software to provide nearly 40 different live TV stations. Microsoft is also preparing to make new Kinect for Windows hardware available in early 2012. The optimized hardware includes components for PC scenarios. Microsoft is planning to shorten the USB cable to ensure reliability across a range of computers. The company will also include a small dongle to help ease coexistence with other USB devices. The new firmware updates for the Kinect will enable the depth camera to see objects as close as 50 centimetres in front of the device without losing any precision. Microsoft calls this “Near Mode” and the company hopes it will enable a whole new class of “up close” apps that have not been made available previously.

  • Josè Daniel

    Microsoft finally found its own niche and it’s making the most out of it! About time to stop following Apple’s footsteps!

    • Anonymous

      Yes, because before Kinect, Microsoft was just copying Apple’s console products and strategy.

    • Test1ngi23

      When did Microsoft get into PMPs? 5 years after the iPod. (Zune)

      When did Microsoft get into touch-centric smartphones? 3 years after the iPhone. (WP7)

      When did Microsoft get into touch-centric tablets? 2 years after the iPad. (Win 8)

      I’m not saying MS is copying Apple but to say that Apple is not having a strong influence over MS’s consumer product strategy is ridiculous.

    • Eingoluq

      Microsoft was busy with multitouch with surface before apple started research in multitouch, so moot point.

    • Anonymous

      Hmm, did you forget about Microsoft’s Windows Mobile line of PDA’s and phones?  The HP IPAQ was around a lot longer than the iPod or iPhone.

    • Test1ngi23

      @sniperboy:disqus    In that case, did you forget about the Apple Newton? It came out in 1993. 1993 is a lot earlier that 2000.

    • Guest

      When did Microsoft get into PMPs? 5 years before the iPod. (Playsforsure)When did Microsoft get into smartphones? 5 years before iPhone. (WM)When did Microsoft get into tablets? 8 years before iPad.
      “I’m not saying MS is copying Apple but to say that Apple is not having a strong influence over MS’s consumer product strategy is ridiculous”

      Apple entered music years after MS. And Jobs in on record saying that their efforts to create a tablet were inspired by what MS was doing circa 2000. Along the way, that effort yielded iPhone because a larger screen wasn’t feasible, and then iPad. So actually according to Apple, you can say that what MS was doing had a very strong influence over Apple’s consumer product strategy, you fired disgruntled ex employee douche.

    • WixosTrix

      “to say that Apple is not having a strong influence over MS’s consumer product strategy is ridiculous.”

      Who said that? The both influence each other.  Apple has just done a better job in consumer products, while Microsoft is king of enterprise with both of them trying to play in each others dominated markets.

    • Anonymous

      Microsoft is historically a software company. For them to have even made the XBox or Kinect is an amazing leap, and to have actually entered that market so strongly is even more amazing.

      The thing that annoys me about MS is it takes them so long to get their research into products (if at all). But at least they fund research; Apple has all the money and give none of it back.

    • trashoner

      from all the trolling you forgot something

      Microsoft did make a touch screen smart phone many years before the iPhone. Just because Apple decided to get rid of the stylus it doesn’t mean that the iPhone was not inspired by Microsoft pocket PC that came out way earlier than 3 years before the iPhone.

      Microsoft did come out with the first tablet ever to be successful unlike the Apple newton that of course you forgot to mention because it was a complete failure.

      I am not saying Apple is copying Microsoft but its ridiculous to think for a second that most of the devices mentioned above, aside from the iPod, were not built on the footsetps of Microsoft earlier work.

      I love apple products but I am not blinded by trend or hipsterism.

      So now apple is trying to come out with a TV, hhmmm … may be they can call it apple imedicacenter.  

    • Andrian Hendrawan

      Leander Kahney of Cult of Mac reports that Apple was the first place that PrimeSense, the company behind Microsoft’s motion-sensing Kinect camera, thought of. 

      “It was the most natural place for the technology,” said PrimeSense CEO Inon Beracha. PrimeSense’s camera and infrared sensor setup was developed by engineers in the Israeli military, who apparently hired Beracha to “shop it around Silicon Valley and find partners to commercialize it,” the report notes.

      Beracha, who called the technology “multitouch that you didn’t even have to touch,” approached Apple, but initial meetings were rocky. According to the report, Apple was obsessed with secrecy, burdening Beracha with “a stack of crippling legal agreements and NDAs.” Eventually, Beracha decided not to deal with Apple, calling the company “a pain in the ass.”

      Beracha eventually went with Microsoft, who wanted to implement the technology for its Xbox 360 gaming console. 

    • Josè Daniel

      I never used the word “copying”. Read it again.

    • PG

      Like apple copying Microsoft and using x86 in their machines, or Apply copying the tablet pc, or copying the numerous Windows 7 features, or copying Xbox Live (gamecenter and achievements), or copying Windows Mobile.

    • Test1ngi23

      The Apple Newton (Apple’s first tablet) came out in 1993. Of course it didn’t sell well, but neither are any current Windows tablets either.

    • Guest

      The Newton was a PDA. To call it a tablet is a joke. For ten years Windows tablets led the industry and sold million per year. Try again, fired douche.

    • Anonymous

      until recently apple couldn’t even resize windows from the left side. enough said.

    • Anonymous

      Yes, MS needs to stop following Apple when it comes to Office, Server, Visual Studio, XBOX, Bing, SkyDrive, Mesh, IE, .NET, Windows 8.  Don’t even bring up WP7 as its a completely different OS and UI from iPhone (unlike Android).

    • Anonymous

      Re: Skydrive, Mesh 
      Apple has tried this twice now, MobileMe and iCloud. Even iCloud still has people complaining about bad syncing etc. etc.

      A lot of big corporate features MS has been coping with for years are trickling down to consumers and Apple is busy buying in expertise, but they lack any experience. Apple can build a $1B data center, sure, but they’re no Azure (you left that one off your list by the way).

    • Josè Daniel

      LOL Dude, I own a Windows Phone and my comment wasn’t meant to be “offensive”. What I meant is that for the first time in quite a while, Microsoft isn’t following a trend (that the average Joe cares about) that Apple has “established” and Microsoft being the underdog, capicce now?

  • Pantou Ekang

    Time is coming !!!???

  • Anonymous

    2 words: holy shit.

    Didn’t know they were advancing so fast with the ongoing development of the Kinect sensor.

    • Sebastian Swann

      here are another two – “Fuck Yeah” 

    • Guest

      They seem to be leveraging MSR quite a bit in this area. But recall that in addition to using PrimeSense’s sensor, MS also bought a PS competitor: Canesta. Hopefully we’ll see some of that for W8 PC’s too. See link:

    • Anonymous

      so not if you’re using Kinect to talk to a hot chick via Skype.  the Kinect can sence your emotions and tell you to go take a cold shower

    • Anonymous

      erection detection??
      oh crap!!

  • Anonymous

    a bit confused, is “Near Mode”a feature that excising Kinects will get in a firmware update or something that the new Kinect Hardware will have?

    I’m buying a Kinect for Christmas and I dont want to be left with outdated hardware a few months later

    • Lewis McCrary

      Doubt it will be out that soon but even still, I’d buy it! :D

    • Кирилл Ярин

      It will be at least a year before Kinect 2, because same source report that Kinect 2 wil be launched simultaneously with next Xbox hardware.

    • Anonymous

      I would go ahead and buy it, especially since its only about $100 or less I believe.  I doubt they will release a new Kinect until they release a new xbox, which shouldn’t be until 2013 (with announcement in 2012).  If you keep waiting you will never enjoy Kinect, as I’m sure by the time Kinect 2 comes out, we will have rumors of Kinect 3.

    • Guest

      Firmware update for existing. Presumably baked into any new model.

    • Guest

      known (according to MS’s statement at least):

      new unit in early 2012
      optimized for Windows
      includes the near mode firmware update


      will firmware update be available to existing Kinect units?
      will the new Windows optimized unit function with an Xbox as well?
      will the existing Xbox-focused unit get an upgrade at that time?

      My guess would be yes, no, and yes.

      With Xmas being the biggest selling season for Xbox and Kinect, MS isn’t going to spell it out any further or risk the Osborne Effect.

  • No

    I wont be buying another Kinnect. They haven’t done a thing with this one, had it for a year and I never use it. Pointless and I feel stupid for being an early adopter. 

    • OMG55

      My Kinect is very useful, I play kinect sports when company comes over and with family, but most of all, I play MW1,2,3, Black-Ops, and many more games on my 360; however, my wife does dance central with my daughter  and we all do the exercise disks that have been release. So in short, it’s a pretty useful device depending on what you’re in to. Apparenty you don’t enjoy exercise heh????

    • Eingoluq

      you are either broke or own a move.

    • Niko

      Kinnect? I don’t think you have ever owned one.

  • Test1ngi23

    Good job Microsoft! I’m about the last person on earth to be called a MS fanboy but I have to admit that Kinect is pretty innovative. The Xbox division has been on a roll the last few years.

    • Guest

      “I’m about the last person on earth to be called a MS fanboy”

      Some people deal better with being fired than others.  

    • Guest

      That’s because they finally got rid of Bach and put someone competent in charge (Mattrick).

  • HAL 9000

    Dave, although you took very careful precautions in the pod to prevent me from hearing you, I could see your lips move.

  • Pantou Ekang

    In this  two year old concept-video (from E3 2009), Micrososf presented a his new technology. Two years later, Microsoft back on this technology with promises of implementations. All this is fine, but many people simply dont believe Microsoft anymore.This technology was supposed to be implemented in Kinect. Microsoft has never admitted that kinect missing voice recognition, lip reading and measurement of emotions.
    All this will be fine after 4 years !!!???Project Natal Video:

    • Devon Garner

      I think that was when they were going to have it at $300-$500. In order to release a product with funcionality and at a decent price they had to cut down on some of the features and include them later. Look at all the things they have let people do with this hardware and just imagine how much is yet to come. So much potential for $150.

    • WixosTrix

      They set their expectations high, so what.  The Kinect as is, is a very capable device and you have to start somewhere.

    • Guest

      Yeah, it’s too bad that project was canceled and that we haven’t seen equivalent technology show up elsewhere yet. While I’m usually critical of MS’s agility, in this case there’s pretty good evidence that current technology just isn’t up to this yet. There’s no way the current Kinect unit could handle this outside of a carefully scripted demo, and it’s state of the art for the industry at this time. People underestimate how long it takes to go from concept to production. Look at multitouch. The foundational work there goes back decades, but it’s only been possible to make that mainstream recently.

  • Guest

    This looks interesting and I’m glad to see them pushing hard here on the innovation side. But over the weekend I saw a voice control for Xbox ad. The first one I’d seen. And it was terrible with a capital T. Really surprised me because normally Xbox is the only decent marketing out of MS.

  • Anonymous

    Kinect 3 will be self aware and the takeover will begin

  • David Abraham

    Excellent…Microsoft’s got the future by the tail on this one