Microsoft patents reveal touch+pen plans, Windows 8 tablet future?

By Tom Warren, on 29th Jul 11 11:04 pm with 69 Comments

Several new patent filings have revealed what appears to be Microsoft’s pen and touch plans, hinting at future Windows 8 use.

Microsoft filed for a variety of picture related gestures in January, 2010. The software giant has just published the patents to these gestures in the past few days. The gestures involve multiple implementations and allow for tablet users to make use of touch and pen based inputs. Microsoft has also supplied a number of images to accompany the gestures.

Copy Gesture

The most basic of Microsoft’s new gestures will allow for images and data to be copied by simply tapping the object to select it and then dragging it with a stylus or pen to the left of the object. The opacity of the object increases the further away it is moved from the source object. Microsoft states “although a specific implementation has been described using touch and stylus inputs, it should be readily apparent that a variety of other implementations are also contemplated,” suggesting that the copy gesture will be available without a pen/stylus input.

Copy Gesture

Staple Gesture

Microsoft’s staple gesture appears to allow tablet users to group objects together into stacks. Objects are selected by using touch and stylus inputs. An initial input is recognised and further taps on objects will allow objects to collate underneath each other into blocks. “The staple gesture may also be repeated to add additional objects to the collated display of a stapled group of objects, further collate groups of already collated objects, and so on,” says Microsoft’s patent filing.

Staple Gesture

Cut Gesture

Cut allows for images or objects to be split apart. The gesture works by a combination of touch and stylus or touch alone. Objects can be separated by cutting across them using the gesture.

Cut Gesture

Punch-Out Gesture

Punch-out does what it says on the tin. The gesture allows users to manipulate an object and punch a section out of it. “It should be noted that although a specific example was described in which the punch-out gesture was input using touch and stylus inputs, those inputs may be switched, a single input type (e.g., touch or stylus) may be used to provide the inputs, and so on,” notes Microsoft.

Punch-out Gesture

Rip Gesture

Rip allows for objects to be separated into two using finger touch input. The gesture simply separates the object by a nature pulling apart of both fingers.

Rip Gesture

Edge Gesture

The edge gesture can be used to create straight lines from objects. The gesture can also be used to cut through other objects in a straight edge manner. The gesture appears to work like a rule, allowing pictures to be used for edging. This gesture uses a variety of stylus of touch inputs.

Edge Gesture

Stamp Gesture

The stamp gesture is perhaps the most impressive of all Microsoft’s gestures. The tool allows users to create rubber stamps of existing objects and reuse them as a copied item.

Stamp Gesture

Brush Gesture

The brush gesture allows users to quickly edit objects. Brush appears to allow users to quick remove a particular part of a picture or object and have it stored elsewhere on the screen. This is particularly useful for those who work with a number of images.

Brush Gesture

There’s a number of other gesture including carbon-copy, fill, cross-reference and link. The majority of the gestures appear to be photo related and have been previously demonstrated on the company’s Surface product (see video below). The patents also appear to be similar to Microsoft’s original pen and touch input plans for its axed Courier project. Microsoft has completed a number of research projects around pen and touch computing. The company clearly believes that the pen or stylus is still an important input object alongside touch and the software maker appears to be working to unite them in new ways.

Microsoft canceled their internal Courier project shortly after the release of the iPad. The project was an innovative new tablet concept that had promised to combine a dual screen book design with finger and stylus input. The leaked promotional materials demonstrated various concepts that would have brought an unparalleled productivity tool to the market. Microsoft is currently reading beta copies of Windows 8 in preparation for its BUILD developer conference in September. The company revealed its new tablet interface in June. The interface puts touch at the forefront using the company’s familiar Metro design of live tiles. Microsoft is expected to detail Windows 8 in full at BUILD and will likely release the first beta bits of the new operating system. A final version of Windows 8 is expected in mid-2012.

Image Credit: Microsoft Corporation/US Patent & Trademark Office

  • http://www.facebook.com/people/Matthew-Williams/713122739 Matthew Williams

    This is pretty cool.

  • Grannyville7989

    All of the examples show use of a stylus and that’s what I’m looking for in a tablet that I want to use in university.

    • Frylockns86

      Agreed. Stylus or GTFO.

      Apple fans can have fun fingering their iToys ;)

    • Joe05

      There is already stylus input option for iPad, please do not let your fanboy mentality make the rest of us MS fans look ignorant.  

    • Anonymous

      its not really a very good option or supported deeply by iOS… so its nowhere near similar to this

    • Guest

      Understand there is NO this, it is a concept at this point so any offering for the iPad is significantly better then any concept.  Moreover have you used AluPen Stylus for iPad along with a drawing app would will find a VERY GOOD option.  

    • http://twitter.com/TheRealestJG James Goldsmith

      Since the ipad is all capacitive, it only works with a capacitive stylus which is only a substitute for your finger. No real pen support.

    • Tono

      There are tablets with Windows 7 that support pen MUCH BETTER than an iPad. So if you want to drawing better go with one of thouse and forget the iPad.

      And this is not a concept. Is real. Go buy one.

    • Guest

      Yes that is why they are selling 10s of 10′s them while ipad are selling 10s of millions.  

    • Guest

      He said it supported pen input better, not that it sold more. Learn how to read.

    • Guest

      He said it supported pen input better, not that it sold more. Learn how to read.

    • It’s me

      The iTards would buy anything from Apple. Jobs could crap in a brown bag and those people would buy it.

    • Jerar

      The sales doesn´t mean that a tablet with Windows 7 is better for pen support , drawing art or ink than an iPad.

      I have a Workstation with Xeon processors and Nvidia Quadro FX cards here on my room. It´s the better machine to work in 3D. This machine doesn´t sales a lot compare to the market of netbooks or iPads, but it´s the best machine for what I do.

      Understand??

    • Guest

      But sales are a reflection of functionality and more functionality the more useful it becomes to people.  You may be one is million with special need requiring a particular product that would be so limited in the market that would not be viable to sale at any price.   You even admit any tablet would not meet your needs.  Thank you for proving my point.  

    • Arsk

      No. I only say that for pen input any tablet with window 7 is far better than the ipad.

      People who are buying iPads are for fun and for play with it.

      If you want to do creative art, desing, ink, etc…. go for a WIndows tablet.

      Windows 8 will just bring to the table also a funny and more easy interface, so you can do the same you are supose to do with an iPad, BUT also all the work a Windows Tablet bring to the table (like drawing, ink, etc,…)

    • D Airey

      I am professional graphic artist for over 15 years and find that I am doing about 70% of my work on the iPad.   Plus I do a great deal of my billing and correspondence with device.  I guess I am saying that while I find my iPad fun, it spends most of it time with me working.  

    • http://www.facebook.com/people/Adrian-Werner/100000621679538 Adrian Werner

      Why then? Seriously? For graphics artist something like thinkpad tablet is infitely superior to iPad. It’s not a matter of opinion, it’s a simple fact. So why do you use iPad? You didn’t do research before and didn’t realize there are much better alternatives out there?

    • D Airey

      I purchased it as the earlier post stated to use mostly for fun but quickly I found it work great for a good deal of my work!  Your right it is not a tool perfect all of the time but it has been a perfect tool many of the times. 

    • Guest

      Yes I too was surprised that I was finding the iPad so handy graphic work.  

    • GP007

      It’s not a concept, they’ve shown it working on the surface hardware, it’s real, and it works.

    • Guest

      Understand there is NO this, it is a concept at this point so any offering for the iPad is significantly better then any concept.  Moreover have you used AluPen Stylus for iPad along with a drawing app would will find a VERY GOOD option.  

    • http://www.about.me/AngeloGopaul Eingoluq

      No….. not really, there is not. A pen for the ipad/iphone is a substitution for a finger. There is no differentiation between a pen and your finger. The screen translates the input in the same way. Because of this, palm rejection is impossible without certain work-arounds like a dead zone on the screen. And accuracy needs work as well.

    • Guest

      HTC Flyer

    • Anonymous

      DOA

    • Anonymous

      DOA

  • http://www.facebook.com/profile.php?id=100000678662470 Vincent Haakmat

    It’s interesting that Apple hasn’ thought of these gestures yet, since they had a head start.

    • GP007

      Apple doesn’t like the stylus I guess.

    • zzz

      no they found that generally stylus are as popular as Palm Pilots these days. 

    • Frylockns86

      But what if I want to write? Surely, I’m not using my finger…

    • Guest

      Buy a iPad stylus, there some really good one out there.  

    • Anonymous

      they actual patented some stylus designs just a while back..

    • http://www.about.me/AngeloGopaul Eingoluq

      Microsoft started research on mutitouch before Apple I believe. They wanted to create a digital architectural drawing table(Microsoft Surface) Apple wanted to create a phone. The pen is very important to architects, so I would like to think they thought of it from the get go. as their scope was larger than what Apple was trying to achieve.

    • Anonymous

      Yeah 100 Surface tables will be sold to architects some day, while Apple has already sold 100 billion iPads!

    • Dhaoracle

      What??? You can’t compare the two totally different experiences and lot more power. It is like comparing a Muscle Car to a Bug.. lol

      100 Billion was funny they won’t even make it to 1 billion once Windows 8 Tablets comes out…

    • Guest

      Check your facts, dumbass.

    • Joe05

      While Microsoft was “wanting” Apple “did”.   Ideas are easy implementation is hard, and implementation is what Apple excels at.  

    • Guest

      Actually they both “did”. The difference is that the market Apple targeted is much much larger.

    • http://twitter.com/oolong2 oolong2

      Actually Apple “bought”, the company was Fingerworks.

      Apple is unique because they’re a hardware and software company so it’s very easy for them to come out with a product quickly and have everything is nice and integrated.  However to do this by sacrificing interoprability and customization….  Everything has to be Apple’s way.

      Microsoft does more research than Apple and Google combined, but they rely mainly on software and partnering for devices..  So they tend to be a lot slower in coming out with new “things” because those “things” have to be interoperable with a lot of other software, hardware, devices, etc.  So when they finally come out with a solution its pretty much all inclusive, flexable, and customizable.

      This has been the back and forth between Microsoft and Apple for 30 years…

    • Anonymous

      That’s why I’m so sick of the comparisons between Apple & Microsoft. Apple is mainly a hardware company that also makes some software. Microsoft is mainly a software company that makes some hardware(X-Box, keyboards, mice, Kinect).

    • B7674567

      What did Apple have a head start in?

      Multitouch gestures have been around for decades, Apple didn’t invent them.

  • Hernan Lozano

    Whelp.
    I’m reasonably convinced a courier app’s gonna come to W8 tablets.

    thank you based god.

    • http://www.facebook.com/profile.php?id=620205438 Aaron Delgadillo

      Don’t get my hopes up! LOL

    • Berry

      I agree and I am super excited. Steve Ballmer mentioned somethingnto the effect ot the Courier being dead as a device but we could see the Courier technology somewhere else. A Courier app for Windows 8 would be awesome and would have me in line, at the Microsoft store on day 1 of the release of Windows 8. I am using an iPad to write this message and I am desparately waiting on the day a tablet exists that it not just an app launcher and email reader/writer. I would love to be able to have “real” pen input and use “real” productivity tools.

    • Anonymous

      I do remember Balmer making the same comments, he hinted that Courier related technologies would appear in the future somewhere else.

      Thats the same problem I have with my iPad , I hate going in and out of apps, I want a real file system so a can actually put my photos where I want and I also would like to actually run multiple programs.

    • Coffii

      I remember MS talking about this 12 years ago :(

  • Anonymous

    This is some good stuff. However, I really don’t think companies should be able to patent human gestures. All of these gestures are things we did with physical objects. Why should we grant patents just because it’s on a digital medium. Apple has also been trying to patent all of these different gestures. I don’t think they are enforceable. Apple has the patent for pinch to zoom and they haven’t sued anyone even though all these companies are using it.

    • Joe

      That’s because all of these companies pay apple licensing fees in order to use things like pinch to zoom

    • Dhaoracle

      Why they are not the first company to ever use pinch to zoom or even invent it. It was invented by a group at the Uniersity of Toronto and Fingerworks which Apple quickly acquired to be their employee.

  • SDreamer

    I really hope MS follows up with this. See, touch is an extrodinary tool for navigation, and the stylus is such an essential tool for manipulating items, that they would compliment each other so well. The best application right now I think is OneNote, where you can use touch to effortlessly scroll through your notes and navigate between notebooks and then jot notes and diagrams down with the stylus, really awesome for the college student. Art programs that support stylus and touch are pretty amazing too. In some such as ArtRage, you can pinch to zoom in, then finish up the smaller details, then freely rotate to adjust to make it easier, then zoom back out and pan around using touch, all while the stylus compliments what it does and that is manipulate what you’re doing. The stylus imo can never, and should never be killed off, and should be valued. I really hope MS keeps using the stylus, it is a major differentiating factor compared to what everyone is trying to do with Apple.

    • Anonymous

      This is what I’ve wanted , I love my iPad but it doesn’t go far enough for what I need.

      I can’t wait to find out more about Windows 8.

    • DerBorgFlüsterer

      The only thing we than need would be a preasure-sensitv stylus. Because all the drawing and even the writing with a stylus become really a natural feeling by more preasure-levels. This would be a great USP espeacilly in the everyday business/university/school world. :-)

    • Anonymous

      Agreed, I would also love a non reflective or non glare screen on my next tablet.

    • AjlueK

      You will get what you want and then some with iPad 3 and iOS 6.  

  • http://twitter.com/oolong2 oolong2

    This is old news….  But yeah stylis input was curiously left out of the Windows 8 announcements.

    Yet we KNOW they have been working on enhanced stylis and gesture input for a long while now with Bill Buxton and Project Gustav, etc.

    I’m hoping that the killer feature for Windows 8 will be MS Courier functionality…  It makes sense that they put this project on hold because given how technology is advancing it was better to include Courier functionaltuy in Windows 8 rather than create a separate product.

    The two problems in Apple’s iPad strategy in the long run is:

    1.  iOS and Mac OSX are two separate things.  ARM processors are powerful enough to run full OS features along side basic touch apps.  Having the best of both worlds in a portable format.

    2. Neglecting the impact that stylus input will have for students, teachers, designers, sciences, mathemeticians, business people, speakers, etc.    There is a broad audience they are neglecting by only focusing on touch.

    If Microsoft can capitalize succesfully on those two things…  Windows 8 will be huge.

    All they need to do is deliver on what Courier promised…

    • Dhaoracle

      You said most of it. This is what is going to bring them ahead of the game with the new UI, Functionaolity, Full OS, running on ARM, and now this. Windows 8 is going to have too much funtionality with smooth graphics and strong userbase that wwill definitely step up if Microsoft does this right.

    • Dhaoracle

      If they introduce their STITCHING feauture in a more productive way besides just touch into Windows 8 I would just be tooo excited to buy a Windows 8 Tablet. Knowing that I could share my photos and desktop instantly with any Windows Tablet or computer is a winning featureany day over apps.

    • http://twitter.com/oolong2 oolong2

      I think touch would be a better use of this feature than a stylis, but the biggest problem would be security. NFC is a great option for sharing data in close proximity and Nokia has done a lot of work with NFC…

      Of course there are a few apps that allow you to share stuff on mobile devices by proximity, but having something baked into the operating system would be awesome….  You could not only share stuff on your phone/tablet, but also data from the cloud (skydrive, social networks, etc.) all seamlessly.

      Tablet is an ideal form factor for doing enhanced sharing.

    • Guest

      And with the Disney marketing guy… :)

    • Guest

       ”iOS and Mac OSX are two separate things”  That is a positive for Apple, the PC environment is the not same as mobile. However, there is a technically fallacy to this vision as well. Even if the platform across devices would be same, application development can’t be unified. Code once, run everywhere does not work in practice. Each device has its strengths and weaknesses. An app which consumes too much data is okay on my work PC connected to DSL, but would not work for my phone with a metered data plan. Network traffic, screen estate, battery constraints, and other factors define app preference on diverse devices apart from the usage scenarios. Consider this: the iTunes app store serves both iPad and iPhone, but several publishers offer different editions of their apps for the two. Ask a Silverlight developer if developing for Windows Phone 7 devices and a Web experience is similar. Even with the HTML5 enthusiasm around, a Web service would prefer to develop a mobile Web site separately.

    • http://twitter.com/oolong2 oolong2

      Having iOS and Max OSX as two separate things was a positive 3 years ago…  However the landscape is changing.

      NVidia, Qualcomm, etc. are coming out with mobile processors that are far more powerful than the laptop I’m typing this on right now.   With the ability to switch on/off cores in order to save on battery life.

      Microsoft has basically shown that different UIs can live on the same device… The desktop UI can live along side a touch UI or just hibernate in the background while the touch UI is in use.  

      As far as development is concerned.    Understand there are different layers to development…

      Adobe PhotoShop would be useless on an iPad.  However on a Windows 8 device Adobe can easily have everything they need for the desktop version of PhotoShop and simply have different UIs (presentation layers) that focus only on touch or stylus input. 

      Here’s the scenario:  You’re out and about and decide to take some videos/pictures using the camera.  You do some basic browsing, triaging of images, and simple edits using the photoshop touch interface.  When you get to a table/desk you can do some more edits using the stylis (cropping, etc).  Then when you get home you dock your mobile device and continue to refine the images using your big monitor, and mouse and keyboard.   All this can be done using the SAME photoshop project instead of trying to transfer images back and forth between different devices.

      The same scenario can be applied to a lot of different programs, such as  business documents, games , engineering, 3D modeling, and would be the essential difference between a toy and a tool.

      Microsoft is headed in that direction.  While Apple is frantically trying to put iOS features into OSX to capitalize on it’s popularity.

    • http://twitter.com/laserfloyd Lewis McCrary

      That’s how I see it.  One OS that can live on multiple devices.  One OS that can support touch/pen/mouse/keyboard.  One OS than can do whatever I need it to do.  That’s what I want.

      oh and…

      In Russia, Windows runs YOU! (sorry I couldn’t resist) :)

    • MSfan

      Replacing PC is what the iOS is doing when it come to consumer who has understand that 80% of what then do in a day can be done on the iPad/iPhone.   They are not upgrading to new PC or even laptops for there is no need for fast more power PC to do email, stream, surf and gaming.  

      Just as people were will to give up top performance of a desktop for the mobility of a laptop we are now seeing them willing to give up the performance of laptop, (which was once considered to be a luxury item) for mobility, batter life and convince of tablets.    

      I do agree that stylus input market is out there.  

  • Patrykvdmeulen

    This refers back to nothing!
    This has been years!
    And Microsoft is a new word, yes really nice you know! not!

  • Anonymous

    The thing is there seem to be so many idea around windows 8 that I fear MS will not be able to bring all these together and make it work as they intend to..  At least they should have a courier like premium app which comes with large sized table

  • Anonymous

    The nice thing about stylus based gestures is you don’t gunk up your screen. Of course this is all single-touch, you still have to use your grubby fingers for multi-touch gestures.

  • trashoner

    Just to be unbiased here. I don’t see how these two inputs will be integrated into tablets. The applications of these patents are obviously meant to work with surface tables instead of tablets. I don’t see how this compairs with the iOS experience at all. The iPad is an app consumer centric device that is just meant to buy and watch stuff online much like a very very nice kindle or nook without business in mind. Yes people get stuff done on them but very little comparing it with a notebook.  This video shows how the two imputs can be use to actually make something more practical than what the ipad can do but in now way comparing it or with the intention on putting it in a tablet…a flamewar here does not makes any sense at all.
     
    I don’t really see how the stylus here does anything more than improving hand writing since most of the other stuff shown in the video can be done without much challenge while using two bare fingers.  Trying to put this on a windows tablet is silly since we  already know that a writing stylus is not enough in the eyes of the consumer to make an impact in the market. I believe that this app is definetely awesome but not because of the stylus. 
     
    This video has been in the net for a couple of years and I think the intention of the pateting here is just with the end of charging others for implementing them in non-microsoft devices such as iOS or Android. Even then this whole thing is silly since you obviously need a very large surface in order to pull off all the tricks this dude is showing.  I only see this working well on a table or touch screen PC with larger than 20″ screens but denfinetely not on any tablet little screen. May be this is why the courier is gone…and gone for good.

    • http://twitter.com/oolong2 oolong2

      Two things:

      1.  While I agree that when “on the go” many of these gestures would be useless, at the same time you can easily lay the device on the table.   As long as the device had rubber feet it would keep it from sliding around.  Then you have a features which would be invaluable to designers, engineers, scientist, mathematicians, people giving presentations, etc.

      2.  Keep in mind that Windows 8 is an installable OS…  We’re not just talking about “tablets”…  Everyone just focuses on that because of the iPad.  Undoubtedly you will see Windows 8 installed on tables, walls, and many other form factors.    ARM opens up possibilities we can’t even imagine today.

  • Guest

    You won’t NEED a stylus though. It’s the same old Redmond. Give you features first, but this time neither OSs will have poor performance (Android does though). It gives you choice (features you won’t get on a iPad/HTC Flyer stylus), but with Mac quality usability (iPhone is worse than Mac).
    -File system access
    -Not locked down
    -Ahead-of-time compilation for .Net eliminates any perceived performances issues
    -App store means recompiling stuff etc. so apps will come quickly
    -More value for the non-wealthy people, laptop & tablet in one, adaptable, no syncing needed so cloud computing takes a back seat and the Continuous Client reigns supreme
    -Android *tablets* are metaphorically dead
    -Chrome OS is useless no matter what (use Haiku instead if you are going to keep you’re “I must not buy Apple or M$!” opinion)

  • http://twitter.com/SimonJStuart Simon J Stuart

    Many of the gestures mentioned in this post are already in use in various graphics applications for touch/tablet users. I don’t see how Microsoft could legitimately patent these without infringing on “prior art”… but hey, that’s never stopped the US PTO from granting patents to massive corporations before!
    Both Apple and Microsoft (as well as “Patent Cartels”) hold plenty of patents on software concepts they never invented to begin with, then sue the original author of said concepts for patent violation.

    If you ask me, software patents are the height of IP evil!