Microsoft is using the multi-touch technology that they implemented into their Surface Table and porting it to normal computers. Check out the video of a demo of a Vista laptop modified to use multi-touch.
Vista Multi-touch Laptop, Based on the Surface Table
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSAlert.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
I’ve seen something like this idea in the Sci-Fi movies for ages …
I’ve seen something like this idea in the Sci-Fi movies for ages …
And now it's no more Sci-Fi
<sarcasm>
Yeah, and if someone comes up with some Warp Engines I won’t be impressed at all, Star Trek has been using those for ages.
</sarcasm>
Now seriously, I fail to see any sign of logic in your statement.
More in topic now, its a pretty cool idea, but I wonder if Joe Average (Who’s had a rough time learning how to use a mouse for the last… 15 years) will feel more comfortable or even more confused.
My bet is that Joe Average will love it.
The real problem will probably be to make all apps (including 3rd party apps) behave in a consistent way in order to avoid confusion.
Today we have:
-single click=selcect
-double click=open/launch
-right click=context menu
Tomorrow we might have things like:
-stretch=zoom in/select larger font/???
-rotate=rotate image/maybe switch apps/undo in word processor
For example context menus should have the same gesture across all applications.
A lot of thought should be given as to whether to use a certain gesture to manipulate the window itself (drag, resize, close) or its content.
Probably all actions over the top bar of the window should be interpreted as affecting the window itself and everything else should be forwarded to the actual application running inside that window.
Another interesting thing would be – I don’t know if it’s possible – to differentiate between different fingers so that for example if you spin an image only using fingers of one hand, it’s spins only by multiples of 90° but if you use both hands, you can spin it more precisely.
All in all I love the demos. Way to go, Microsoft!
My bet is that Joe Average will love it.
Sure they will. The display booth at the tech fair will definitely be filled with Joe Averages oohing and ahing over the technology. But that doesn’t mean it works in real life.
How would this work when browsing the web (think hitting small targets)? What about writing emails? How to accurately bring up a context menu on a specific item? I could go on forever.
Sure you can make up intuitive gestures for some actions (zoom in, rotate, back) but there are hundreds of actions that you need to be able to perform, and most of them won’t be representable by a gesture.
It’s not useless, but I guarantee that it will not be a general purpose interface anytime soon (if ever).
I don’t think it’s meant to be a general purpose interface, and I don’t see why most people think it would be. If this is integrated into a tablet PC it will be in addition to the stylus (and keyboard and trackpad) not instead of.
I disagree.
How would this work when browsing the web (think hitting small targets)?
If your finger hits more than one target, it zooms in and you can pick which one you want.
What about writing emails?
On screen keyboard or something like Dasher.
How to accurately bring up a context menu on a specific item? I could go on forever.
Same at clicking on small targets, zoom then try again.
Sure you can make up intuitive gestures for some actions (zoom in, rotate, back) but there are hundreds of actions that you need to be able to perform, and most of them won’t be representable by a gesture.
If they are learned, they don’t have to be intuitive to everyone, just a learned shorthand for an individual user.
It’s not useless, but I guarantee that it will not be a general purpose interface anytime soon (if ever).
Yopu just need to think of it outside of what can be done with a mouse right now. Don’t think of it as a drop in replacement for a mouse. Adapt the window manager and environment to support the use of your fingers…
If your finger hits more than one target, it zooms in and you can pick which one you want.
So, more complex than using a mouse, both in selection and in programming (have to find intersection on everything).
On screen keyboard or something like Dasher.
On screen keyboards lack tactile feedback, and Dasher is not exactly ideal. Coincidentally, I am running a fairly large user study comparing three on-screen keyboards right now (Dasher being one) and a lot of people are very confused by Dasher’s interface.
Same at clicking on small targets, zoom then try again.
Once again, a simple single step process becomes a multi-step one. How is this improvement?
If they are learned, they don’t have to be intuitive to everyone, just a learned shorthand for an individual user.
Sure but that only works for small numbers of actions. Eventually you run out of gestures and/or memory to remember what you taught the system to do. Don’t even think about using someone else’s computer with a system like that.
Yopu just need to think of it outside of what can be done with a mouse right now. Don’t think of it as a drop in replacement for a mouse. Adapt the window manager and environment to support the use of your fingers…
Fair enough, I think there are definitely software changes that can be made to make interaction easier, but when it comes down to it, you’re replacing a system with 100+ keys and arbitrary 2D input (keyboard and mouse) with a device capable of at most 20 inputs (realistically more like 5) on a 2D plane. For any action requiring more than about 5-10 inputs (typing being the obvious example), a keyboard/mouse will be more efficient.
So having this in addition to a keyboard and mouse would be excellent for some tasks, but you won’t be replacing the standard input devices.
Speaking of sci-fi – you talk like a Vulcan.
“I’ve seen something like this idea in the Sci-Fi movies for ages …”
I just saw something like this over on Apple’s website. It was called the iPhone. Leave it to Microsoft to rip something off and then act like they invented it.
Um, Apple hardly invented multitouch.
You should really read this:
http://www.billbuxton.com/multitouchOverview.html
Hey, thanks for the interesting link! I didn’t mean to imply that Apple invented Multi-touch rather, my point was to simply to convey that Microsoft ripped it off but acts as though it’s new or that they invented it.
“my point was to simply to convey that Microsoft ripped it off but acts as though it’s new or that they invented it.”
Where have they said they invented this type of technology? In the dozens of articles I’ve read about MS and surface computing, not once have I seen them say “multi-touch was invented at MS research”.
“Hey, thanks for the interesting link! I didn’t mean to imply that Apple invented Multi-touch rather, my point was to simply to convey that Microsoft ripped it off but acts as though it’s new or that they invented it.”
You do realize that the article for which you’re thanking for the link was written by a Microsoft guy, right? So, a Microsoft guy is detailing the the history of multitouch, so Microsoft is clearly NOT claiming that they invented the concept. Indeed, the only people I’ve seen claiming that a particular party should get full credit for inventing the concept are Apple fanboys and Jeff Han fanboys.
Edited 2007-06-21 23:51
my point was to simply to convey that Microsoft ripped it off but acts as though it’s new or that they inve
Kind of like what Apple did?
Just like Apple “invented” the MP3 player. Of course they didn’t, but they made is usefull and accessable to the average user. And they marketed the heck out it. Microsoft does the same thing. It doesn’t always matter who invents something. Sometimes it’s who makes it practical and usefull for everyday use.
So that’s why all the computers in Star Trek are able to communicate with each other.
The multi-touch screen found in Surface uses cameras. The one in a laptop would be similar to any touch screen (touch resistance) but with the distinction between multiple fingers.
I think MS needs to spend more time working on the actual human interface… pushing down rotates the object? Maybe when they get the iPhone and study how Apple implements the UI will they get a clue…
8 days and waiting for the launch of the iPhone
IMHO, of course
Jb
Edited 2007-06-21 20:13
I got one better. What if the computer learned the gestures that make sense to the user. When the user does a gesture that the computer doesn’t understand, it asks, “What did you want to do?” Then the user can select a new action or select a previous gesture that the user was trying to do.
Good idea!
This would be especially useful for handicapped people.
I’m thinking of people with less than 10 fingers, perhaps only one hand or guys with Parkinson’s.
However, MS should do extensive studies (if they haven’t already) to come up with sane defaults.
Edited 2007-06-21 20:50 UTC
This would be especially useful for handicapped people.
As someone who develops software and devices for people with disabilities, I am very skeptical of that. The example you mention (Parkinson’s) would be particularly unsuitable. For someone with a condition that impedes precise motor control (Cerebral palsy, muscular distrophy, multiple sclerosis, etc) a system based around recognizing complex (more than on/off) gestures would not work very well. And if you expect to be able to control every function on your computer with a gesture, you need a very wide variety of them. Using gestures for some tasks and then switching back to a keyboard is not an option for most people with these types of disabilities (due to the need to place the input device in the right position, you can’t just switch between devices easily)
Yeah, but in this case I was referring to the parent’s suggestion of the computer adapting to the user rather than the other way around.
I didn’t mean to suggest this new interface was especially suited for people with Parkinson’s.
It is of course not. But your right, the way I put it, it was easy to misunderstand.
“However, MS should do extensive studies (if they haven’t already) to come up with sane defaults.”
Microsoft spends billions annually on UI research as as well as accessibility, so I would expect their studies in the surface computing realm to be nothing short of thorough.
“Microsoft spends billions annually on UI research as as well as accessibility, so I would expect their studies in the surface computing realm to be nothing short of thorough.”
MS spent billions on Vista, and that money was obviously wasted. Why would you expect them to do anything else right?
I have a question for you, Supreme Dragon. I take it, from your English skills and your rabid hate of privacy and property rights, that you are European. So here’s a question for you.
If European governments so obviously want to do away with Microsoft (and other American software companies), why don’t they just release Windows as open source? They already have access to the source codes of Windows and Office, through GSP*. Because they are national governments, if they declared they had sufficient rights to release Microsoft’s source code under the GPL (they can justify it as ’eminent domain’ or a ‘national emergency’ or whatever) or as public domain, there’s no way for Microsoft to prevent this; even if Microsoft does bring this to court, the government would decide whether or not what it did was right. Brazil has already done something like this**.
You’d get what you want: Windows would no longer be secure because its source would be open; the sudden upshoot of hackers and attacks on Windows PCs would make Windows as insecure and unsafe to use as you claim it is; Windows users would have to deal with the sudden proliferation of dozens of Windows distros with inconsistent standards and qualities just like Linux users have to deal with; Microsoft would have to go out of business, at least as a computer company; and only companies that obey to the letter the will of your precious unreproachable standards bodies (that is, the uninnovative companies) would be able to even do business (and they’d probably have to be non-profits or charity cases, like Canonical).
So why hasn’t any EU country pulled this ‘nuclear option’ yet?
* http://www.microsoft.com/gsp/
** http://news.bbc.co.uk/1/hi/business/1505163.stm (That was a life-and-death situation, although the way most Europeans talk, it seems to me Europeans feel that that imposing copyleft onto software and IP is more important than the lives of the creators or the consumers.)
Edited 2007-06-22 04:42
“I take it, from your English skills and your rabid hate of privacy and property rights, that you are European.”
“That was a life-and-death situation, although the way most Europeans talk, it seems to me Europeans feel that that imposing copyleft onto software and IP is more important than the lives of the creators or the consumers.)”
Congratulations! You just sank way way down to the same lame level where the rabid anti-U.S crowd dwells. Well done.
“I take it, from your English skills and your rabid hate of privacy and property rights, that you are European.”
I don’t hate privacy and property rights, and I am not a European. I want people to have a choice, people should not be forced to buy Windows because a Monopolist locks them in with closed standards and OEM’s have been coerced to sell almost all computers with Windows.
“If European governments so obviously want to do away with Microsoft (and other American software companies), why don’t they just release Windows as open source?
You’d get what you want: Windows would no longer be secure because its source would be open; the sudden upshoot of hackers and attacks on Windows PCs would make Windows as insecure and unsafe to use as you claim it is”
Open source bad code would be little better than closed source bad code, people need open standards so they can choose what OS they want to use. There needs to be more no OS and Linux computers offered, people should not be forced to pay MS every time they buy a computer. If MS was not a monopoly they would probably go out of business because of garbage like Vista. Give people a choice!
“MS spent billions on Vista, and that money was obviously wasted. Why would you expect them to do anything else right?”
From a financial perspective, Vista has been a runaway success for Microsoft. It has also been well accepted by the general public…only about 1 in 10 of Dell’s new machines still ship with XP (and customers do have a choice there).
Even moreso (since we’re talking about UI’s here) the UI changes in both Vista and Office 2007 have been extremely well received, so I’m trying to figure out exactly what you are talking about here. I believe there is one thing almost all computer users can agree on (both pro and anti MS): The Windows/Office GUIs blow the competition away in terms of ease of use and consistency, and this is due to the sheer amount of UI usability testing that MS performs on all software they release.
“From a financial perspective, Vista has been a runaway success for Microsoft. It has also been well accepted by the general public…only about 1 in 10 of Dell’s new machines still ship with XP (and customers do have a choice there).”
Vista is pathetic, MS could have saved billions by just offering another service pack for XP.
“The Windows/Office GUIs blow the competition away in terms of ease of use and consistency, and this is due to the sheer amount of UI usability testing that MS performs on all software they release.”
KDE/Gnome offer an excellent UI, that does not need excessive system requirements and is free. OpenOffice is excellent free software, with a good UI. MS software is just not very impressive, certainly not worth what MS is charging for it.
Doesn’t anyone else find the demo extremely funny? Given that we have those *awesome* iPhone demos.
For example, the system doesn’t react immediately to his gestures, zooming looks clumsy and that box of wires on the back or whatever that seems to come from a DOS era is the punchline. Yeah! We can slap this on top of that (that’s how Windows was done) and it sorta works and look – I can use two fingers! Whatever!
And on top of that they had to find the geekiest geek with a bad geeky voice instead of somebody more attractive…
It looks like a sad sad joke, innovation aside, that “product” doesn’t have *passion* in it!
Edited 2007-06-21 20:24
Now, come on! It’s a freaking prototype!
Don’t be so fast to knock it down.
The sensors had a very limited area and I’m sure things will be far more optimized and a lot faster when this stuff hits the shelves.
That’s not to say that it doesn’t need a lot of resources, which it probably does.
But my gut feeling tells me that it should be pretty easy to use several cores for this (or even dedicated hardware, who knows) so Moore’s law’s second incarnation should come to the rescue.
If you just want shiny demos where everything works, you’ve gotta go to his Steveness
This puts it in perspective http://uk.youtube.com/watch?v=CZrr7AZ9nCY
I found this far more interesting than the iPhone commercials. This actually gave me an idea of how the tech works and what it might be like in the real world. The iPhone stuff is just slick marketing and to be honest I still have no idea how the iPhone will really in real life. I only know how the marketers want me to think it works.
QUOTING dagw:
“I found this far more interesting than the iPhone commercials. This actually gave me an idea of how the tech works and what it might be like in the real world. The iPhone stuff is just slick marketing and to be honest I still have no idea how the iPhone will really in real life. I only know how the marketers want me to think it works.”
I don’t quite understand what you mean. Apple has released a 25-minute (actually 24:40) QuickTime movie, “iPhone. A Guided Tour.” Maybe this is what you were missing?
The video includes:
o Multi-touch user interface
o Design
o Ports and buttons
o Mobile Phone
– Calling
– favorites
– Recents
– Contacts
– Keypad
– Visual Voicemail
– Conference Calling
o iPod
– Music
– Cover Flow
– Video
– On-screen Controls
– Customizable soft buttons
– Included Headphone/Mic use
o Photos
– Integrated camera
– Viewing
– Scrolling
– Zooming
– Wallpaper
– Emailing photos
o Internet
– Safari full-featured web browser
– Keyboard
o Email
– Using the intelligent soft keyboard
– Rich html
– Viewers for PDF, JPEG, Word, Excel
o Application Integration
o SMS Texting
– Integrated with phone
– History
– Sessions
o Live Stocks
o Notes
o Clock
o Live Weather
o Google Maps
– Satellite view
– Zooming
– Scrolling
– Bookmarking
– Traffic info
– Driving Directions
o YouTube Wireless Streaming
o Settings
– Airplane Mode
– Ringtones
– Wallpaper
– Mail Settings
http://www.apple.com/iphone/usingiphone/guidedtour.html
This should be more than enough for you to be able to compare both implementations of the multi touch technology.
Edited 2007-06-23 12:06
Multitouch interfaces are like the Microsoft Bob of 2007. Sure in some limited tech demo it looks neat (ooh I can resize things with two fingers) but for general use it won’t be very useful. I don’t know about you, but I don’t tend to throw my documents around in piles and resize images all day. For real computer use, a keyboard and mouse is still far superior in the vast majority of use cases.
Even for very specialized uses (like their music mixing device) the interface is not ideal because there is no tactile feedback. And people doing photo editing need far more precise control than a multitouch interface gives them. I’m sure there are some people and applications that it’ll be great for, but I don’t believe for a moment that it will be a good interface for the masses.
“
”
Whats exciting is not what it can do now, but what it will do in the future. Think laptops that have dual touch screens (using the lower screen as a keyboard). The ability to dynamically redesign the input device on the fly depending on the user and the specific applications will become far more pracical for every day use for the average user than a clumsy touch pad mouse and keyboard.
The only drawbacks i can see thus far are:
* running *nix without X11 installed
* battery consumption (though hopefully by the time this technology becomes mainstream i batteries would become better or a new means of mobile power would become practical)
* running *nix without X11 installed
Sorry, not sure what you’re talking about with regards to that, especially since Peter H. is currently doing amazing work on getting multiple pointers working on X11 at once (even without confusing applications that only know about pointer!)
Combined with hotplugging pointers (stuff that Daniel Stone has been working on), X11 is providing all the needed infrastructure for applications like this.
All you need are drivers that add and remove pointers as you touch the screen, applications that know about multiple pointers, and you’ve got multitouch.
The software part I guess. I read somewhere that the Surface Table uses several cameras inside to implement multi-touch, not a touch-sensitive surface.
edit: found it. From Microsoft’s PR blurb:
Edited 2007-06-21 20:30
http://www.youtube.com/watch?v=ulwA3n8AYM0&eurl=
it reminds me of some videos I saw last year!!
posted August 2006:
http://www.youtube.com/watch?v=PLhMVNdplJc
http://www.youtube.com/watch?v=Yx9FgLr9oTk
http://www.youtube.com/watch?v=dQkSObRtw0o
and there was 1 by SGI, but I can’t find the url.
But for me, I work in Education, can you imagine using this type of computing with kids? And sticky fingers, or worst yet, in the early years with 3 and 4 year olds, I can see a finger going through the LCD.
It seems to me the technology is slower, or that version of it because while doing it, you’re also wary of damaging the LCD.
This version of the technology, which more people will appreciate I think.
http://www.popularmechanics.com/technology/industry/4217348.html
It’s not about who invented what first. It largely doesn’t matter except for the patent office, historians, and ubergeeks.
It’s all about who made it commercially feasible, as in accessible to regular people and capable of being mass-produced. Price will always fall when demands drive increased production capacity.
That’s it, period.
That Pong arcade you have fond memories of? Took off because the Atari crew under Bushnell’s command made it easy to mass-produce. And it was *easy* to operate, unlike massively button-driven interfaces that predated the joystick and the dial.
The 2600? Same thing, except that you could actually buy yourself one and lug it around to your friend’s place.
The Wii? *Same thing*, Nintendo hardly invented infrared RCs and accelerometers and I guess nobody claimed they did! But the Wii will be remembered forever because of how the whole package was put together.
Apple *hardly* invented the dial or the touchscreen, but they wrapped it into the click wheel interface. *This* was innovative.
So multi-touch was invented in 1982. Big deal. Until now, it’s been nothing but some novelty poised to become the next big thing on some futuristic device somewhere in the future.
Thankfully, Apple broke the wishful thinking inertia, put together pieces of technology that are feasible *today*, and set off to mass-produce a device that takes advantage of it.
While Microsoft once again lost itself in the “futuristic devices” loop and delivered… Something that fits their “house of the future” vision. In other words, extremely sophisticated technology that’s great for showcasing but that no regular person will remember 10 years from now, because no mere mortal ever got to buy one. More likely than not some ideas of Surface will make a comeback in a not-so-distant future, but Surface itself? That 10k+ coffee table? Sporting a whole computer, a big screen and 5 cameras? How hot does it get inside? How heavy is that? How much power does it draw?
At that cost, there’s *no way* Surface will be mass-produced, and, chicken-and-egg-ly, there’s no way its costs will be driven down. On the other hand, I can easily see iPhone costs going down, cheaper versions being produced, next-gen for the price of last-gen, that is, the iPod all over again.
That “cheap” sells, that’s a given. But there’s a whole world beyond being simply cheap. Shiny sells. Colourful sells. But portable sells more. Usable sells more. Affordable sells more.
When it’s not simply about the pricing, it’s the packaging. Always the packaging. Always.
You know that “aggregate value” talk? That’s it.
Edit: better wording
Edited 2007-06-21 22:24
“What” is the packaging? What is “it”? It’s one thing to talk about giving credit where credit is due; quite another to patent something as new just because you have in reality refined it. I don’t care if we’re talking about Apple or Microsoft, both of them patent obvious refinements of existing technology and it always decreases the potential quality of competitors’ products by preventing them from making certain design decisions.
Yeah right, so different from millions other companies around the world.
It’s eat or be eaten, pal. They patent not because they have a sadistic gusto for twisting Joe Smalldev’s arm, but because it’s a necessary business practice of covering their own rears.
If they don’t patent it, someone else will, and they’ll have to license something obvious from someone else. When you’re a large software/hardware vendor, this is usually not an option. Unless you’re more than satisfied to have your code increasingly more encumbered with licensed IP.
Boo-hoo, Microsoft is evil, Apple is evil, Creative is evil, Sony is evil, Philips is evil, heck, EVERYONE who ever stepped inside the *PTO is evil!
Yeah, right. Don’t blame the USPTO (the one most responsible for the current situation and for propagating these practices to almost all WTO members), don’t blame the fact that they grant patents on stupidity, don’t blame the fact that THEY turned this into a business practice necessary for staying alive and relevant. Don’t blame the fact that they enabled the existence of companies whose sole purpose is IP trolling.
I don’t condone patents, on software or otherwise, but I’m not na"ive to pretend I live in a world of idealised manicheism, us versus them, black versus white. I can’t really subscribe to the idea that companies who spend millions in R&D and produce actual innovations based on their intellectual property is evil.
Pure-breed patent trolls, however, are.
Edit: “code encumbered with increasingly more licensed IP” -> “code increasingly more encumbered with licensed IP”
Edited 2007-06-22 20:49
Response time seems to be slower than on iPhone.
Vista already includes the Tablet APIs as an optional component.
I wonder if the Surface APIs (or at least, the Multitouch APIs) will be part of Service Pack 1?
– if this vid was recorded in the nineties
– if microsoft had advanced their speech recognition technology (which came standard with os/2 warp in 1995) to recognize multiple voices at once
– if windows os was open source so we can all learn how to not to design your own software in that bad way
– if democracy really would work in this world
– if people really can make peace
but hey microsoft is a stubborn organisation, not spending too much on comprehending already existing and documented technology, but just invent it a new for themselves.
and sometimes i think thats a good thing. so let them be happy with their discoveries in their little world…
ps. btw i think this infrared thing remote controll is really great for games on xbox, just a hint.
Edited 2007-06-22 05:11
Is this video really new? If it is then this guy must have been locked into his office in a bunker with no contact with the outside world for quite some time.
A guy who has seen the Jeff Han, iPhone and Surface demos wouldn’t take this approach to a demo would he?
I wonder though, how thin can MS infrared multitouch actually be? It seems that since it’s based on projecting and scanning light it has to be rather thick to work. I could be wrong though.
The technologies that Apple and Jeff Han has demoed are very thin, fast and accurate. Excellent for mobile devices. If all these are in the same price range, then MS will have a hard time competing with them.
I have a gut feeling that Apple will be the first one to make an ultra portable with multi-touch available to the public. And it might even be this year. (not talking about the iPhone)
Leopard is clearly indicating this in the UI department.
Multi-touch will be available to the public on the iPhone next week!
why this thing remember me so much XGL or beryl + linux??
“ctrl+c”???
…….
http://www.youtube.com/watch?v=Yx9FgLr9oTk
I really like it and wish I had something like that on my linux box but… you surely noticed the “multi” thing on the Multi-Touch headline, didn’t you ?
yeah, its what i call: “ripped + tweaked”
its easy to rip a code and improve it. Or even rip the idea and improve it.
pfff….