No keyboard, no touchscreen either

A new device lets you control your computer with hand motions. But its software is inconsistent and frustrating, says  David Pogu.

By now, we all know what the future will be like; movies and TV shows have described it in detail. We know about the flying cars (thank you, “Blade Runner”), holograms (thank you, “Star Trek”) and robot butlers (thank you, “Jetsons”).

So when will we really get those technologies? Probably on the 11th of “Don’t hold your breath.”

 There is, however, one exception. As of this week, you can buy your own little piece of “Minority Report” and “Iron Man”: controlling your computer by making hand motions in the air.

The Internet has been buzzing about the much-delayed Leap Motion Controller ($80 or Rs 4,726) since its first public demonstrations over a year ago. 

Imagine controlling on-screen objects just by reaching into empty space, just like Tom Cruise! Imagine gesture recognition just like Microsoft’s Kinect game controller, but on a much smaller, more precise scale! Imagine the future, plugged into a USB jack on the Mac or Windows PC you own today!

The Leap Motion sensor is beautiful, tiny and self-contained. It is a sleek, glass-and-aluminum slab (1.2 by 3 by 0.5 inches), with nonskid rubber on the bottom. A single USB cable (both a long one and a short one come in the box) stretches away to your computer; a light comes on when it’s working.

If you have a desktop computer, you put the sensor between your screen and keyboard. If it’s a laptop, you park it on the desk just in front of the keyboard. Soon, Leap says, you’ll be able to buy a PC from HP or Asus that has the sensor built right in.

You download the Leap software, and presto: a somewhat buggy tutorial instructs you to insert your hands into the space – an invisible two-foot cube – that’s monitored by the Leap’s cameras and infrared sensors.

This device is like the Kinect in that it recognises body parts in space. But not only is the Leap far smaller and less expensive, it’s also far more precise. According to the company, it can detect the precise positions of all 10 of your fingers simultaneously, with a spatial accuracy to a 100th of a millimeter – 200 times as accurate as the Kinect.

And remember, the Leap adds gesture recognition not to your TV, but to your computer. A machine that can run millions of different programs for all different purposes. Games, sure, but also office work. Creative work. Communication.

Entertainment. Surely this little wonder is a very big deal.

Unfortunately, it’s not. The Leap’s hardware may be simple, attractive and coherent – but its software is scattershot, inconsistent and frustrating.

The first crushing disappointment is that no software recognises your hand motions unless it’s been specially written, or adapted, for use by the Leap.

There are 75 such apps already on the Leap’s app store, Airspace; some are free, some cost a few dollars. Not all work on both Mac and Windows.

Most are games. In the best of them, you control the action in 3-D space, just as with the Kinect but without having to stand up. For example, Boom Ball ($5 or Rs 295) is the classic Breakout game, where you try to knock out bricks by bouncing a ball against them – but your paddle is attached to your finger in vertical space.

In Disney’s clever Sugar Rush ($2 or Rs 118), a spinoff from the “Wreck-It Ralph” movie, you bake yourself a racing car shaped like a wedge of cake, and then steer it by holding both sides of an invisible steering wheel. When you play Dropchord ($3 or Rs 177), you hold two index fingers out in space; you’re defining a line between them that you use to slice dots and avoid X’s. Cut the Rope is here, too (free).

There are some interesting music-making programs, which makes sense, since hand motions are generally associated with playing instruments. Air Harp ($1 or Rs 59) is just what it sounds like. Chordion Conductor is a sweet-sounding arpeggiator (generates music from chords you select).

A few simple educational apps are available, like Molecules (rotate molecules on the screen; free), Cyber Science 3D (pull apart a skull; free) and Frog Dissection (you guessed it; $4 or Rs 236).

There are some stunningly obvious problems with all of this. First, the motions aren’t consistent from game to game. In some apps, “clicking the mouse” involves jabbing toward something on the screen; in others, you have to point to the object and remain perfectly still while a cursor “fills up.” As on the Kinect, that waiting period is presumably to avoid accidental selections. But keeping your finger absolutely motionless in space is darn hard.

In some apps, you go back to the previous screen with a quick finger-swipe to the left. In others, you waggle your hand horizontally.

There’s just no rhyme or reason. Learning the gestures in one app gives you zero applicable skills for the next one.

The finger-tracking itself works inconsistently, too. In some apps, the computer seems to know precisely where your fingers are; in others, you feel ignored or misunderstood. The result is mushy frustration, like trying to play the guitar wearing boxing gloves.

The second huge problem: Half these apps would work much better with mouse and keyboard, or at least a touch screen. In Neko’s Puddle game ($5 or Rs 295), you conduct dripping liquids along various surfaces by tilting them, but the arrow keys would do just as well.

The third huge problem has to do with the app Touchless. It’s the one app that goes beyond gimmicky demoware and games, the one that comes closest to the promise of “Minority Report.” It’s an app that lets you operate the cursor and scroll windows right in your operating system (OS X or Windows).

The thing is, when your finger is up in the air, how is the software supposed to know when you’ve “clicked” something? There’s no glass to tap, as on a touch screen, and no button to click, as on a mouse.

Leap’s solution: there’s an arbitrary invisible wall in space over the sensor. You can’t see it, you can’t feel it – but when your finger crosses that line, you’ve just clicked.

Painting programs like Corel Painter Freestyle (free) require the same technique.
As you’d guess, this method is a disaster: fussy, frustrating, imprecise. The motion gesture you’ll probably use at this point is throwing the stupid thing out the window.

In short, the Leap Motion Controller is a solution in search of a problem. It desperately needs a killer app, some program that couldn’t exist without the Leap’s special talents: tracking the three-dimensional motions of 10 individual fingertips with incredible precision. Most apps don’t even avail themselves of that 10-finger tracking feature. Most detect only a single finger or hand.

It’s very exciting that the Leap controller has attained excellent hardware design, high precision, excellent speed and low price – today. Unfortunately, the software that will justify its existence still lies in the future.

from:NJYTOUCH