So, I’ve been designing an app that is basically meant to be used by car drivers. It’s tentative going to have full Voice Control for enhanced usability and safety. It’s going to be a great app. I think.
Although I can totally identify with my lead user, a salesman type of guy that drives all over town, the app wasn’t really resonating with me now that I work out of my home office and spend most of my ‘commute’ either walking, biking, or on mass transit. My automobile time is often with wife and family, so Voice Control would be terrible in-effective over the dins of a car full of talkers. So what about me?
I’m a big fan of Apple’s new Voice Control, but although society apparently now accepts hearing one side of a phone conversation from people walking down the street, are they ready to accept that somebody saying, to no one in particular, ‘Next Item’, ‘Back’, ‘Postpone’, ‘Play songs by the Shin Shins’, isn’t a little crazy, or at least a little socially inept? I’m not ready to be that bold trend setter.
Some grope motions we can recognize
So, how can we interact with our mobile devices while walking down the street? If I presume that I’m wearing headphone, then I can image holding my phone in my pocket and using my phone for input if my app is giving me adequate audio feedback. The phone could say into your ear “Your next appoint with Mr. Smith is in 15 minutes. Tap once to acknowledge or tap twice to remind you again in 10 minutes.” I’m excited about this and can image a whole set of utilities and games that I can use while moving around the city. I’m calling this method of input “Grope Control,” though I’m hoping to get suggestions for a more politically correct name.
For the sake of completeness, I’ve started to enumerate some common input and output methods and came up with some names for some of the combinations, namely Classic Control, Voice Control, Grope Control, and Spy Control. I like to name things. I think the introduction from The Non-Designer’s Design Book put it well when it’s introduction concluded that “Once you can name something, you’re conscious of it. You have power over it. You own it. You’re in control.”
Output Options
| Eyes | Ears | Touch | Comments |
|---|---|---|---|
| X | X | X | Great when the device has the users undivided attention and the user is holding the device. |
| X | X | Good when the user is not holding the device, say, its sitting on a desk or is mounted to a wall or dashboard, but the user can still focus on the device. | |
| X | Great when your hands and eyes are occupied, like when you are one the move, either driving or walking. | ||
| X | Would be great for silent two-way communication, but the mainstream technology is currently limited to vibration on most devices and little 'clicks' on some devices like the Blackberry Storm. |
We’ll ignore mouth and nose as useful output options, though its fun to think about. Hmm, my phone smells musky, and tastes bitter, it must be time for my 3:30 appointment.
Input Options
| Gesture | Tap, Pinch, Zoom, Twist - requires two hands or a mounted device. One to hold the device and one to make the gesture. |
|---|---|
| Gesticulate | Waving the device around, like shaking the device to 'undo' for the iPhone, or bowling for the Nintendo Wii. Requires arm movement. |
| Thumb | One handed thumb control, where you hold the device in, say, your right hand, and then use your thumb for input. |
| Voice | Say a command. Good in quiet places like car or living room. Not good for a bus or subway because its too loud and people would think you are crazy. |
| Buttons | Some Apple headphones let you single click the button on the microphone to stop the song, double click to advance to next song, and press & hold. |
Named I/O Combinations:
| Name | Outputs | Inputs |
|---|---|---|
| Classic Control | Eyes, Ears, Touch | Gesture |
| Wild Classic Control | Eyes, Ears, Touch | Gesture, Gesticulate |
| Wild Muted Classic Control | Eyes, Touch | Gesture, Gesticulate (Wild Classic with the Mute on) |
| Voice Control | Ears | Voice |
| Grope Control | Ears, Touch | Thumb |
| Wild Grope Control | Ears, Touch | Thumb, Gesticulate |
| Spy Control | Touch | Thumb |
| Specialized Control | TBD | Buttons or specialized hardware |
Scenarios
| Scenarion | Attributes | Solutions |
|---|---|---|
| Driving | Limited Eyes. Good audio. Not holding the device, so no haptic (touch) feedback. | Voice Control |
| Walking down the street with headphones | Maybe loud. Don't want to make a scene of yourself. Hands want to be in your pocket during winter weather. | Grope Control |
| Walking down the street w/o headphone. | Maybe loud and you don't want load noises coming from your pocket where you are holding your phone. | Spy Control |
| Business Meeting | Don't want others to hear or see your inputs. | Spy Control |
| Movie Theater | The phone is in your pocket, you can't see the screen and you can't hear the commands | Spy Control |
| Standing in line at the airport | Your hands are free, you aren't doing anything useful, but you don't want to draw attention to yourself. | Muted Classic Control |
| Being old | Many older people are moving away from computers, let alone iphone, because poor eyesight are decreasing finger control makes mouse usage difficult. | Voice Control, (Wild) Grope Control. |
Implications for the app producer
- Know your user personas. How and where will they be using your app? Ask yourself whether you want to limit your app to a single I/O combination, or do you want to allow, and budget for, multiple IO combinations?
- You might also plan on spending some extra marketing effort in explaining and showing the benefits of your non-classic I/O options. Can you imagine the virus buzz you’ll get as the first Spy Control app published. Can you make a killer YouTube video to go along with it?
Implications for the app programmer
- You’ll simply need a solid gesture library. Apple is clearly working in this direction and now has a built-in and refined “Shake” detector.
- Have a standard voice control browser for your platform would be useful. You can test your voicexml stuff at voxeo.com’s site before putting into your app.
- What would the voicexml equivalent for SpyControl be?
Implications for the DoD
- Patent 6899539 addresses putting a trackball onto a weapon to provide (http://bit.ly/Patent899539) some mechanism to get feedback into a computer w/o taking your hands off of a weapon, say, your rifle. The trackball approach presumes that your see a screen, which means you are not focused on your target. Fundamentally the soldier needs more combat appropriate input methods.
Image from patent with rifle and trackball
- Software robustness: If, say, a screen is damaged or, untrusted, should there be another input method available to the operator?
Recent Comments