Below is a quick demo of me "typing" a short phrase and some numbers using a modified version of my Blind Input iPhone web app. This is a proof of concept app that doesn't integrate into the phone's other text entry forms, and is arguably more difficult to learn than braille. In case it appears I've solved some great accessibility problem, Siri and the iPhone's native accessibility features has me beat in spades, and my app is only an interesting idea I wanted to experiment with.
The current manager of my group at work, Dennis DeVendra, is blind. He was the manager of my group 2 re-orgs ago, from 2003 to 2005, and he is again after our most recent re-org in late 2010. (If you're in IT and you aren't working for a startup, you're familiar with re-orgs - they happen about as often as Superman's origin story gets re-written, and for roughly the same reason.) Anyway, we've had several conversations over the years about what being visually impaired is like. He introduced me to the world of blind advocacy, the story of blind skier Mike May (Crashing Through), and Dennis himself is an accomplished wood turner, as shown in the video below, making bowls, pens, perfume bottle stoppers, and the like as a hobby. (And you should buy his wares from his Zibbet store, they make great Christmas presents for your discerning, artsy kin.)
At work, he uses a screen reader, JAWS, to interface with his computer, and having only touch to go on, is a more effective touch-typist than I am. It's likely he could keep pace with my 70ish WPM average, despite not being able to see the keyboard or the screen. Despite his intuition and skills, however, if an application is not built with accessibility in mind, it becomes an order of magnitude more difficult to use. Hold that thought for a minute.
When coding my HTML5 spirograph app (a work in progress, it's semi-functional for iPhones, and you can test it here), I had to figure out exactly how touch events work. Without going into too much detail, touches trigger event listeners the same way mouse events do, but the object passed to the listener contains, instead of a single event, an array of all the active touches (in most cases), up to a maximum of 5.
Once I got my head around that, it was a small jump to the idea of using multiple touches instead of a pop-up keyboard for text entry. I toyed around with different ways to express the idea of chording on an iPhone, trying to find a way that would be fun and easy to learn. I abandoned a couple early ideas as being too difficult to learn or code, and put the project back on the to-do pile for a while.
Sometime later, my group at work had a long team meeting. In hour 2 of the meeting without getting up to stretch or grab a coffee, my mind predictably started wandering, and I glanced over at Dennis while thinking about the chording idea. I quickly jotted down a note for myself:
"Blind input?"
I spent some of that night lying in bed and tapping on a book in various ways, experimenting with different ways to enter data, and drawing a few odd glances from my wife. The final method I settled on involves chording in four "quadrants" on the phone's screen- basically, tapping the screen's corners. The method has a few simple rules:
The screen is divided into four equal parts by drawing axis lines down the horizontal and vertical middles of the screen. Touches are identified by what quadrant they fall into. Chords of up to three simultaneous touches define characters. A quandrant can contain no more than two touches - any more than that is uncomfortable to try to squeeze into one quarter of an iPhone screen. Each time a new finger touches the screen, all current touches define the active chord. When the number of touches reaches 0 (when the last finger lifts off the screen), the active chord's character is typed.
The last two rules basically mean that as long as no new finger touches the screen, once you have your "chord" defined, you can remove your fingers from the screen in any order, slowly or all at once, maybe accidentally dragging a finger across an axis as you lift it, and still get the correct character.
For two-touch chords, there are 10 possibilities using the combinatorics with repetition formula (n+r-1)!/r!(n-1)! where n is 4 (4 quadrants) and r is 2 (two touches). Observe:
(n + r - 1)! = (4 + 2 - 1)! = 5! = 120 r! (n - 1)! = 2! (4 - 1)! = 2! 3! = 2 * 6 = 12 120 / 12 = 10
The 10 possibilities lends itself to representing the 10 digits, which I assigned like this:
Quadrants 1 2 3 4 Positions Character 11 1 12 2 13 3 14 4 22 5 23 6 24 7 33 8 34 9 44 0
In other words, tapping two fingers in the upper-left types the number 1. Tapping a finger in the upper-left and one in the upper-right types 2. Etc.
For three finger touches, the math gets more complicated since each quadrant can only repeat one time. It was a small effort to enumerate through all the possibilities, so I skipped the math. There were 16 possible three-touch chords. 16 plus 10 is 26, so this is ideal for a "mode" for alphabetic entry, and another for numbers and symbols.
Here is my current draft of how the chords map to characters (Tapping quadrant 1 alone puts you in shift mode, 2 alone puts you in number mode):
Positions Normal Shift Number Shift + Number
1 Shift Normal Normal Normal
2 Number Number ShiftNum Number
3 Space Tab , <
4 Enter Enter . >
11 a A 1 !
12 b B 2 @
13 c C 3 #
14 d D 4 $
22 e E 5 %
23 f F 6 ^
24 g G 7 &
33 h H 8 *
34 i I 9 (
44 j J 0 )
112 k K - _
113 l L = +
114 m M [ {
122 n N ] }
123 o O \ |
124 p P ; :
133 q Q ' "
134 r R / ?
144 s S ` ~
223 t T
224 u U
233 v V
234 w W
244 x X
334 y Y
344 z Z
For example, if I want to type "The quick brown fox", as I did in the video example above, I would enter the following screen touches:
What I have so far is still in its infancy, and my hope is that it can be fleshed out to be a full-featured text input system that interfaces well with the iPhone's VoiceOver features. The app can be played with here: http://etchapps.appspot.com/key.html, however as I said above, it isn't integrated into anything, and it would be pretty clumsy to cut and paste out of this web app into other apps. For now. Maybe this will turn out to be Just The Thing for blind input in a future build, but for now it's only an interesting experiment.
As a closing note, Dennis told me that the iPhone's native accessibility features combined with Siri make it the first electronic device that has been fully usable by him out of the box, with no third party add-ons. That's one hell of an endorsement.
No comments:
Post a Comment