share
Stack OverflowBest programming aids for a quadriplegic programmer
[+125] [7] Peter Rowell
[2010-04-26 00:30:21]
[ python robotics ]
[ https://stackoverflow.com/questions/2710537/best-programming-aids-for-a-quadriplegic-programmer ]

Before you jump to conclusions, yes, this is programming related. It covers a situation that comes under the heading of, "There, but for the grace of God, go you or I." This is brand new territory for me so I'm asking for some serious help here.

A young man, Honza Ripa [1], in a nearby town did the classic Dumb Thing two weeks after graduating from High School -- he dove into shallow water in the Russian River and had a C-4/C-5 break [2], sometimes called a Swimming Pool break. In a matter of seconds he went from an exceptional golfer and wrestler to a quadriplegic. (Read the story ... all of us should have been so lucky as to have a girlfriend like Brianna.) That was 10 months ago and he has regained only tiny amounts of control of his right index finger and a couple of other hand/foot motions, none of them fine-grained.

His total control of his computer (currently running Win7, but we can change that as needed) is via voice command. Honza's not dumb. He had a 3.7 GPA with AP math and physics.

The Problems:

  1. Since all of his input is via voice command, he is concerned that the predominance of special characters in programming will require vast amount of verbose commands. Does anyone know of any well done voice input system specifically designed for programmers? I'm thinking about something that might be modal--e.g. you say "Python input" and it goes into a macro mode for doing class definitions, etc. Given all of the RSI in programmer-land there's got to be something out there. What OS(es) does it run on?

  2. I am planning on teaching him Python, which is my preferred language for programming and teaching. Are there any applications / whatevers that are written in Python and would be a particularly good match for engaging him mentally while supporting his disability? One of his expressed interests is in stock investing, but that not might be a good starting point for a brand-new programmer.

  3. There are a lot of environments (Flash, JavaScript, etc) that are not particularly friendly to people with accessibility challenges. I vaguely remember (but cannot find) a research project that basically created an overlay system on top of a screen environment and then allowed macro command construction on top of the screen image. If we can get/train this system, we may be able to remove many hurdles to using the net.

  4. I am particularly interested in finding open source Python-based robotics and robotic prostheses projects so that he can simultaneously learn advanced programming concepts while learning to solve some of his own immediate problems.

I've done a ton of googling on this, but I know there are things I'm missing. I'm asking the SO community to step up to the plate here. I know this group has the answers, so let me hear them! Overwhelm me with the opportunities that any of us might have/need to still program after such a life-changing event.

Update: I just registered computingforquads.org and I'll be creating pages for all sorts of solutions to all sorts of problems. Thanks for you help so far and keep those answers coming!

(4) Maybe the good guys at superuser.com have good insight about accessibility softwares. Can't say for more programming-related tools, though. Sorry that I can't help. - zneak
(3) Hmm, I would suggest rather using some custom build language than trying to build something which will work perfectly for Honza. There are languages like COBOL which are really keyword based and "human-readable" (and can be spoken out). As with robotics, why not start with LEGO? IMHO some language with turtles (robots - something which moves through the screen and can draw lines) and simple syntax can be pretty good. I have some difficulties programming myself and I understand that being "different" is nothing much, but in this case it look reasonable for me. - Gabriel Ščerbák
(7) We've discussed programming by voice before (basically, it's still rough, experimental, and largely unsupported). See stackoverflow.com/questions/1408874/… and stackoverflow.com/questions/87999/… . - dmckee --- ex-moderator kitten
(1) This is an interesting question of it's own, hopefully you'll get some answers. Best wishes to Honza. - Damien Wilson
(4) Sadly I don't have a good answer for most of this. I notice in the article, though, that the index finger movement is enough that he can control the small joystick on his wheelchair. You may wish to look for a small joystick-style USB mouse. I've seen a few out there before. Though not particularly cheap (too low-volume an item), it may prove helpful as an additional input device. - Nicholas Knight
@Nichols: Actually, this sounds like a really good idea. I will be meeting Honza and Brianna for the first time on Tuesday and this will definitely be on the list of things to discuss. There are so many weird USB input devices out there something must be usable within his range of motion. Thanks! - Peter Rowell
@RCIX: Was that meant as sarcasm or was it a helpful reference that I'm too dense to figure out? We're fully aware of Dragon NaturallySpeaking. Helpful answers have included Andre's pointer to macro sets for emacs with C++ and Python. Also interesting is the Dasher Project, and a suggestion about miniture USB joysticks. If your comment was meant to be helpful, please expand on it so I understand what you are referring to. If it was meant as sarcasm, you might want to spend some time working on your personal sensitivity skills. - Peter Rowell
(2) @RCIX: I understand and I appreciate your enthusiasm. Online misunderstandings like this go back at least to PLATO IV Notes in 1973, where we learned that typing does not convey important body language and tone of voice. There was one FlameWar in 1974 (which grew out of a misunderstanding) that was the basis for what was probably the first Master's thesis on interpersonal interaction in a computer-based social environment. June 2 & 3 there will be a PLATO @ 50 conference at the Computer History Museum in Mt. View, CA. There's gonna be a lot of greybeards hobbling around. :-) - Peter Rowell
[+92] [2010-04-26 01:37:29] Anycorn [ACCEPTED]

I have sports injuries, and I cannot type more than few characters without serious pain.

Instead, I use emacs together with Dragon NaturallySpeaking. I have written macros and scripts to help it get by. The system is not perfect, but it works. I program mostly in C++, but I also use python.

If you want to, I will help you with that. I have to warn you, it takes few months to get used to speech to text software and train it. moreover, I am not native English speaker, am sure that gets in the way

Do not despair, there are solutions.

here a link to emacs and Dragon files (unfortunately have not documented yet)

http://code.google.com/p/asadchev/source/browse/#svn/trunk/home/Dragon

http://code.google.com/p/asadchev/source/browse/#svn/trunk/emacs/elisp

also, if you need more info, my number is 515 230 9363 (United States, Iowa). I will be glad to help you if I can


(1) Please don't let this being check as the "answer" stop you from posting more possibilities. I'll also add that (gasp) I've been a vi (and now vim) guy since 1980, so I'll have to be creating a "vi macro set" to complement the work that Andre has already done, and probably also a Notepad++ set, or an Aptana set (Eclipse + PyDev). - Peter Rowell
Wow, aaa, I did not know that until I read your post. All of your posts I've seen so far does not show it... at least I know Dragon NaturallySpeaking works well for you. - Xavier Ho
1
[+19] [2010-04-26 01:33:35] Norman Ramsey

It's worth looking at the Dasher Project [1], which makes it possible to enter text reasonably quickly even for the severly disabled. Dasher is built on a probabilistic model of languages, so that more likely utterances are easier to enter into the system. The demonstration system comes with a fairly impressive collection of natural languages. It should be easy to get a large corpus of programs written in Python, load Dasher with the corpus, and create a special-purpose version for entering Python programs.

[1] http://wol.ra.phy.cam.ac.uk/

Excellent suggestion! I'm an old neural net jock from the 80's and this sound very promising. - Peter Rowell
2
[+8] [2010-04-26 02:39:56] PaulMcG

This isn't part of any professional software, but when I saw this, I've thought it would be good for text entry using eye movement tracking or minimal mouse movement. See Ken Perlin's Processing page [1], and look at the applets for "pen input".

[1] http://mrl.nyu.edu/~perlin/

Thanks! Great link! This may be particularly useful with cursor control, which I believe is still a bit of a challenge for him. It's not until you actually have to address these types of problems that you begin to understand that some of these people are climbing Mt. Everest every single day just to do what we think nothing about. - Peter Rowell
3
[+5] [2010-04-26 01:44:22] inspectorG4dget

I know someone in a village in India who is a paraplegic, who uses Dragon Speech to Text [1] software to write on her computer. I don't know how well suited it is for a programmer (she is not a programmer), but it's a start.

You might also want to look into Natural Point [2]. It's an eye controlled mouse, which might help Honza

Hope this helps

[1] http://www.topshareware.com/dragon-speech-to-text/downloads/1.htm
[2] http://www.naturalpoint.com/

Honza is currently using Dragon fairly well. We were especially focused dealing with the verbose way you enter the special characters that are all over the place in programming. - Peter Rowell
4
[+3] [2010-04-26 04:46:40] Mark Harrison

iPython with completion

On the python side, iPython shows parameters, functions, etc, and has command completion. Perhaps it could also be customized to respond to the various input devices as well?

http://ipython.scipy.org/moin/


5
[+1] [2010-04-27 04:03:19] RCIX

One thing that may help (i got it from this question [1]) is http://shorttalk-emacs.sourceforge.net/ . It seems to be an interface between emacs and speech recognition. And regarding languages, i would recommend using Lua [2] over Python. It has a more natural English flow to it.

[1] https://stackoverflow.com/questions/1408874/speech-recognition-programming
[2] http://www.lua.org

6
[+1] [2012-02-13 10:56:12] user131948

I know this question is quite old now. I wonder how things are going for Honza with respect to programming. It would be nice to hear back.

For what it's worth, I suffer from RSI and now try to minimize use of the keyboard and especially the mouse.

My own experience of voice recognition is that this stuff DOES work. I use Windows's inbuilt speech recognition software for Windows 7 (WSR). I've also used voice finger (http://voicefinger.cozendey.com/) to help move the mouse pointer. Some key points I would mention are:

Learn the shortcuts. You can do almost anything using shortcuts and speaking them works great using Voice Recognition when in "typing mode" (see below).

Use Typing mode. Unless you are dictating text this is great for speaking short cuts to the computer or for spelling weird words. Interestingly it is not a clearly "advertised" function of WSR.

Phonetic Alphabet. To make good use of typing mode learn the phonetic alphabet: http://en.wikipedia.org/wiki/NATO_phonetic_alphabet You can't realistically get by without this when using any form of speech recognition.

VIM. (or emacs I guess - not sure). Vim is a great for editing text without touching the mouse - ever. This makes it great for editing texts using WSR. I am only a VIM beginner myself but find it incredibly helpful.

Web browsing. In my experience web browsing is still an extremely difficult thing to do without a mouse. There are simply too many situations which require you to hover with the mouse in order to get to the underlying commands. This is a great shame. Nevertheless there are some really good Firefox plugins to help browsing without a mouse.

  1. Mouseless browsing: https://addons.mozilla.org/en-us/firefox/addon/mouseless-browsing/ This is a neat little tool with lots of configuration options for putting numbers next to all links. You can then type these numbers to activate the link
  2. Vimperator: http://vimperator.org/ This goes quite a lot further than mouseless browsing. It basically gives you complete vim like control of firefox. I find this works better than mouseless browsing, but can be annoying in instances where it changes default Firefox behavious.

These are just my own personal experiences. It would be great to hear back about how Honza is getting on.


7