When I consider the imminent takeover of Cloud Computing and virtualization (the separation of software from its hosting hardware), it makes me wonder why that general paradigm hasn’t migrated into how we look at computer input methods. Before the advent of virtualization, Operating Systems where inextricably linked with specific hardware. One Windows system controlling one box. Sadly, in much the same way, the archaic keyboard and mouse remains an indelible part of how we picture ourselves using technology when it doesn’t need to be. One wonders if this is a symptom of repeated exposure rather than it being that the keyboard mouse solution is perfectly optimal for us. Lately, I have joined the chorus, alongside Elon musk and others, in lamenting how primitive our current computer input solution really is. Relying on 10 meat sticks to communicate is hardly an improvement over the voice and gestures of our ancestors.

The Problem

A while back, when I was trying to get a handle on some devastating repetitive stress injuries to my forearms, neck and shoulders, the added insult was the sheer scope and scale of misinformation, dogmatism, and faux expertise that surrounded the area. It was here I started to learn that branches of science in their infancy don’t initially attract skilled and talented practitioners eager to conquer new territory. Instead they draw the hordes of crystal ball wielding pseudo-scientists ready to give their unqualified two cents on the matter. As Hitchens said, “Humans would rather have a bunk theory rather than no theory at all”. With a lot of digging through that mountainous pile of bad advice I did manage to pull some diamonds out of the dunghill. The other side of the coin when it comes to Web 2.0 is that with a bit of homework you can bump into a handful of bright, talented, counter-cultural figures online who are beginning to actually make sense. Here’s the crux of it according to them:

“Static human positions repeated and sustained for long enough will result in painful and immobilizing adaptations of the body, right through to the cellular level. Add into the mix the fact that those positions had compromising geometry in the first place and it’s bad times all around.”

There is no chair out there that represents the panacea of all chairs either, ready to silver bullet this problem. Even if it has a modern-Sheikh marketing campaign of YouTube videos with narrow depth of field, a whitewash background and a few half-hearted mentions of cherry picked research papers. Spending 6 months mastering the newfangled, triple Lotus 500 position didn’t seem to get to the kernel of the issue either.

Like all things, it came down to systemic change across many aspects of my day, especially the activities that are repeated and habituated. What was required in the end was an overhaul of my outlook on things. Real progress came about by moving to a paradigm of high variety and high quality movement, with equal emphasis to both.

The school of thought proceeding this one is probably that of the weekend warriors. Having paid my dues as one myself, I would class this type of person as someone who compartmentalizes their physical well-being into a narrow one-hour chunk at the end of their day. The expectation is that if you fill that hour with a Herculean effort on the treadmill or under the bench press you’ll be able to offset how you spent the better part of your day.

Instead of compulsively trying to counteract a deeper-seated problem with Band-Aid solutions, what about uprooting it at its base by fundamentally changing the way we interact with technology. Can we change to a system that from the get-go promotes good posture, variety and mobility while using the computer? If it would be a ridiculous notion drive down to a local networking event and frantically shake everyone’s hand for an hour to make up for spending your work days watching Videos, I don’t see why the same thing doesn’t hold when thinking about health and physical well-being.

The Solution

Now time for the rubber to meet the road. Interaction with the computer is split cleanly into only interaction two activities:

1) pointing devices to navigate graphical user interfaces. Mice and trackpads etc.

2) using keyboards to input strings of characters to represent different types of information and communicate. English prose, programming, communication through email or blogging platforms , it’s all strings of characters when broken down to the atomic level.

Overhauling the mouse and trackpad

For navigating GUIs an eye tracker is used to move the cursor around the screen. They are accurate enough and affordable enough to handle cursor movement. The problem of registering the half-dozen mouse events (clicks, scrolls) is handled with the new wireless tap strap across the fingers.

The tap strap is a corded keyboard. If you program certain tap combinations to the more one used keys of the keyboard and then in turn map those keys to different mouse events you are golden. All remapping is done with AutoHotkey. For an example of the whole mouse replacement, see the video below. This frees up my neck, wrists, arms and body to be position agnostic as I can tap and look from anywhere which is why I think the solution is the auto hot keys to the Kingdom.

Voice control the bleeding edge

Apple fan boys. I certainly was one. I worked there and everything. So when I begin evangelizing about Windows and how amazing it is for properly overhauling your workstation(circa 2018)., perhaps you’ll believe me a bit more . When it comes to progressive workstations, Windows is a flourishing ecosystem where Apple is a post-apocalyptic disaster where all that remains is a desolate landscape with amoeba fighting for the remaining resources. Dragon 15 on Windows is the gold standard for voice control for at least 3 reasons.

– The software itself is mature and feature-rich, a.k.a. it works, it doesn’t break and it can actually do loads of stuff as the Application is supported by over 100 developers.

– Secondly, Dragon 15 now leverages deep learning algorithms placing it squarely in the category of bleeding edge technology.

– Thirdly, Windows lends itself beautifully to being controlled by keyboard shortcuts alone. If you just know every single shortcut you can go a long way. Dictating keyboard shortcuts generalizes your commands and means you can start operating in new apartments right off the bat.

Even with all these in toe, shit didn’t get serious until someone who worked there created an extension package, Dragonfly, giving a back passageway to the recognition engine and allowing you to program your own commands in Python. This brings us to the main course.

Voice Coding

Dragonfly is incredible because it heralded in the new area of programming by voice. Repetitive strain injury, bad posture back neck and shoulder pain vaporized. The tutorials for this are noticeably absent online but with a little persistence if you just figure out where to modify their template files it’s a piece of cake. Here is a demonstration of me controlling the computer with voice commands and programming with only voice.