Mission Statement

Project alpha is an attempt to investigate what the alternatives are to using keyboards for text entry into computing devices. Given the inevitable paradigm shift on the horizon due to the VR/AR revolution, it might be time to take a look at the most used and most overlooked part of a computer, namely: the keyboard.

Part One: Chorded Keyboards

VR on the rise

VR Pioneer Aaron Brancotti

Virtual reality is by no means anything new. Attempts have been made since the 80s. However it was only when legendary programmer John Carmack of Quake fame announced his Occulus project that I believed that it had any chance. Since then the Occulus project has been bought by facebook – an event that cooled my interest in that particular enterprise. However, the avalanche had started. What in particular made me believe in the advent was an astute observation made by a man well worthy the title of genius. Carmack’s observation was that latency was key to the experience. The computer architecture stack that we are using today has evolved organically. Step by step, improvements have been added, each of them adding a little bit of latency. Each update had such a relatively small of an effect and seemed to give so much advantage over the previous, so this largely went unnoticed. However, for the illusion of a virtual reality, even the slightest bit of delay may totally destroy the immersion. As such the key thing for any VR/AR solution is to attack these delays, all other objectives are secondary. If you don’t believe me – try using a window manager from the 90’s – such as fluxbox – and you’ll notice how snappy it feels. Every action is instantaneous. To get back to this while retaining the features that we’ve come to love and adore, a total overlook of the entire stack is necessary. Even putting common assumptions, such as using the kernel abstractions, into question. In order to cut down latencies, process isolation practices with roots in mainframe systems from the 70’s might have to be thrown away. Tighter integration with the kernel circumventing security policies are cumbersome at best and outright counterproductive for gaming systems. Apart from that, even the hardware input stack needs to be reconsidered at a fundamental level. Video streaming will probably be a key element of VR solutions, but here, careful attention has to be paid to buffer sizes and timings of I- and P-frames. No longer will there be room for programmers who load framework on framework, adding GB after GB of libraries out of incompetence and laziness onto software solution. Only programmers with a scarcity mindset will thrive in this environment.

Dodging the Consumption Only trap

The worst case scenario is navigating a virtual keyboard using two dimensional input, apart from a system deliberately attempting inefficiency. This system is however accepted as commonplace in 10 foot interfaces, such as gaming systems and smart-TV solutions. Whereas this might be sufficient for finding the latest Justin Bieber or Lady Gaga song on Youtube, for us who still see computers as an incredible tool for innovation and creation, it remains terribly inefficient. Imagine typing a ‘z’ followed by an ‘o’, this would mean going up two steps and to the right about 6 steps, depending on layout. So that is in total 8 keypresses just to find the keys, not to forget two key presses to actually select the keys. For those who have studied computer science, it might be interesting to compare this to a linear search. Research on user interface has shown that each finger has a potential bandwidth of c.a. 4 bits per second and information theory estimates that the information required to store one word of english text is 1 bit. Presenting this data set in a worse fashion could only be done via implementing a truly linear search instead of a quasi linear search.

So if we are to use our VR controllers to point at virtual, laggy representations of decades old technology with only two pointers, we are worse off than old people writing with only their index finger. This would make VR environments not only impractical but borderline hostile to anyone attempting to create instead of mindlessly consume. Failing this, your 1000 USD state-of-the-art gaming rig would only be usable for you as a means for you to be manipulated by psychological methods leveraging your dopamine system by Silicon Valley companies. The systems meant to free and liberate us become the chains to enslave us; a dystopian nightmare reminiscent of Huxley’s A Brave New World.

B..B..But what about speech recognition?

float InvSqrt(float x){ float xhalf = 0.5f * x; int i = *(int*)&x; // store floating-point bits in integer i = 0x5f3759df - (i >> 1); // initial guess for Newton's method x = *(float*)&i; // convert new bits into float x = x*(1.5f - xhalf*x*x); // One round of Newton's method return x; }

Try activating Siri and get her to write and compile the code above, and you’ll realize that there is a long way to go before speech recognition is ready for productive use.

— EDIT: Opinions vary on this one, see comments —



I intend to elaborate on the pros and cons of speech recognition as input. The main point being that upon errors, there is no simple way to correct them. With one of the main sources of errors being similarities between probable phrases, there will be plenty of them. Due to the “high cost” of error correction in terms of time, even with an extremely low error rate, it would still be extremely damaging to performance. One of our primary enemies here is context switching and having to painstakingly correct a speech recognition would be an extremely good example of this. With a keyboard on the other hand, you probably are correcting errors even without noticing it.

What is there to be done?

If you can find it in your heart to forgive me for my longwinded rant, I’d like to introduce you to the work that I am doing. By no means do I claim to have all the solutions to the user interface solutions. What I do have is the aspiration to compile a list of existing venues along with some of my thoughts on them. Stay tuned and you’ll have an as comprehensive analysis on all the alternative input methods that I can find.



Starting with chorded keyboards.

My aim is to do this on an open source collaborative basis. So I might not be actively trying to provoke you by writing glaring inaccuracies – I would not mind having any errors corrected.

Feedback / Discussion

lobste.rs is probably the best forum for computer science at the moment. Check it out for comments from people with minds far sharper than mine. None mentioned none forgotten.

https://lobste.rs/s/jkeeh8/case_for_new_input_method