2023-10-05, updated: 2024-03-12
The role of keyboard design in advanced shell programs
Abstract
On why the keyboard is likely to remain the leading input device for knowledge workers, how keyboard design impacts ergonomics, and the consequences that keyboard firmware has on the design of advanced shell programs (those that expose the OS to end users).
Input devices
Unlike typical users who interact with information systems in the way they were meant to be used, knowledge workers constantly craft them as to attain optimized workflows. Notice that the process is continuous, since the tasks they use their systems for are a moving target.
Keyboards have been the primary input device for these systems. As their adoption became widespread, more intuitive interfaces have been devised, namely voice interfaces. While intuitive, they are unlikely to replace the keyboard for knowledge workers altogether. Besides being physiologically challenging to use one's voice over long periods of time, it also accounts to a single serial channel, while the keyboard provides ten channels that can be almost regarded as parallel.1
Voice input may be used to enter a first draft of textual information to be subsequently edited with the keyboard. Note that error correction by voice is inefficient. More broadly, from all of the instructions that users send to these systems, few fall in the category where voice input excels.
Some have gone to great lengths to perform complex tasks such as moving the cursor, selecting pieces of text or managing windows by voice.2 It should be noted that the motivation for such endeavors are RSI (repetitive strain injury) and sheer curiosity. The claim that it is pinnacle of efficiency has no support, given the design of today's systems.
The popularity of voice user interfaces is on the rise and it seems reasonable to anticipate that it will become the preferred input method for typical end users. For knowledge workers, however, it may be a mere complementary input device, similar to the role played by pointing devices such as mice, trackballs or touchpads.
Keyboard design and ergonomics
For decades, researchers have been conducting experiments that measure arm strain and proposing alternative keyboard designs such as split keyboards, whose patents appear as early as 1915 by F. Heidner.3 4 The next paragraphs detail how the design of modern split keyboards improve ergonomics, namely by avoiding ulnar deviation, wrist extension and arm pronation.
Take the line that connects the elbow to the wrist, and another connecting the middle of the wrist to the tip of the middle finger. Ulnar deviation is observed when these two lines aren't parallel. Split keyboards avoid it by enabling typing at shoulder width.
Keyboard feet (below the Fn-keys row) cause wrist extension. Since this is undesirable from an ergonomics perspective, providing the user a better overview of the keys is a plausible explanation for its existence. But all knowledge workers are touch typists, thus making them useless. From an ergonomics standpoint, the keyboard should lie flat or at a negative angle (for instance, by placing feet where the wrists rest).
Arm pronation results from laying the palm of our hands down on a table. This isn't the most natural position and keyboard "tenting" mitigates it. The "tenting" is achieved by placing feet on the inner side of each keyboard half.
While fingers move along straight lines, the ANSI/ISO keyboard features row stagger (the A and Z keys are indented with respect to the Q key). Unfortunately, as the mechanical typewriters predate modern keyboards, they have inherited this poor design that prevented the mechanical jamming of keys. Column stagger, on the other hand, makes perfect sense since it accounts for the different lengths of the fingers. The standard keyboard makes little use of the thumbs. Given their strength, it is unfortunate that the space key is so wide since the thumbs could cover more keys.
As a side node on the topic of ergonomics, it is often argued that reaching out for the the mouse must be avoided at all costs. It is inefficient to do it several times per minute (e.g. to adjust the cursor), but that doesn't mean that ditching pointing devices altogether is wise. The golden rule in ergonomics says that sustained periods in any position are to be avoided.
The Sofle keyboard5 in an example of a design that takes all of these considerations into account.
Keyboard Firmware and QMK
QMK6 is a project centered around developing firmware for computer input devices. Find a very superficial overview of some of its features below.
Layers. Think of Fn key, found on laptops, as a way to assign more functions to the same set of physical keys. In QMK, a layer is a 2D array whose dimensions are dictated by the physical keys of the keyboard and the values are keycodes. A keymap is a list of these layers (a 3D array), i.e. a stack of them and only a single layer is active at any given time. If you are familiar with the Fn lock button, then the concept of momentary and latching layer keys is also clear.
Mod taps. On tap they behave like regular keys, while on hold they act like a modifier keys. The consequences are far reaching. There is no longer the need to assign a key to act exclusively as a modifier key. A typical configuration, known as home row mods7, assigns the home row keys to mod taps symmetrically. For instance (on QWERTY), holding F or J triggers Shift, holding D or K triggers Ctrl, holding S or L triggers Alt and A or ; triggers Super. The common 4 modifiers keys map perfectly to the 4 fingers and the mirror symmetry offloads some of the strain. For example, to produce Ctrl+f both hands should be used: holding K to produce Ctrl with the right hand, and tapping F with the left hand.
Tap dance. Send a different keycode depending on how many times a key is tapped.
Combos. Press several keys together to produce a particular action. Example, press Q and W together and trigger Esc.
Mouse emulation. Control the pointer with the keyboard. It isn't as accurate as other pointing devices but it avoids reaching out for one for simple tasks.
In short, this non-exhaustive set of features enables squeezing the number of required physical keys and thus keeping the fingers closer to the home row position. There is a vibrant community doing intense experimentation that culminates in systems such as Miryoku.8 9
For those who frequently use the laptop's keyboard or don't have a keyboard running QMK, it is possible to get a taste of its set of features thanks to Kmonad10. Find below how a basic keymap that implements home row mods and a navigation layer would look like in a Kmonad configuration file.
(defcfg
input (device-file "/dev/input/by-path/platform-i8042-serio-0-event-kbd")
output (uinput-sink "KMonad: Dell XPS 9350 (ISO keyboard)")
)
(defalias
;; Home row mods
;; "A" key on tap, Super key on hold
*a (tap-hold-next-release 175 a meta)
;; "S" key on tap, Alt key on hold
*s (tap-hold-next-release 175 s lalt)
;; "D" key on tap, Ctrl key on hold
*d (tap-hold-next-release 175 d lctl)
;; "F" key on tap, Shift key on hold
*f (tap-hold-next-release 175 f lsft)
;; Mirror modifiers keys on the right hand side
*j (tap-hold-next-release 175 j lsft)
*k (tap-hold-next-release 175 k lctrl)
*l (tap-hold-next-release 175 l lalt)
*; (tap-hold-next-release 175 ; meta)
;; "H" key on tap, enter Navigation layer on hold
*h (tap-hold-next-release 175 h (layer-toggle nav))
)
;; Declare the keymap of the original keyboard, the usual ISO laptop keyboard
;; (slightly differs from ANSI).
(defsrc
esc f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 prnt ins del
` 1 2 3 4 5 6 7 8 9 0 - = bspc
tab q w e r t y u i o p [ ]
caps a s d f g h j k l ; ' \ ret
lsft lsgt z x c v b n m , . / rsft
lctl lmeta lalt spc ralt rctrl left up down rght
home pgup pgdn end
)
;; Defines the default layer. The CapsLock/Ctrl swap illustrates how easy it is to remap keys.
(deflayer default
esc f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 prnt ins del
` 1 2 3 4 5 6 7 8 9 0 - = bspc
tab q w e r t y u i o p [ ]
lctrl @*a @*s @*d @*f g @*h @*j @*k @*l @*; ' \ ret
lsft lsgt z x c v b n m , . / rsft
caps lmeta lalt spc ralt rctrl left up down rght
home pgup pgdn end
)
;; Navigation layer
;; The "_" means that the keycode from the default layer is used.
(deflayer nav
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _
_ pgup home up end _ _ _ _ _ _ _ _
_ pgdn left down rght _ _ lsft lctrl lalt meta _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _
_ _ _ _
)
In the sections above, the keyboard has been analyzed from the perspective of ergonomics and firmware. Note that while Kmonad gives a taste of the latter, it improves the former by a very small margin. Nonetheless, it can be an interesting bridge solution for those considering experimenting with these ideas.
Consequences for advanced programs
Software that does keyboard configuration on the OS-level (AutoHotKey, xmodmap, etc) becomes obsolete. Given the importance of the keyboard in human-machine interaction, it is desirable to keep the configuration saved on the device as to ensure that it behaves the same regardless of the system it communicates with.11
Many advanced programs go to great lengths as to lessen the quirks of keyboard design. Emacs and vi bind commands to keys that are easier to reach on standard keyboards at the cost of intuitiveness. For instance, the motivation for binding C-n/p/f/b to cursor movements is grounded, but loses its actuality as soon as pressing the arrows keys becomes just as easy. Among QMK users, it is common practice to have a layer dedicated to navigation keys which is activated by holding down a thumb key. This way, it is possible to eliminate the discomfort of reaching out for these keys and to prefer semantic keybindings over cryptic ones, i.e. PgDn (PgUp) is easier to learn than C-v (M-v).
These programs also frequently implement keyboard macros as a feature of their own, which is not very efficient since each program needs to re-invent the wheel. This can theoretically be solved by having a cross-platform library that would make it easy to implement such a feature. But it would still be subpar. Keyboard macros obviously belong to keyboard firmware, and QMK provides it. It may not yet be as advanced as Emacs' implementation, but it's a solid step in the right direction.
The same argument as above can be made with respect to repeat keys. Emacs implements the feature (repeat.el), and so does QMK. But the latter provides a single implementation and constant behaviour for arbitrary programs and operating systems. QMK also defines an alternative repeat key that performs the dual behaviour of the last pressed key. For instance, if the hit last key was Backspace, then the alternative repeat key would trigger Delete. The same applies to key-chord.el (which is covered by QMK's combos).
The downfall of modal editing as implemented in vi or Kakoune12. The problem with the approach is that, when using several of such programs, the user needs to remember or check the modality state for a fraction of second when cycling between them. Even if we take 3 programs and each of those features 2 modes of operation, then the state space size is 8. In short, this approach doesn't scale. The idea behind modality is certainly interesting but it can be attained in different ways.13 If modality is implemented exclusively on the level of keyboard firmware, then the aforementioned issue vanishes since keeping track of mode toggling when using a single keyboard is easier. It is also possible to achieve modality with a momentary or latching switch. I.e., a layer that serves the purpose of vi's normal mode can be enabled while holding a key, or when a key is pressed and released (then another key press is needed to return to the former layer).
Conclusion
It is not the user-space software's responsibility to account for the idiosyncrasies of poor keyboard design standards. It is urgent to bring modern keyboard design (both hardware and firmware) to the mainstream among knowledge workers. Keyboard firmware that enables user customization is paramount. Tweaking the keyboard's firmware must be as easy as binding a command to a key in Nyxt.
Advanced shell programs become smaller when their users use proper keyboards, which allows for the developers to focus on the project's key goals and tame its complexity. In Nyxt's concrete case, the implementation of the repeat-key command, cruise-control-mode, macro-edit-mode and modal keyschemes would become redundant when using the ideas shared in this article. While redundancy is a characteristic of robust systems, there's still an important dimension to take into account, namely that of health and ergonomics.
Footnotes
https://www.ergo.human.cornell.edu/DEA3250Flipbook/DEA3250notes/keyboards.html.↩
Rempel D. The split keyboard: an ergonomics success story. Hum Factors. 2008 Jun;50(3):385-92. doi: 10.1518/001872008X312215. PMID: 18689043.↩
https://patentimages.storage.googleapis.com/b9/18/ee/7d1163a9aab3d8/US1138474.pdf↩
Sofle. Nowadays, hobbyist can assemble DIY split keyboards or buy them assembled from small vendors. There are plenty of open designs for PCBs or even case models for 3D printers. Keycaps, switches, and microcontrollers and other parts are also readily available.↩
https://github.com/manna-harbour/miryoku/tree/master/docs/reference↩
For an example of a keymap definition on QMK, see the author's configuration.↩
Kmonad deserves an honorable mention since it works on GNU/Linux, macOS and Windows.↩
Oftentimes, some mistakenly claim that Emacs isn't a modal editor. Emacs and vi achieve modality by different approaches, but both are modal. The argument can be found in Mickey Petersen's book "Mastering Emacs".↩
Did you enjoy this article? Register for our newsletter to receive the latest hacker news from the world of Lisp and browsers!
- Maximum one email per month
- Unsubscribe at any time