Class-based, functional configuration

Class-based, functional configuration

One of the strongest assets of Nyxt is its configurability: anything in the Common Lisp code base can be changed and extended.

While technically possible, it proved practically difficult to design a system that users could easily leverage. We were in dire need of a configuration framework that would accompany and empower the user beyond the capabilities of pure Lisp.

Case study: Emacs

The Emacs text editor is a model of extensible computer applications. Its legendary customizability has given birth to thousand of extensions.

Nyxt draws much from Emacs when it comes to extensibility, namely, by allowing the user to reprogram everything from their configuration file.

When Emacs was designed in the 1980s, it's likely that the developers didn't expect third-party extensions would play a significant role in the Emacs community. Nor did they expect, I suppose, that someday hardcore fans would use Emacs for everything, including their window manager.

It isn't all perfect though. Emacs customizability comes with its load of limits, and it's common to read stories of "configuration bankruptcy" – a term coined to mean a user is overwhelmed by the complexity of their own configuration.

A frequent source of Emacs bankruptcy is inherent to its architecture: state. The state consistency of Emacs is one of its biggest weaknesses since much of its configuration is based on unguarded global values. This can lead to hard-to-debug configurations and conflicts between extensions.

What we need

With Nyxt, we envision the growth of a community and, hopefully, a vibrant library of third-party extensions! At the same time, we realize that the global state of Emacs is a limiting factor that must be fixed from the root, lest we reach a point of no return.

Starting from a configurability similar to Emacs (also using a Lisp language), here follows our specification:

Initial attempts

Common Lisp comes with a very powerful object-orientated system: CLOS. We figured out that we should leverage it since it does most of the heavy lifting when it comes to structuring data.

Almost all the data in Nyxt is structured into classes. Thus, configuring an option amounts to configuring a class slot (sometimes called an attribute in other languages).

The first obvious strategy that comes to mind is to make the slot default value point to a global variable which the user can customize:

(defvar *default-modes* '(web-mode base-mode))
(defvar *search-engines* (list (make-instance 'search-engine
                                              :shortcut "wiki"
                                              :search-url "https://en.wikipedia.org/w/index.php?search=~a"
                                              :fallback-url "https://en.wikipedia.org/")))

(define-class buffer ()
  ((default-modes *default-modes*)
   ;; More slots...
   (search-engines *search-engines*)))

The drawbacks are obvious:

This approach is not sustainable. We need a better way. In particular, we wanted to follow a more "functional" paradigm (in the sense of functional programming) – without seeking purity.

CLOS is complex and powerful. Some parts are left unspecified. We initially thought we could leverage this hole in the specs to our advantage.

  1. We first tried to set the slot default value directly by leveraging the introspection library closer-mop. It worked with the SBCL compiler but was not portable and very brittle, it was particularly difficult to handle inheritance correctly.

  2. We tried to replace the class definition directly, for instance

    This hack works as long as there is no inheritance. The new definition won't propagate to its children which results in inconsistent class definitions. This is too confusing and hard to maintain.

  3. Use closer-mop:ensure-class to redefine the class as above. This would trigger the CLOS internals to update all the child classes, thus fixing the inheritance issue. Problem: How do we redefine a single slot without rewriting the code for all slots? Using the above buffer example, if we write

    it would define a new buffer class without any slot but the default-modes. It seems there is no good way to say "insert the rest of original slots here".

Class composition

In computer science, it can be considered good practice to use composition instead of inheritance for increased flexibility and less complexity. This is what led us to implement what we call "user classes": slot-less classes that only inherit from a list of classes, namely the original class followed by the specialized classes. To clarify, let's consider this example with the buffer class:

(define-class buffer ()
  ((default-modes *default-modes*)
   ;; More slots...
   (search-engines *search-engines*)))

(define-class user-buffer (buffer))

Whenever we want to instantiate a buffer, we call

(make-instance 'user-buffer)

Why is this interesting? Because now you can safely redefine the user class without touching the original buffer class:

(define-class my-buffer ()
  ((default-modes '(my-mode web-mode base-mode))))

(define-class user-buffer (my-buffer buffer))

Our user-buffer override now inherits from my-buffer and buffer. Parent classes are ordered by priority, with the highest priority being first. This means that user-buffer will use the default-modes slot from my-buffer and the rest of its slots from buffer.

This resolves our problem statement: no globals (beside user-buffer), we keep access to the original value, we compose all settings and we can revert any change. Indeed, should you decide you want to remove the defaults of my-buffer, you can redefine user-buffer without it.

Even though the syntax is relatively light, we offer the define-configuration helper macro:

(define-configuration buffer
  ((search-engines (append my-search-engines %slot-default))
   (bookmarks-path (make-instance 'bookmarks-data-path
                                  :basename "~/personal/bookmarks.lisp.gpg"))))

This macro comes with some benefits:

Conclusions and thoughts

Our approach is both powerful and flexible. The user has the ability to change any value at runtime – without risk of corrupting global state. Each buffer, or any other class instantiated objects, can be safely manipulated and reverted without fear of repercussion.

We would like to encourage other programs and programmers to empower their users using similar techniques. Give users the tools they need to adapt and modify their programs to suit their workflow!

A tool that works for the 80% is great when you are doing common work, but when you are doing extraordinary work, that extra 20% makes the difference.

Thanks for reading :-)