This year, I will write a shitty GUI for my Emacs clone

Following the comments, we will then come across the central data structure in Emacs redisplay: glyph matrices, defined in dispextern.h. The authors have made it flexible enough to support three text rendering/layout scenarios: Emacs GUI windows, Emacs TUI frames, and Emacs buffer windows nested in TUI frames:

Each glyph matrix contains a bunch of glyph rows, whose contents are filled by display_line, which accepts a glyph iterator and repeatedly push the iterator forward, filling out glyph slot one by one.

It still sounds quite simple… until it doesn’t.

It turns out that “pushing the glyph iterator forward” in display_line is just not as simple as it seems: not all texts are read from left to right, and the glyph iterator does not always move one character at a time. Here is an example:

(let ((ss '("star" "כוכב")))
  (cons '("string" "0" "1" "2" "3")
        (mapcar (lambda (s) `(,s . ,(mapcar #'string s))) ss)))

Notice how "star" is rendered as s t a r (of course), but כוכב is rendered in a different order? The first Unicode character s[0] is כ, which is rendered at the rightmost column, while the second Unicode character s[1] (ו) is rendered as the third column counting from the left.

This is because the כוכב text is in Hebrew, which should be read from right to left. And when we mix English and Hebrew texts in the same line, any competent Unicode rendering engine will make sure that the Hebrew text is rendered from right to left. And that means, for right-to-left texts, our glyph iterator will need to jump until the end of that segment and iterate backwards.

I would describe this as a solved problem: Unicode has a well-defined algorithm for rendering bidirectional texts. However, Emacs’ bidi.c is around 120 kB, or around 4k lines, which still doesn’t include the code to pack and unpack a whole Unicode database. I guess that gives a good idea of how much work it might take to support bidirectional text in Emacs, if you don’t use an external library.

By the way, Emacs supports BIDI rendering even under terminal sessions, which I don’t think a lot of TUI programs are doing these days. That also makes me a bit ambivalent about discussions on some new, shiny TUI programs: many of them just don’t support BIDI rendering, and, depending on what they claim to do, that’s … quite telling.

For some other languages, BIDI rendering is not enough. Their characters might “compose” into more complex glyphs, or, “ligatures”. For example, the following Arabic texts ² must be rendered together as a single glyph to be readable for Arabic speakers:

Characters: “ड् ड ب س م”
Rendered:   “ड्ड بسم”

Supporting ligatures is not a simple matter. It has to know about the font(s) used, and thus needs to handle platform-specific font databases, support font fallback, etc., and have decent performance. The industry standard for this is the HarfBuzz library, and there is a reason that its home page references the xkcd “Dependency” meme:

Emacs also makes use of HarfBuzz, and does support ligatures. It’s only that, Emacs being Emacs, it allows you to customize the whole process, with Lisp code.

Emacs supports ligatures through its character compositions. When iterating through a glyph iterator, it calls composition_reseat_it to produce the corresponding composed glyphs (if any), and then walk the iterator into those glyphs. But, instead of directly delegating to HarfBuzz, Emacs checks composition-function-table first, where the user might put a regexp precondition and then any Lisp code to compose glyphs as they wish.

For some (or many?), this can be annoying since the default precondition regexps exclude most ligature combinations in programming fonts. For example, - > will not compose into something like → even if you use a font like Fira Code. To support that, you will need to configure the composition-function-table manually, or use ligature.el.

The above is only about text layout - what glyphs to use on screen, and where to place them. But, like any text rendering engine, of course Emacs allows you to style your text, with something called text properties. For example, by setting a proper face property for a text span, one can customize its font, size, colors, and text decorations. Also, there are some more self-explanatory properties, like invisible, line-height, line-prefix, or display-line-numbers-disable.

Basically, it’s just CSS for text, except that Emacs has made some parts of it … nearly Turing complete?

So, there is another text property called display, which can contain display properties. For example, with a string display property, the text span will be replaced by that string. For another, by setting a height display property, one can control the height of the texts:

And, by setting that height display property to a symbol…

… one can control the height of the texts dynamically, as the Emacs redisplay routine will now call the function pointed by that symbol for a height value.

• tab-width: Of course, you can control the tab width.
• Character display control: Emacs lets you control how a character is displayed. This is orthogonal to text composition.
  • ctl-arrow: Non-nil means display control chars with uparrow ^, like ^M.

  • glyphless-char-display: For glyphless chars, one can display them by hex code, tofu boxes, etc.:

    

  • Display tables like buffer-display-table: This allows you to override the display of characters in a buffer by replacing them with a string in the table.

    

• composition: Another special text property. One may use this to overlap glyphs together. This is purely visual and only works under GUI Emacs.

  

• Overlays: Lisp objects to “overlay” text properties on texts.
• Mode line: Emacs has a mode line for each buffer window, serving as a status bar. One may use an :eval property to put custom content produced by arbitrary Lisp code into the mode line.

• composition-function-table
• New line character overrides through display tables or text properties
• Synchronous Lisp code execution from redisplay

• A buffer sync mechanism: texts, properties, and overlays.
• Proper display of propertized texts.

• Sync mechanism for display tables and glyphless char display tables.
• Asynchronous Lisp code execution from redisplay (text properties & mode lines)