Exploring Speculative JIT Compilation for Emacs Lisp with Java

Currently, Juicemacs aims to explore a few things I’ve been wondering about for a while:

Transparent Concurrency

I hope to use a JavaScript-like concurrency model, but make it transparent by using Java’s virtual threads ¹ and sending each input event to a separate virtual thread – mounted on the same physical thread to reduce parallelism issues.

Concurrent GUI

I have yet to fully grasp the whole picture. But, for example, to display the mode-line in Emacs, the redisplay routine needs to evaluate Lisp expressions in mode-line :eval clauses or some display text properties, which can block redisplay and thus the whole Emacs. A straightforward approach we might explore is allowing out-of-date mode-lines: we only update them asynchronously when we’re “free” (with an empty event loop).

Just-In-Time (JIT) Compilation

That will be the topic of this post. :)

However, a main problem with explorations in Emacs clones is that Emacs is a whole universe. And that means, to make these explorations meaningful for Emacs users, we need to cover a lot of Emacs features before we can even begin.

For example, I can already see some issues in my concurrency design: what if the execution of different events gets out of order? This is very likely, especially when the events involve blocking hooks. But to see how this would affect real-world editing (and how we could fix this), we need a fully-functional “Emacs”.

For example, one of Emacs’s features is that it supports many encodings. Let’s look at this string 洲﨑神社: it can be encoded in both Unicode and Shift-JIS, a Japanese encoding system.

(let* ((jis-encoded-bytes  "\x1b$B='yu?@<R\x1b(B") ; 洲﨑神社
       (emacs (decode-coding-string jis-encoded-bytes 'iso-2022-jp))
       (utf8 (encode-coding-string emacs 'utf-8-unix))
       (utf8 (decode-coding-string emacs 'utf-8-unix)))
  utf8)

emacs: 洲▯神社 (#x1420A4 > #x10FFFF (max unicode))
other: 洲�神社 (#xFFFD replacement)

But currently, Unicode does not have an official mapping for this “ki” (﨑) character. When we map from Shift-JIS-encoded bytes to Unicode, in most programming languages, you end up with something like �, a replacement character. But in Emacs, it actually extends the Unicode range by threefold and uses the extra range to losslessly support characters like this. (In this case, the “﨑” character is transformed into a #x1420A4 code-point, which is outside the Unicode range 0 - #x10FFFF.)

Unfortunately, if I put that #x1420A4 character as is, encoded in Emacs’ utf-8-emacs coding, Nikola, my blog generator, seems to break. Watch the EmacsConf video (or run the code in your Emacs) if you want to see how Emacs handles and displays that.

By the way, the 洲﨑神社 example is from Unicodeにマッピングされていない文字 - 清水川のScrapbox (“Characters that are not yet mapped to Unicode” in English).

So if you want to support this feature, that basically rules out all string libraries with Unicode assumptions.

Another challenge is supporting Emacs’s regular expressions, which are quite … irregular. :P

For example, it supports asserting about the user’s cursor position, using the \= predicate. It also uses character tables that can be modified from Lisp code to determine case mappings in [[:lower:]] or [[:upper:]].

All these make it really hard, or even impossible, to use any existing regex libraries.

Also, you need a functional garbage collector (GC). You need threading primitives, because Emacs already has some threading support. You might also want the performance of your clone to match Emacs, even with its native compilation enabled. Not to mention, you also need a GUI for an editor.

When I was writing the “Why Rewriting Emacs Is Hard” post, I actually planned (and I’m still planning!) to write a “Why Emacs Redisplay is Hard” post next. And, I’m now fully stuck on this – GUI is much, much harder than the things I mentioned above.

That certainly depends on what feature sets you want the GUI to have, but I guess it just can’t be easy when you’re aiming for:

• A server/client model & protocol,
• Supporting large files (>GiBs),
• Incrementally updated client-side info,
• Bidirectional text rendering support,
• Long line support,
• And Emacs compatibility.
  • Display tables (with which you can even rework newlines).
  • Text-replacing display properties.
  • Self-evaluating display properties (have a look at the height entry).
  • Overlays.
  • …

For Juicemacs, building on Java and a compiler framework called Truffle helps achieve better performance; and by choosing a language with a good GC, maybe we can focus more on the challenges above.

Quoting from Emacs docs:

Because it takes some time to load the standard Lisp files, … --temacs tells temacs how to record all the standard preloaded Lisp functions and variables, so that when you subsequently run Emacs, it will start much faster.

During pdump, we let Emacs (or Juicemacs) load all the files needed for bootstrapping. Then it dumps the memory to an emacs.pdmp file to be loaded later, giving us fast startup.

$ build/native/nativeCompile/elisp-repl --dump=pdump

load: ../elisp/emacs/lisp/loadup.el
Dump mode: pdump
Using load-path ("../elisp/emacs/lisp")
load: ../elisp/emacs/lisp/emacs-lisp/debug-early.elc
load: ../elisp/emacs/lisp/emacs-lisp/byte-run.elc
load: ../elisp/emacs/lisp/emacs-lisp/backquote.elc
load: ../elisp/emacs/lisp/subr.elc
...
Loading leim/leim-list.el (source)...
Loading leim/leim-list.el (source)...done
Waiting for git...
Waiting for git...
Finding pointers to doc strings...
Snarf-documentation not implemented
Finding pointers to doc strings...done
Pure-hashed: 0 strings, 0 vectors, 0 conses, 0 bytecodes, 0 others
Dumping under the name emacs.pdmp

When we try to load that dump back, currently Juicemacs throws some frame errors because it doesn’t have an editor UI or functional frames yet. Otherwise, it can already run quite a bit of Lisp code.

$ build/native/nativeCompile/elisp-repl --dump-file=./emacs.pdmp

Loading ../elisp/emacs/lisp/subdirs.el (source)...
java.lang.UnsupportedOperationException
party.iroiro.juicemacs.elisp.forms.BuiltInFrame$FFrameParameters.frameParameters(BuiltInFrame.java:684)
...
<elisp> frame-parameters(Unknown)
<elisp> frame-notice-user-settings(frame:171:0)
...

Juicemacs’ pdump directly builds on Fory, a (de)serialization framework. Of course it’s not as fast as Emacs, which mostly dumps and reloads its heap memory as is, but going from tens of seconds to less than one second is still quite significant.

For example, this code uses the benchmark library to measure the performance of a Fibonacci function.

(defun fib (x)
  (if (<= x 1) x
    (+ (fib (1- x)) (fib (- x 2)))))
(cl-loop for i from 0 to 10 do
         (funcall #'message "(fib 37): %S seconds"
                  (car (benchmark-run 1 (fib 37)))))

Loading benchmark...
Loading cl-lib...
Loading cl-loaddefs...
Loading cl-macs...
Loading gv...
Loading time-date...
Loading subr-x...
Local variables list is not properly terminated
(fib 37): 0.7413439900000001 seconds
(fib 37): 0.453432999 seconds
(fib 37): 0.397605726 seconds
...

As we can see above, the JIT engine is already kicking in, making the execution faster.

In addition, with a few workarounds, Juicemacs can also run some of the ERT, or Emacs Regression Test suite, that comes with Emacs.

(require 'pp)
(defalias 'pp 'princ)
;; ert asserts about newlines produced by pp
(defun cl--assertion-failed (&rest _) t)

(require 'ert)
(load "../test/src/data-tests")
(load "../test/src/floatfns-tests")
(null (ert-run-tests-batch)) ; don't print the huge info object

...
   passed  86/87  logb-extreme-fixnum (0.000496 sec)
Test special-round backtrace:
  backtrace-to-string(#f(compiled-function (&optional arg1) #<bytecode
  #f(compiled-function (arg1 &rest rest) #<bytecode 0xffffffff8e4e542a
  #f(compiled-function () #<bytecode 0xffffffffa66d753b>)(#f(compiled-
  ert-run-or-rerun-test(#f(compiled-function (arg1 arg2 arg3) #<byteco
  ert-run-tests(#f(compiled-function (arg1 arg2 &optional arg3) #<byte
  ert-run-tests-batch(#f(compiled-function (&optional arg1) #<bytecode
  nil()
Test special-round condition:
    (ert-test-failed ((should-error (funcall f n)) :form (funcall ceiling -1.0e+INF) :value -9223372036854775808 :fail-reason "did not signal an error")
   FAILED  87/87  special-round (0.000754 sec)

Ran 87 tests, 61 results as expected, 26 unexpected (2025-10-30T05:29:18.077226393Z, 5.447236 sec)

26 unexpected results:
   FAILED  bignum-expt
   FAILED  bignum-round
   FAILED  bignum-to-float
...

There are a bunch of test failures, which means we’re not fully compatible with Emacs yet and need more work. But the whole testing procedure runs fine with proper stack traces, which are quite useful for debugging Juicemacs.

So with that, a rather functional JIT runtime, let’s now get to today’s topic: JIT compilation for ELisp.

You probably know that Emacs has supported native-compilation, or nativecomp for short, for some time now. ² It mainly uses GCC to compile Lisp code into native code ahead of time. During runtime, Emacs loads those compiled files and gets the performance of native code.

However, for example, for installed packages, we might want to compile them when we actually use them instead of ahead of time. For that, you can use the native-comp-jit-compilation flag provided by Emacs.

With this flag enabled, Emacs will send loaded files to external Emacs worker processes during runtime, which will then compile those files asynchronously. When the compilation is done, the current Emacs session will load the compiled code back and improve its performance on the fly.

When you look at this procedure, however, it is ahead-of-time compilation, done at runtime. And this is what current Emacs calls JIT compilation. But if you look at some other JIT engines, you’ll see much more complex architectures.

Looking at the results here… In the first three entries (fibn, fibn-tc, fibn-rec) Emacs nativecomp executes instantaneously. It’s a total defeat for Juicemacs, seemingly. But if you’re into benchmarks, you know something is wrong: we are comparing different things.

So let’s look under the hood, again with the disassemble command.

This is actually great! It shows that nativecomp knows about pure functions and can do all kinds of things like removing or constant-folding them. Juicemacs just does not do that.

However, we are also concerned about the things we mentioned earlier: the performance of number additions and function calls.

• Iterative fibonacci: Fixnum arithmetic
• Recursive fibonacci: Intensive function calls + a bit of arithmetic

So, in order to let the benchmarks show some extra things, we need to modify them a bit by simply making things non-constant. ⁴

• Not all functions (+, -, < …) have fast paths?

  1199:	call   *0x2900(%r13)    ; calls the "<" function
  11a0:	test   %rax,%rax
  11a3:	je     12d0
  

Looking at the assembly, it seems Emacs nativecomp does not have fast paths for operations like additions, subtractions, or comparisons, which is exactly what Fibonacci benchmarks are measuring. In the compiled code, Emacs has to call some generic, external functions for them, and this is slow.

In fairness to nativecomp, I also ran the same benchmark in Common Lisp ⁵, with SBCL:

• Comparison with SBCL:

  Fibonacci Speed:
  
  - 🥝Juicemacs > SBCL (untyped) ≈ 🟣 nativecomp
  
  - SBCL (safety 0) >> SBCL (typed) ≈ 🥝Juicemacs
  

Nativecomp is already fast compared to untyped SBCL, because it turns out that SBCL also emits call instructions when it has no type info. However, once we declare the types of the arguments, SBCL is able to compile a fast path for fixnums, which makes its performance on par with speculative JIT engines (that is, Juicemacs), because now both of us are on fast paths.

Additionally, if we are bold enough to pass this (safety 0) flag to SBCL, it will remove all the slow paths and type checks, and its performance is close to what you get with C.

Well, probably we don’t want (safety 0) most of the time. But even then, if nativecomp were to get fast paths for more constructs, there’s certainly room for performance improvement.

The hidden question is actually: is it worth it? And my personal answer is: maybe not.

Moreover, as shown by some benchmarks above, there are some low-hanging fruits in nativecomp that might get us better performance with relatively lower effort. (For example, fast paths for +, -, =, <, >, ..., in combination with declare type hints?) Compared to this, a JIT engine is a huge, huge undertaking.

And there are some JIT techniques that do not require a speculative runtime. For example, you can do many things already with inline caches.

Truffle is a meta-compiler framework that lets you write an interpreter and automatically get a JIT runtime.

In Java code, most operations are just as simple as this:

@ELispBuiltIn(name = "1+", minArgs = 1, maxArgs = 1)
@GenerateNodeFactory
public abstract static class FAdd1 extends ELispBuiltInBaseNode {
    @Specialization(guards = {"isSafeLong(number)"})
    public static long add1LongSafe(long number) {
        return number + 1;
    }
    @Specialization
    public static Object add1Long(long number) {
        return ELispBigNum.forceWrap(number).add1();
    }
    @Specialization
    public static double add1Double(double number) {
        return number + 1;
    }
    @Specialization
    public static Object add1BigNum(ELispBigNum number) {
        return number.add1();
    }
}

And it will then have fast path support for integers, floating numbers, and big integers.

For example, I tried to constant-fold more things. (Inspired by how TruffleRuby handles some of their variables.)

Many of us have an Emacs config that stays largely unchanged, at least during one Emacs session. That means many of the global variables in ELisp are constant. With speculation, we can speculate about the stable ones and try to inline them as constants.

🤔

Real-world implications?

However, for now, we have no idea whether this will improve performance or not, since we still need a full editor to get real-world data, in real-world Emacs sessions.

I also tried changing cons lists to be backed by arrays, because maybe arrays are faster, I guess?

It is actually inspired by the many element types of ararys in V8. I mentioned above that Juicemacs suffers from Java having to box integers (allocate Integer objects) to fit it in to a cons list, and this is one way to improve on that.

But in the end, setcdr requires some kind of indirection, which actually makes the performance worse.

And for regular expressions, I also tried borrowing the “static backtracking” technique from PCRE JIT, which is quite fast in PCRE, but it is unfortunately unsupported by the Java Truffle runtime.

So, looking at these, explorations can certainly fail. But with Truffle and Java, these are not that hard to implement, and very often, they teach us something in return, whether or not they fail.

And explorations are fun! Just for Fun. No, Really.

• Project: https://github.com/gudzpoz/Juicemacs or https://codeberg.org/gudzpoz/Juicemacs
• Accompanying blog post (this post): https://kyo.iroiro.party/en/posts/juicemacs-exploring-jit-for-elisp/
• ERT testing results: https://emacsen.de/dev/tests/
• Zulip chat (devlog + discussions): https://juice.zulipchat.com

The benchmarks above mainly compare between nativecomp and Juicemacs. But, with our analysis, we know that nativecomp is not that fast yet. That means those benchmarks tell us very little about Juicemacs itself!

To see where we can improve Juicemacs, we need to compare with fast languages, and I did perform a few non-rigorous tests running JavaScript (Node.js) and Java, which might be of interest to you. :)

• With Java (floating point operations): In the other blog post (Writing a Lisp JIT Interpreter with GraalVM Truffle), I ran a mandelbrot benchmark in Emacs nativecomp, Juicemacs (AST & bytecode) and Java. Juicemacs AST JIT is only slightly slower than Java. However, the bytecode JIT is 3x slower, which I assume comes from the integer boxing costs.

  

• With JavaScript (function call costs): I also ran the recursive fibonacci benchmark with Node.js. Here are the results:

  	Node.js	Juicemacs AST	bytecode	Native image	Native image bytecode
  Time	~840 ms	~630 ms	~1350 ms	~870 ms	~1700 ms
  Time	1x	0.75x	1.6x	1.04x	2.0x

  I still need to make sense of the performance difference here… But in order to do fast pdump, we will most likely stick with “native image bytecode”, so think of Juicemacs as 2x slower ⁸ than JavaScript (V8).

  Another interesting thing is that “mutual recursion” makes the code execute faster in both Node.js and Juicemacs (actually Truffle/Graal):

  const fib1 = (x) => (x <= 1 ? x : fib2(x - 1) + fib2(x - 2));
  const fib2 = (x) => (x <= 1 ? x : fib1(x - 1) + fib1(x - 2));
  

  (defun fib1 (x) (if (<= x 1) x (+ (fib2 (1- x)) (fib2 (- x 2)))))
  (defun fib2 (x) (if (<= x 1) x (+ (fib1 (1- x)) (fib1 (- x 2)))))
  

  Notice how fib1 calls fib2, and fib2 calls fib1. Truffle offers us some options to look into how things are compiled, like the engine.TraceInlining=true option:

  sh -c "cd app && $(./gradlew -q :app:jvmCmd) --extra-option=engine.TraceInlining=true --dump-file=../emacs.pdmp"
  

  With this, we can look into the inlining depth of the fibonacci function:

  [engine] Cutoff fib   |Recursion Depth      3|...|Depth      3
  [engine] Cutoff fib1  |Recursion Depth      3|...|Depth      6
  

  So it seems that when inlining functions, Truffle imposes a self-recursion depth limit, and using mutual recursion roughly doubles the actual inlining depth, potentially making the code more compact and performant. I am curious if V8 does the same.

In complex programs, instead of emitting deoptimization instructions for every operation, the compiler very often can remove many of these slow paths: it can deduce that, for example, “we’ve had a deoptimization above, so this object must be an integer”. And that will make the code more compact.

Here are some more benchmarks on cons lists:

• bubble: bubble sorting; pcase: simple cons pcase, inclist: increment list members by one
• type-hints: nativecomp (safety 0) (where signaling path is removed), unsupported by Juicemacs (signaling pathway (Java exceptions) preserved)

Here are some more benchmarks on floats and bignums:

• Floating point intensive: mandelbrot, nbody, inclist-th-float
• Bignum intensive: inclist-th-bignum, pidigits (GMP v.s. JDK BigInteger)
• Are slow paths frequent?

Here are some more benchmarks on closures and function calls:

The following is what I hope these benchmarks to provide insight into. But, because Emacs nativecomp might get bottlenecked by other operations, the analysis does not hold.

Direct calls

fibn-rec

Indirect calls via funcall/apply

flet, fibn-rec-advice

• Advices are not (yet?) native-compiled.

Indirect calls via mapcar/...

map-closure, inclist-...

But the principle is that if Juicemacs is not much faster than nativecomp, then there is a bottleneck to be solved. (Let’s hope boxing also explains the slowdown in these inclist-mapcar benchmarks.)

By the way, if you want to run the benchmarks yourself with Juicemacs, you will need some workarounds to avoid calling Org-mode and unimplemented functions. Refer to ELispBenchmarksTest.java for ELisp code and comments.