Speeding Up Emacs Lisp: A Practical Guide to Native Compilation

Problem: The Hot‑Spot Bottleneck in Emacs Lisp

When you write a function that loops over thousands of elements, or a package that runs many small utilities on startup, the interpreter overhead of Emacs Lisp can become noticeable. Even a well‑written algorithm can feel sluggish if the interpreter has to fetch byte‑code, resolve symbols, and perform type checks on every call.

Thesis: Native Compilation Turns Lisp into Machine Code

Emacs 29 introduced a native compiler that transforms byte‑code into platform‑specific machine code. The result is a measurable drop in runtime overhead for “hot” functions – those called frequently or doing heavy computation. Benchmarks show speedups ranging from 2× to 10×, while the interpreter’s flexibility is preserved for the rest of the system.

What Is Native Compilation?

The native compiler operates on Emacs Lisp source files and produces a compiled file with the .eln extension (ELisp Native). When Emacs loads a file, it first looks for a matching .eln; if found, the machine code is linked and executed directly, bypassing the byte‑code interpreter. This process is transparent for built‑in functions that are compiled automatically, but custom packages require manual compilation or a hook.

Key Concepts

• Byte‑code: The normal intermediate representation that Emacs loads and interprets.
• Native frame: A compiled function that runs as machine code.
• ELn file: The compiled artifact that Emacs loads in place of the source.
• Async compilation: The compiler can run in the background to avoid blocking the UI.

Enabling Native Compilation

Adding a single line to your init file turns on async reporting of warnings and errors, which keeps the UI responsive during background compilation:

(setq native-comp-async-report-warnings-errors nil)

To compile a file manually, run:

M-x native-compile-file RET <path-to-file.el> RET

Replace <path-to-file.el> with the full path to your Lisp source. The compiler will generate file.eln next to the original. When you reload Emacs or evaluate the file again, the .eln will be used automatically.

Worked Example: Measuring the Gain

Suppose you have a utility that counts prime numbers up to n. Create prime-utils.el:

(defun count-primes (n)
  "Return the number of primes less than N.")
(defun is-prime (x)
  (cl-loop for i from 2 to (sqrt x) never (zerop (mod x i))))
(defun count-primes (n)
  (cl-loop for i from 2 below n
           when (is-prime i) sum 1))

First, benchmark the interpreted version:

(time (count-primes 20000))

Note the elapsed time in the echo area. Now compile the file:

(native-compile-file "~/.emacs.d/lisp/prime-utils.el")

Run the benchmark again:

(time (count-primes 20000))

A noticeable drop in the elapsed time indicates native compilation is active. Remember that the first run after compilation may still include the interpreter overhead; subsequent runs will use the compiled code.

Trade‑Offs and Limitations

• Startup Time: The compiler may add a few seconds to Emacs’ startup, especially if it compiles many large packages. The async flag mitigates this, but on slow disks the delay can be noticeable.
• Portability: Compiled .eln files are platform‑specific. Moving them between architectures or major Emacs releases can cause errors. It’s safest to compile on the target system.
• Debugging Complexity: Stack traces will show native frames, and breakpoints set in the source may not map cleanly. Debuggers like gdb or the built‑in edebug are less effective with native code.
• Unsupported Constructs: Dynamic code generation (e.g., eval) and certain byte‑code features fall back to the interpreter, so the speedup is limited to the compiled portions.
• Experimental Status: While stable in Emacs 29+, the native compiler is still considered experimental. Production use should be preceded by thorough testing.

Practical Checklist for Adopting Native Compilation

1. Ensure you’re running Emacs 29 or newer.
2. Add (setq native-comp-async-report-warnings-errors nil) to your init file.
3. Compile your most frequently used custom packages manually or via a hook.
4. Benchmark key functions before and after compilation.
5. Monitor startup time and memory usage in your workflow.
6. If you rely on debugging tools, verify that breakpoints still hit expected lines.
7. Recompile when you upgrade Emacs or move to a different architecture.

Native compilation is a powerful tool for squeezing extra performance out of Emacs Lisp. By following the steps above, you can evaluate its impact on your own code and decide whether the trade‑offs align with your productivity goals.