The Animated Guide to Symex

[This is about Symex.el, the Emacs extension for structural editing with Lisp. If you are unfamiliar with Symex, you could read the post introducing it, or TL;DR: just know that “symex” is short for Lisp symbolic expression / S-expression, and this extension provides a full-featured modal interface / Evil state for editing Lisp code.]

This tutorial is modeled after the excellent Animated Guide to Paredit.

Using this Guide

If you are learning for the first time, you can use this as a tutorial and go through it sequentially. The Table of Contents below also doubles as a quick reference card, and you can click on any item to visit that section. The animations are short, typically lasting no more than a few seconds (knowing this may help you mentally bound the animations as you watch, since they cycle continuously). At the end of each animated section, you will find a ⬆ glyph. You can use this to return to the top of the page and the table of contents. The emoji 👉 signifies a tip or general insight.

Movement

The main thing about movement in Symex is that the “point” (i.e. cursor) always indicates a symex and never rests in arbitrary locations in the buffer. Motions move the cursor structurally, i.e. from one symex to another.

h l — Left and Right

These go left and right at a particular (tree) level, not in terms of how it looks on the screen, so for instance, they would also move across expressions at the same level that are aligned vertically. ⬆

k j — Up and Down

These aren’t very interesting, but the key thing is that they are structural motions with precise meaning. Also …

… with branch memory, the same keystrokes remember your positions as you go up and down. ⬆

f b — Forwards and Backwards

These traverse every element / expression in the tree. ⬆

C-f C-b — Traversing More Quickly

A faster way of getting around, these are like f and b but visit every third element instead of every element.

👉 The Control key sometimes (as in this case) functions as the comparative “more”, but at other times is a generic modifier[1]. ⬆

F B — Skipping forward and backwards

F – skip forward. This traverses forward like f but doesn’t enter nested expressions. It is a useful way to go forward without getting into the weeds, and often quicker than C-f.

B – skip backwards. ⬆

{ } — Leaping to Other Branches

Leap backwards / forwards to a neighboring branch, preserving position along the branch. The branches need not be adjacent, as long as a spot in the same position on the other branch can be found.

Leaping stays within the current tree (e.g. a function definition), but you can also “soar” across trees using M-{ and M-}. ⬆

gk gj — Coordinates-based Up and Down

Go up / down by buffer coordinates (not structural).

👉 Avoid using these as a crutch since e.g. leap branch is often more appropriate. ⬆

C-k C-j — Climbing and Descending

These motions can be useful in navigating unfamiliar trees where you don’t already have familiar positions that would be reached using j and k.

👉 Here, Control functions as the comparative “more.” ⬆

0 $ — First and Last

Alternatively M-h M-l, these go to the first or last symex at the present level.

👉 The Meta key sometimes (as in this case) functions as the superlative “most”, while at other times it is a simple modifier[1]. ⬆

M-j M-k — Lowest and Highest

Go to the lowest (root) or highest (most nested) symex in relation to the current position. M-k goes to the highest position accessible by ascending the current branch — it will not jump to neighboring branches that may be higher.

👉 Here, again, the Meta key functions as the superlative “most.” ⬆

Editing

Symex’s editing style is essentially Vim-like. While Vim’s Normal mode can be considered a Domain Specific Language for text editing in general, it has the following drawbacks when it comes to editing symexes: (1) it doesn’t have an appropriate noun to refer to symexes (e.g. it only has “words,” “lines,” “paragraphs,” etc.), and (2) the nouns it has are for the most part irrelevant when editing symexes. Thus, it isn’t exactly the right language to use. Symex.el attempts to be “the right language” here by supporting just a single noun — symexes — which, by virtue of its singularity, need not ever be stated and is implicit in every action[2]. But aside from this, Symex mimics Normal mode idioms and conventions. Therefore, a lot of these usage patterns should be very familiar from Vim/Evil even though they aren’t the same — they are the same ideas applied to symexes (along with other symex-specific ideas).

i — Insert

The behavior of i is sensitive to whether the symex is an atom or a form. For atoms, it inserts under the cursor. For forms, it inserts at the beginning of the contents of the form.

👉 In some cases you may want to just enter an insertion state at point without considering the context; for this, use <Enter>. But if your reason for doing this is to quote or unquote an expression, use the bindings in Quoting and Unquoting instead. ⬆

a — Append

The behavior of a is sensitive to whether the symex is an atom or a form. For atoms, it appends at the end of the atom. For forms, it appends at the end of the contents of the form. ⬆

I — Insert before

For forms, i inserts inside the form, while I inserts outside (before) the form. ⬆

A — Append after

For forms, a appends inside the form, while A appends outside (after) the form. ⬆

o O — Open Newline Below or Above

These open the newlines before or after the current expression, not the current line. They also enter an insertion state, just like their usual behavior in Normal state. ⬆

( [ — Create

👉 I find that I don’t use these very often since it’s just as easy to do I( as (i. ⬆

) ] - — Wrap and Splice

) and ] wrap the symex with the indicated delimiter. Contrastingly, - removes a wrapping level.

👉 I use these fairly often. “Splice” is a confusing (but standard) term since it sounds like “slice” which has the opposite meaning! Really what we are doing here is “splicing” the contents into the containing expression by “slicing” the delimiters using the sharp - key. Glad we cleared that up 😝. ⬆

w W — Wrap and Insert or Append

w is equivalent to )i, while capital W is equivalent to )a.

👉 I use these often. ⬆

x — Delete

You can also use capital X to delete backwards. ⬆

c — Change

⬆

C D Y — Change, Delete, or Yank Remaining Symexes

The above animation depicts C (change remaining). D and Y are analogous, respectively changing, deleting, or yanking (copying) the remaining symexes at the present level. ⬆

y p — Yank (Copy) and Paste

You can also use capital P to paste before. ⬆

- C-- — Splice and Clear

“Splice” removes the delimiters, and “clear” removes the contents.

We already saw “splice” earlier — it is just here again because I didn’t realize it was duplicated when I made this animation 😀 But also, it is useful to see its symmetry with “clear”. ⬆

s — Substitute

Similar to c but changes the contents and not the whole.

👉 This also works with strings. ⬆

S — Change Surrounding Delimiter

⬆

H L — Shift Left and Shift Right

I use these all the time.

👉 The Shift key usually functions as a verb, in this case to “translate” left and right, S-h S-l [1]. ⬆

K — Raise

Promote an expression by replacing the containing expression with it.

👉 Once again, the Shift key “translates” the symex in a direction, in this case, “up.”[3] Also see swallow which is similar and is more appropriate in some cases. ⬆

C-H C-J C-K C-L — Emit and Capture

Include or exclude expressions at either end of a form. These are also known as “barf” and “slurp” in paredit.

👉 Here, too, Shift functions as a verb. ⬆

z Z — Swallow Head / Swallow Tail

z consumes the head of the symex.

Capital Z consumes the tail of the symex.

In either case, the rest of the expression is promoted into the containing expression.

👉 I use these often. Also see raise which is similar and is more appropriate in some cases. ⬆

| & — Split and Merge

⬆

> < — Insert Newline / Join Previous Line

⬆

C-> C-< — Append Newline / Join Next Line

You can also use J to join next line, as usual in Vim. But note that these add and remove lines in relation to the current expression rather than the current line.

👉 The Control key functions as a modifier here. ⬆

M-< M-> — Collapse and Unfurl

Collapse the indicated symex to a single line, or unfurl its contents across multiple lines. You can also use C-M-< and C-M-> to collapse and unfurl the “remaining” symexes at the present level, which are often more useful.

👉 The Meta key functions as the superlative “most” here. ⬆

C-' C-, ` C-` — Quoting and Unquoting

C-' cycles through configured quoting prefixes (defaults to ' and `), while C-, cycles through configured unquoting prefixes (defaults: , and ,@). ` adds an extra quoting level and C-` removes a quoting level.

👉 This avoids the need to enter an insertion state to do a relatively common activity in Lisp (quoting). The quoting and unquoting styles can be customized. ⬆

M-H M-L — Shift “the Most”

Shift the expression to the left or to the right as far as possible. This stops each time at indentation boundaries. ⬆

<Tab> M-<Tab> — Indenting

<Tab> works by indenting the expression in the usual way for your major (e.g. Lisp) mode, but this doesn’t work when the form is badly indented in nested forms. In this case you can use M-<Tab> to traverse the indicated symex and indent it recursively. ⬆

; M-; — Commenting

; comments the indicated symex while M-; comments the remaining symexes at the present level. ⬆

Control

Symex also provides IDE-like features, but these don’t benefit from animated demonstrations, so I’ll just mention a few of them here:

e — Evaluate

There are many ways to evaluate code of interest — the most common are e to evaluate the symex at point, d to evaluate the containing definition, and E to evaluate the remaining symexes at the present level. ⬆

r — REPL

⬆

t — Scratch Buffer

Scratch buffers are a context-preserving alternative to working in a REPL that can be an efficient way to prototype things. Some examples of this pattern are Emacs’s Scratch Buffer for ELisp code, DreamPie for Python, and GroovyConsole for Groovy. I’m sure there are more — these are just the ones I’ve seen. In support of this style of prototyping, in Symex, t opens a scratch buffer in the appropriate Lisp mode. ⬆

M — Messages Buffer

Reviewing messages and logs is a common activity, making this a useful feature especially for ELisp development. ⬆

R — Run

Evaluates the entire buffer. ⬆

? — Lookup Documentation

Looks up the symbol at point using the Lisp mode’s documentation facilities. ⬆

H-h — Toggle Highlighting (beta)

Highlights the symex at point. As an implementation note, this feature uses Emacs’s region functionality at the moment but this tends to interfere with other functionality offered by Symex. It would be better to highlight using some kind of overlay eventually so that it can be seamless. ⬆

[1] The conventions around the Control, Meta, and Shift keys in Symex are inherited from the Rigpa framework.

[2] Another possible approach here could be to augment Normal mode with a symex noun bound to ( and ) in Lisp modes since “sentences” aren’t useful for editing symexes anyway. It has the other benefit that the text objects is and as could be read as “in symex” and “around symex.” This approach would allow Normal mode to be used for editing symexes without requiring an entire extra state, but it would mean that editing symexes specifically may be less efficient than in the approach taken in Symex.el (since the noun wouldn’t be implicit, e.g. c) vs just c). Still, this approach could be worth exploring as well, if anyone wants to take a crack at it.

[3] — Technically, this should be J rather than K since we’re translating the symex “down” the tree. But as J already has a standard binding in Vim/Evil, we use K here since we’re at least translating the symex “up” on the page. Of course in any case, all of these keys are user-customizable, and indeed, some users flip the orientation of j and k.