This post was written for Day 8 of the Racket Advent Calendar. “#lang” is a Racket term referring to declaring the programming language in use, and this post uses Racket as a platform to discuss one approach to writing good programming languages. I’ve made an effort to explain the Racket-specific ideas as they are introduced, so if you are interested in language design but aren’t a Racketeer, I hope you’ll still read it and that you’ll find it interesting.
When a way of expressing a complex idea becomes small enough to fit in your head and doesn’t require special effort on your part to remember, it has, in a useful sense, become a language.
Take Racket’s module system, for example (which, just like those of other languages, allows you to group source code into units typically contained in files). By just writing “require” (like python’s import), we gain the ability to describe with great and effortless nuance in just what manner we mean to include those modules — whether to retain the names of the included definitions or change them in some particular way, or include or exclude particular definitions, and more. And these various ways are composable, so we could do, without any fuss:
(require (prefix-in r: (rename-in (only-in racket/list range)
[range ronge])))
(r:ronge 1 10 2) ;=> '(1 3 5 7 9)Of course we can do all these things. There’s no need to look up docs. This language fits in your head, and it expresses everything you might want to do when requiring modules. As Racketeers know, it’s hard to go back to other languages once you’ve been spoiled by Racket’s incredible module system!
Now compare that to another language facility, exceptions.
To raise an exception, in practice, we use one of these interfaces:
raise-argument-error
raise-arguments-error
raise-arity-error
...What is the syntax of raise-argument-error? It’s impossible to know — you just have to look it up each time. How do you define and raise your own exceptions? It’s not easy. Of course, contrary to what I said at the beginning, this is a language in a formal sense. It’s just not worth calling it one in a practical sense, if we are to mean anything useful by “language.” It doesn’t model all of the things you’d like to do with exceptions, and consequently, the syntax has no correspondence with the actual behavior you seek. It’s small but not in the right way, and it doesn’t fit in your head.
The good news is, just like with everything else, Racket (that is, the brilliant people behind it) takes some very farsighted decisions in the foundations of the language, so that you could build your own exception language using the basic primitives of raise (which can raise any value and not just what the host language decrees as “exceptions” — as far as I can tell, Racket’s built-in exceptions aren’t special in any way) and with-handlers (which supports arbitrary predicates and handler functions). Racket also provides every conceivable power tool for building arbitrary control structures, and a powerful object-oriented class system which could be used to model exception hierarchies. Now, if only someone would use all of these goodies to make a nice exception language that fits in our heads. 🤔 (I don’t got nuthin’. I wish I did!)
There are more such “facilities” for which we already have nice languages (like runtime contracts, pattern-matching), and others for which we’d like to have them (iteration, lazy semantics, and more), and I’d like to find out how it would feel to have them pervasively.
Now, it’s easy to get the wrong idea here, and think that a language necessarily means something very clever, if we recall the example of languages like Common Lisp’s LOOP (and, I think, some stylistically similar ones nowadays like, I am told, Cucumber). Those languages are powerful and expressive, but they present the appearance of human language-like fluency without actually exhibiting that level of flexibility. So they end up just feeling gratuitous and fragile, idiosyncratic languages you have to learn though they resemble something you already know. Yeah I know, shots fired. Ouch. Why are people stoning me??? I hath no sin in my heart!!!
(Disclaimer: I’ve never used either loop or cucumber! So if you feel this is a mistaken view, please feel free to stone, er, chime in in the comments.)
In any case, since we’re going boldly (and foolishly) down this road, please indulge me as I share some modest experiments I have actually done along these lines.
One of the main projects I work on in the Racket community is Qi, which is a language for expressing flow-oriented computations that is provided as an ordinary library. Like any such language (typically called a domain-specific language or DSL), the way you normally use it in your code is via a macro. Qi’s macro happens to be ☯, so that you can write, in Racket, (map (☯ (~> sqr add1)) (list 1 2 3 4 5)) to square each element in a list and add 1 to it, producing '(2 5 10 17 26). “map” here is just the usual higher-order function familiar from functional programming languages. This function accepts a function as its first argument, which will be used to transform the elements of the list (the second argument).
As we see here, instead of writing a name of a function in that second position or writing out a lambda inline, we use ☯ to describe the function using Qi, a language purpose-built for describing functions. We are able to do this even though the entire surrounding expression is a Racket expression. That is, we are embedding Qi into Racket here using a macro — a standard approach to DSLs in Racket and beyond.
Not long ago, Ben Knoble developed curlique (pronounced “curli-cue,” for non-native English speakers), a way to embed Qi into Racket syntactically without explicitly writing a macro form, so that something like this works:
(map {~> sqr add1} (list 1 2 3 4 5))(i.e. note the curly brackets instead of (☯ …)).
That got some of us in the community thinking about “#lang qi” — would it be useful to provide a top-level language (like Racket itself, which is indicated in source code via #lang racket. In the Racket ecosystem, even the Racket language isn’t special — by design!) that seamlessly supports flow-oriented expressions in this way?
And at that point, well, if we’re going to do that, why not address some of these issues regarding “small” languages that fit in your head? Racket is a big collection of raw materials for making languages. It’s not really (even trying to be) a cohesive language on its own. Why not make a cohesive “#lang qi”, if we’re going to consider making a #lang (pronounced “hash lang,” for Racket non-natives) at all?
Now, there is already an exciting language effort in the community that has a lot of momentum — Rhombus — and it is aiming to be a specific, cohesive, language (and not just raw materials). It also has syntactic capabilities that feel truly space-age to me. So maybe Qi could be embedded into Rhombus in some way? The thought has crossed my mind before. But as Rhombus is an infix language, it seemed like some of the nonlinear operators in Qi (like the tee junction, -<) would not be trivial to embed. It would certainly be worth attempting, but in the meantime, why not just start somewhere simple, and try to imagine a nice Racket-like #lang that you can build “in an afternoon,” which fits in your head, and naturally accommodates flow-oriented (Qi) expressions?
This is, perhaps, how the righteous lose their way. The power of macros proves too great, and mortals can’t resist the temptation to just fix every annoyance and make everything just so.
Like, take that map expression above. Always having to type (list ...) is annoying! Why, if only we could use box brackets there, that would be nice, wouldn’t it?
I had always assumed the reason Racket didn’t use box brackets for lists was to preserve the ambivalence about paren shape in other settings like let binding forms, where either (let ((a 1)) ...) or (let ([a 1]) ...) bind the variable a to the value 1, and the latter is favored for improved readability.
But Ben pointed out (without necessarily endorsing it) that by using syntax-parse‘s paren-shape parsing, together with Racket’s interposition points (which are really an under-sung superpower of Racket — the fact that even the most basic semantics of the language are facilitated, behind the scenes, using macros that we can override just like we can any other macro), we can have box brackets mean unquoted lists, so that this works:
(map {~> sqr add1} [1 2 3 4 5])… even while retaining ambivalence about bracket shape in binding and other settings:
(let ([a 3]
[b 5])
(map {~> sqr add1} [1 2 a 4 b]))Note that a and b above are variables. Usually in Lisp languages if we use '(1 2 a 4 b), called a “quoted list,” a and b would be elements of the list as symbols and not their values. But [1 2 a 4 b] evaluates a and b, which is usually what we mean.
Pretty cool, eh!
So, to explore things like this and make a language that, as Matz would say, maximizes my happiness, I started a “language experiments” repository to play with these ideas, to try and see if we can compose, from the high quality raw materials afforded us by the Racket platform, a nice language or two that “fit in your head,” and which are made up of even smaller languages for specific facilities like exceptions and iteration that each fit in your head. A small language made up of small languages!
There’s nothing special about these language experiments — #langs abound in the Racket ecosystem, and these particular ones are just variant compositions of raw materials already widely available in the community (or which can be created as needed and made available in this manner). Still, if this simple format inspires experimentation and the creation of useful languages for specific facilities (like exceptions!) for broad use, that would be a valuable outcome. Syntax space is a big place, and it’d be interesting to see what else we find there!
Speaking of the repo, here it is:
To give you an idea, it is set up so that each language is available in a separate indexed folder, named 1, 2, and so on. To try a particular language, just use #lang raqit/<index>, and there are some included examples to get you started.
Currently, language 1 has Clojure-like syntax (which proved quite unpopular when I brought it up at a Qi meeting a little while back!), and language 2 is more Rackety (and in fact a bit Rhombus-y). I’d like to explore generics more in the next language, whenever I get around to it.
So try ’em out! Make your own! And commit them back so others can try them too. And please, have fun! That’s the whole point.
Happy Day 8 of Advent of Racket!