Introducing Mindstream for Emacs

Mindstream is a new Emacs extension for any kind of writing, including programming. It removes the barriers to starting, provides moment-to-moment versioning recording every change you make, and usefully organizes your work for you to consult or resume later.

When I introduced Mindstream a few months ago at a Racket meetup, I began by asking the following questions:

1. Where do you write?
2. How do you start coding?

That is, for the first question, things like drafting emails, blog posts, and so on. Where do you write those?

I assumed that most people use the Web UIs of services like GMail or WordPress, but was surprised to discover that the majority of attendees used their preferred editor (e.g., Vim or Emacs) for this locally, and then pasted the result into the web UI if needed.

For the second question: how do you start a new coding project, or try writing out a simple algorithm, or start experimenting with a new tool or library?

Some reported experimenting in a REPL and transitioning to saved files from there as needed. Others started in a “scratch buffer” in an editor like DrRacket, and then saved the result to a file if needed. Still others reported creating a temporary file at a standard location (like ~/work/sandbox/temp12.py) for the coding session and then moving it to a permanent location later if necessary.

How about you? How would you answer these?

Think about it first before reading further.

Interestingly enough, I was fairly confident that Mindstream solves the second problem well — that of providing a good experience for coding. But I felt that if most Emacs users didn’t already use Emacs to write emails and blog posts, then that might present an obstacle to their using Mindstream for writing tasks in general. But, it seems, that may not be a problem after all. In retrospect, of course Emacs users use Emacs for everything. I don’t even know why I was surprised!

In reflecting on how we carry out this basic daily workflow of writing, I want to draw attention to three stages of the experience and make some observations about them to motivate what Mindstream does for you. Let’s examine these three stages:

1. Before you start writing
2. During the act
3. After you finish

First, before you start writing, there are choices to be made: Where should I write? What should I name the file? Where should I put it? These choices represent hurdles, obstructions to setting pen to paper in service of the muse.

Then, during the act of writing, we need to be cautious so that we don’t accidentally lose work. We sometimes reach a point where things look good, but we want to experiment further and try different things. As we write, we have to constantly decide whether to save the file, how often to save it, whether to make a copy before trying a different direction, and so on, resulting in filenames like draft2, draft_final, draft_final_final1. These small, constant, choices contribute to anxiety.

Finally, after we’re done writing, we may like to return to our creations in the future, perhaps to consult them and refresh our understanding of some topic that we wrote about, or which we explored while programming. But when we look again at these files after a few weeks or months, we find them completely opaque. We no longer have any idea what the file draft2 contains, and we prefer not to worry about it. For coding explorations, if we were learning about a new idea, say, closures in lexically-scoped programming languages, we might save our progress in a file called “closures.py.” If we were learning about macros in Lisp-adjacent languages, we might name that file “macros.rkt”. These carefully curated examples are potentially valuable records, but reading these now to try and reconstruct our thought process from months or even years ago is not easy. It is sometimes easier to start afresh and just do it over again. Thus, these records of past writings often aren’t useful.

Which brings us to the conclusion:

From beginning to middle to end, every stage of the writing process on a computer exhibits a poor user experience.

In some respects, this is a regression from using plain old notebooks, where we start writing just by finding the last unused page and the nearest pen. We don’t have to worry about our work going up in flames because of one false move or the whims of capricious technology. And the entire record of our writing, including every intermediate stage, is preserved for posterity.

With all of modern technology, the experience of writing today should be unreservedly better than before, not worse! Yet, in these essential ways, as we can see, it really is worse.

And it is these essential problems that Mindstream seeks to address.

Mindstream is an Emacs extension that allows you to enter a writing session immediately. The session can be set up just the way you like it, starting from templates you provide. You needn’t worry about choosing a name for the session as it is taken care of by a natural and useful filing scheme. You also don’t ever need to worry about backing up different stages of your file, as the session is backed by Git, and it records a fresh version every time you save the file.

In short, it removes the hurdles so that you can begin writing right when inspiration strikes. It removes the anxiety about data loss by recording every change you make. And finally, if you decide the session is worth keeping around, you can give it a descriptive name at that stage, in retrospect. These saved sessions are much more useful than ordinary files, as you can easily step through your entire thought process from start to finish to understand how you got there, months or even years after the fact, without having to painstakingly parse through the polished version you got to at the end. And then, when you get there once more, you can pick up right where you left off, like pressing play on a paused video.

A good tool participates where it is needed and otherwise stays out of your way. There are already standard, powerful, time-tested ways to organize files (a filesystem), and standard, powerful ways to manage versioning (version control systems, especially Git). Mindstream relies on these and does not reinvent the wheel. It also does not interfere with major modes and configuration you already have for each specific writing task, which are fully available to you as they always are. When you use Mindstream to write, the result of your writings is standard files in standard folders versioned by standard Git repositories. As a result, you transparently retain the full power of these general tools. Mindstream itself leaves no persistent artifacts in your work.

So there you have it: lightweight stream-of-consciousness versioning for any writing task. Give it a try: you can install Mindstream today from MELPA and follow updates at the project repository.

May the muse forevermore remain at your side.

Although Mindstream is new in the sense of being recently announced (and this is the first blog post about it), it has been in development for a few years, and its design has benefited greatly from the contributions of early adopters. I especially want to recognize the help of Greg Hendershott, Noboru Ota, Stephen de Gabrielle, and user @dasoju on GitHub.