Turn your .emacs.d Into an Emacs Distribution (with Straight.el)

[TL;DR: skip to the steps]

I finally decided to follow the recent trend towards using straight.el for package management instead of the built-in package.el. I made the switch because there was community interest in a package I’m working on (after my talk about it at EmacsConf), but which isn’t yet on a package archive such as MELPA. I wanted to make the package available to install in a convenient way, but without the level of commitment to backwards compatibility of design that a MELPA release might warrant. There was also the possibility that some folks might just want to use my Emacs configuration wholesale, and I wanted to support that possibility as well. Straight.el seemed like just the ticket.

Straight.el bills itself as a way to manage Emacs packages that’s declarative, explicit and reproducible. As someone who has attempted to set up new Emacs users with my own .emacs.d in the past, I’ve run into some of the issues that straight.el takes aim at, including implicit or “magical” config not present anywhere in your init files. Although there are ways to mitigate this using use-package (such as remembering to include explicit use-package declarations for any packages you install locally, or using :ensure t to install packages when they are missing), these still cannot guarantee 100% reproducibility. In a sense there’s a big difference between 99% and 100% — it’s the difference between uncertain and certain.

Fully reproducible configuration is compelling for reasons beyond just convenience, since an .emacs.d configuration that’s fully reproducible could be treated as a flavor or distribution of Emacs in its own right, on the same playing field as Spacemacs, Doom Emacs, Prelude, Scimax, and others (except without the dedicated fans!).

So if you want to turn your .emacs.d into a “flavor” of Emacs that can be used by others, whether that’s people in a certain specialized field, or friends that you’ve introduced to Emacs for whom your config would be a convenient starting point, or people following along in a blog tutorial that you’ve prepared, then read on for a reliable way to do this using straight.el.

First of all, this post is excellent. Start there, follow the directions, and you’re almost done, especially if you’ve already been diligent about using use-package. Then follow these steps:

1. If you’re using a recent version of Emacs (>= 27), you’ll need to manually disable package.el (in previous versions of Emacs, straight was able to do this automatically). You will want to do this since it could otherwise lead to undefined behavior; in particular, implicit locally-installed packages (e.g. in the /elpa folder) may allow your local Emacs to work, without your configuration actually being reproducible. To disable package.el, edit (or create) the ~/.emacs.d/early-init.el file and add this line to it:

(setq package-enable-at-startup nil)

2. For any local .el files that you have somewhere in your “load path” that are to be included in your config:
   • If these are simply modules grouping related configuration (e.g. if they contain use-package declarations), then just load these modules directly.
     (load "my-module-name.el")
     
   • On the other hand, if they provide distinct functionality on their own (e.g. third party modules that you downloaded at some point), then these should be treated as individual package repositories for the purposes of straight.el. Here are the instructions on how to do that. Essentially, move every such file into a new subdirectory of the same name, and then use:

(use-package my-package-name
 :straight
 (my-package-name
  :local-repo "~/.emacs.d/path/to/my-package"
  :type nil))

Note that such packages should be committed into your .emacs.d git repo if they are to be part of your “distribution.”

3. For any packages for which you want to use a local Git repository (e.g. if you’re actively developing these packages, or if you’re using a custom version including your own modifications), use:

(use-package my-package-name
 :straight
 (my-package-name
  :local-repo "~/.emacs.d/path/to/my-package"
  :type git))

These files are present in their own git repositories, so do not commit these into your .emacs.d repo.

4. If you’ve got a package installed whose “feature name” happens to be different from its “package name” (this is rare, but I encountered this case with TeX/auctex), you’ll need something like this:

(use-package tex
 :defer t
 :straight auctex)

5. Restart Emacs, adding any missing use-package declarations as needed until it works.

Now commit everything and you should hopefully be in good shape. Except for one thing: any local repos you’re using (item 3 above) are not guaranteed to be (and likely won’t be) present on any system other than your own [if this doesn’t apply to you, skip ahead to step 8]. To truly turn your .emacs.d into a distribution, you’ll need to address this. On the one hand, you’d like to continue to use the local repos on your system for ease of development. On the other, you’d like your users to be able to use your configuration out of the box without any extra steps.

Here’s what I recommend: create a separate git branch in your .emacs.d repo as the “public” version of your config. In this branch, all Elisp repos in step 3 will point to the public versions of those repos instead of the ones locally on your system.

6. Create the new “public” git branch

git checkout -b public

7. Change all use-package declarations in step 3 (not step 2, leave those as they are) to look more like this:

(use-package my-package
 :straight
 (my-package
  :type git
  :host github
  :repo "my-github-name/my-package"))

In other words, point them to publicly visible repositories (e.g. hosted on GitHub or GitLab). Commit these changes into the public branch and push it to a public location. Now you can use the master branch on your system but your users can use the public branch, and everyone is happy. But this flexibility will take some maintenance on your part.

First, you will need to periodically rebase public onto master to keep it up to date. When you do, be sure to also add fresh commits to change any new use-package declarations matching Step 3.

Second, as you yourself will be using the main branch, for these packages in Step 3, your config is only guaranteed to be compatible with your local development repos rather than the publicly-visible (e.g. MELPA) ones you use on the public branch. If there are upstream changes in these packages, your config could still fall out of sync with those changes. To address this, you can “freeze” the versions on the public branch each time you do the rebase, via M-x straight-freeze-versions. This creates a new file in .emacs.d/straight/versions containing the exact versions currently in use, which Straight will honor if it is present — guaranteeing that your config is exactly reproducible and will always work in the future. Just don’t forget to commit this file into your public branch! You could use the following .gitignore config to allow you to whitelist the versions folder while still ignoring the rest of the straight folder:

straight/*
!straight/versions

There’s just one other thing:

8. If you have any packages that have external dependencies that you installed outside of Emacs, you should defer or even disable these packages in the public branch, because otherwise any config at init time that relies on these dependencies being present would fail. If this is a major part of your configuration, you might consider using a system-wide package manager like Nix or Guix to manage these external dependencies. While these come with benefits of their own, this option would require that your users install these as well, and as a result relying on them may mean that your distribution would not be platform-independent [1].

[1] As of this writing, Nix is available for GNU/Linux and Mac OS, while Guix is available on GNU/Linux.

(use-package my-package
 :disabled t)

And you’re done, you’ve just created your very own flavor of Emacs. Watch out, Henrik Lissner!

On a side note, as you’re doing all this you may be wondering, if it’s so easy to create your own Emacs distribution, and equally as easy to publicly host your own packages, what does that mean for package repositories like MELPA? Do we even need them?

Having published a package on MELPA in the past, one thing I can safely say is that the quality of packages on MELPA (and of course, ELPA) in general is likely to be high. I received a lot of very useful feedback in the code review process, and learned a great deal about the package development ecosystem in the process of making the submission. I think both the value provided to developers (expertise), as well as the value provided to users (quality) are why we will continue to need something like MELPA, perhaps always. Having more lightweight package distribution options with something like straight.el serves a different, and I think, complementary need.

For reference, here’s my .emac.., I mean, my flavor of Emacs. What’s yours?