Developing using Visual Studio Code with Nix Flakes

One thing that irks me about Visual Studio Code is that, as an Electron app, there can only really be one instance of it running at a time. While this has its advantages, a notable downside of this is that the environment of a given Visual Studio Code window will essentially be inherited off of that of whatever already running instance you have, even if you started a new window from a different shell. Even if you wanted to run a new instance, you'd need a new profile, every time.

Can we have our cake and eat it too?

User Namespaces to the rescue?

Linux is really powerful. If you have user namespaces enabled, you can do some really cool stuff without needing root or setuid at all. Here's my attempt to write a script that will use user namespaces to make a temporary "fork" of our vscode profile, using overlayfs, that will only exist in RAM and disappear later. I call it, imaginatively, fcode:

#!/bin/sh
set -e

# Give ourselves a new namespace.
if ! [ $FCODE_UNSHARED ]; then
	FCODE_UNSHARED=1 exec unshare -rm "$0" "${@}"
fi

# Need real mount to avoid NixOS SUID wrapper
# https://github.com/NixOS/nixpkgs/issues/42117
REALMNT="/run/current-system/sw/bin/mount"

# Temporary dir to set everything up in
TMPROOT="$(mktemp -d -t fcodeXXXXXX)"

# Mount tmpfs to store overlayfs work files; makes things cleaner outside the namespace.
"$REALMNT" -t tmpfs tmpfs "${TMPROOT}"

# Make overlayfs subdirectories.
mkdir "${TMPROOT}/upper" "${TMPROOT}/work" "${TMPROOT}/mnt"

# Mount overlayfs. TODO: this does not escape $TMPROOT properly
"$REALMNT" -t overlay overlay -o "lowerdir=${HOME}/.config/Code,upperdir=${TMPROOT}/upper,workdir=${TMPROOT}/work" "$TMPROOT/mnt"

# Start vscode as a user. NOTE: chrome-sandbox still needs suid, so we need to disable it atm :\
unshare -U -- code -n --no-sandbox --user-data-dir "${TMPROOT}/mnt" "${@}"

Assuming you have user namespaces enabled, this should let you spawn an unlimited number of new Visual Studio Code windows! They still share extensions with each-other, and having a normal Code instance running in the background might cause funny things to happen, too :) This is possibly not a very good idea, but it does seem to work. What's the worst that could happen, anyways?

More Nix local development improvements

Firstly, if you're using Nix and you're not using flakes, I strongly recommend you start using flakes. At the very least, you really ought to go enable the nix-command and flakes features, at least until they are enabled by default.

Secondly, here is another wrapper, even more creatively named ncode, that wraps fcode into a flake's devShell:

#!/bin/sh
nix develop path://$PWD -c fcode "${@}"

This way, you can just type ncode . in a flake directory and get everything set up. In case you're wondering why I am using path://$PWD instead of ., well...

Developing in non-flake projects

Not everyone is drinking the Nix kool-aid (yet). This is generally not a huge issue, as you can often work on projects by using development shells of derivations directly in Nixpkgs with something like nix develop nixpkgs#dolphin-emu-beta. However, even though that does work, sometimes you'll want more: for example, you might want to pull in clang-tools so you can get a good clangd experience. In that case, you'll at least want some kind of flake setup in your workspace.

Crafting a flake.nix file for this is quite easy, and there's a lot of ways to do it. I'll show the somewhat complicated one from my current kio-extras workspace, right in the root of the git checkout:

{
  description = "kio-extras.";

  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.simpleFlake {
      inherit self nixpkgs;
      name = "kio-extras";
      overlay = final: prev: {
        kio-extras = rec {
          defaultPackage = final.libsForQt5.kio-extras.overrideAttrs (finalAttrs: prevAttrs: {
            src = self;
          });
          devShell = defaultPackage.overrideAttrs (finalAttrs: prevAttrs: {
            nativeBuildInputs = (prevAttrs.nativeBuildInputs or []) ++ [ final.clang-tools ];
          });
          testShell = final.mkShell {
            shellHook = ''
              export QT_PLUGIN_PATH="${final.lib.getBin defaultPackage}/${final.libsForQt5.qtbase.qtPluginPrefix}:$QT_PLUGIN_PATH"
            '';
          };
        };
      };
    };
}

It's a little more convoluted than it probably could be (maybe making a utility for this specific use case wouldn't be a bad idea either way, though) but the basic concept is quite simple: take your upstream nixpkgs package (in my case libsForQt5.kio-extras) and override src with self. Then, the devShell can just be that derivation, but overridden with additional nativeBuildInputs for your dev tools. Once you have a flake.nix file, you should be able to run nix build to make a build of the flake's default package, nix shell to get a shell with the flake's default package in the $PATH, and nix develop to get the devShell environment, for local incremental builds and dev tools. However, it probably won't work for you yet, because of one more thing.

I said I'd explain why I needed path://$PWD, now I will.

Git

Okay, so you did all this, but the nix command complains that it can't find flake.nix. An annoying (though admittedly helpful) feature of Nix flakes is that if you are working in a git repository, it will mask out untracked files. Unfortunately, this makes working on non-Nix native projects difficult. Thankfully, if we use path://$PWD in place of ., all of the Git-specific logic goes away and it will see all local files.

You may still wonder what to do about the fact that you have a bunch of untracked files in your working directory. You can add them to .gitignore, but then you need to commit that .gitignore upstream for it to really solve the problem.

Thankfully though, you don't! Git actually already has a solution for this. You can instead edit .git/info/exclude which is local-only and has the same exact effect as the root .gitignore. Cool! Here's mine from my local kio-extras:

# git ls-files --others --exclude-from=.git/info/exclude
# Lines that start with '#' are comments.
# For a project mostly in C, the following would be a good set of
# exclude patterns (uncomment them if you want to use them):
# *.[oa]
# *~
.build
compile_commands.json
flake.nix
flake.lock

You may be wondering why I put .build and .compile_commands.json in there. Well, ...

Clangd

I mentioned this already but one useful thing you can do is set up clangd in your devShell. Here's how I get clangd working in my environments for CMake projects:

# Set up a local CMake build directory.
cmake -B .build -S . -D CMAKE_EXPORT_COMPILE_COMMANDS=1

# Link compile_commands.json back into the cwd.
ln -s .build/compile_commands.json .

# Optional: Make a build. This is useful if your project has generated code, or for testing.
cmake --build .build -j$NIX_BUILD_CORES

Install the vscode clangd extension, and ensure that your development shell has clang-tools in it. You'll need to periodically re-run CMake configuration to get updated compile commands, but that's it: once you've done this, the setup should continue working without much maintenance.

There are other ways to generate a compile_commands.json file depending on your exact environment, though: it's not limited to just CMake projects. It'd be really nice if we could find a way to generate it from nix build somehow, so that it could be more generalized, but that seems like it could be a little tricky.

Once you have all of this set up, though, you should have a clean Git working directory, the ability to make builds directly against the build system, a working clangd setup, and a way to run multiple Visual Studio Code instances in separate windows.

Future Work

• I should probably improve on these shell scripts and make a NixOS module of some kind for this. Right now, it's just sitting in my user directory. Luckily, this is trivial to resolve with nix writers.

• Maybe look into whether this is the right way to use user namespaces. I'm really unsure that I really want unshare -U, but it does seem to work.

• Chrome sandbox workaround? It seems like the flatpak devs have a solution that's better than just passing --no-sandbox, can we use that here?

• Not having a way to version the local flakes is bad. Maybe putting the flake.nix directly in the upstream Git working directory is a bad idea after all.

Am I doing this right?

Of course, maybe Visual Studio Code is just a poor choice of text editor for Nix, but that aside, I do wonder what people think of this particular development setup. I've been crafting it on-and-off as I continue my long descent into using NixOS as my only OS, and admittedly I have no idea what anyone else is doing. Maybe there's a better way to do this. Maybe I've gone completely off the deep end and none of this makes any sense. I'm not really sure.

Either way, in the event that anyone actually sees this post, feel free to yell at me via angry Internet comments if I'm doing something stupid. You can also shoot an email to john@jchw.io, if you want, but I am horrible at responding to private correspondence, so don't take it personally.