Nix: An Idea Whose Time Has Come

Today we explore Nix, a purely functional package manager.

Why Nix?

Joe Armstrong, one of the creators of Erlang, once described Erlang as the quest for programs that you “write once, run forever.” Nix, in comparison, might be the quest for programs that run wherever, whenever. Nix often scares newcomers and experienced devs alike, because it proposes a fairly radical overhaul to how we think about package management and how we run software in general. In this post, I’m going to illustrate which problems Nix solves and argue that this change in perspective has profound implications for software tooling.

Dealing with package managers is core to today’s developer experience, yet remains riddled with pitfalls. We tend to outsource our trust to a package cache, like NPM or Hex, and then incorporate whatever they return into our systems. Most of the time this works fine, as the package manager has incentives to act honestly. As an example of where this central point of failure becomes problematic, attackers can modify a package to include malware in what is known as a digital supply chain attack. What we really want, and what no package manager can give us today, is reproducibility.

By reproducibility, we mean the ability to recreate, bit for bit, the build of a package on your own system. Whatever package is thrown your way, you can rebuild it locally and check whether you have the expected build output. This is analogous to verifying a binary downloaded from the Web by computing its sha256 hash, and comparing it to the one provided by the publisher of the binary. In Nix’s case, however, you’re building the package from source. Why is this important? Because it removes the need to trust the provider and makes the cost of detecting a failure or an attack much lower for the developer. Reproducibility means security for your systems and applications, and peace of mind for your developers.

The way in which one gets to this reproducible state is by assuming nothing about the local environment. Everything in Nix must be declared explicitly by the user. Most build tools or environments rely on assumptions about the local environment and try to deal with it as best they can, but if you want to provide a way to build anything, anywhere, you have to forgo any tether to the local environment. Nix does this by forcing you to declare the packages you want, and how to fit them together. Because every part of your environment or build is explicit, you can achieve isolation and reproducibility.

Ok, But What Is Nix?

Nix is both a package manager–a source of prebuilt packages which one can download and run–and a functional language to help us to write “build expressions” in a reproducible way. A Nix expression is a function with one side-effect: creating the spec of the build itself.

“The main idea of the Nix approach is to store software components in isolation from each other in a central component store, under path names that contain cryptographic hashes of all inputs involved in building the component”
– Eelco Dolstra‘s Ph.D. thesis

A simple idea, with some deep implications. Let’s unpack it.

First, everything built with Nix gets put in a central store, by default `/nix/store`, which is a departure from your Linux FHS (Filesystem Hierarchy Standard). You know very little about a package installed on your system the traditional way: how was it built? What does it depend on? Nix makes package definitions self-contained.

Second, every built artifact is identified with a hash, which is the summary identifier of the complete dependency graph of the package, the platform-specific build steps involved, and the Nix expression which defines how to build it.
For example, my local version of “bat” v. 0.18.3, a popular cli replacement for “cat”, is stored under the following path: “/nix/store/fm6m39f7fr94bch5q1nssrrv6w6c8d2n-bat-0.18.3/bin/bat” with the following graph of dependencies.

This graph completely describes the flow of dependencies. Change the order, or change any dependency of bat, and the output hash changes, resulting in a new entry in the nix store. As far as Nix is concerned, it would be a totally new package. Since you know the hash of the output before you build it, you don’t need to backup packages themselves, just their build expressions. It means you can share the full spec of a local environment with a coworker, and know it will be executed exactly as it was run on your machine. It means you can have multiple environments, and multiple versions of a package, coexisting peacefully on your machine. And it gives you those assurances across languages and platforms: you gain a unified interface for packaging software, with everything you need to run it. Most of the time, it “just works”.

This alone is a welcome development, but another perhaps less appreciated benefit is that, while sharing software through space is easy today, sharing it through time is often much harder. For a made-up example, let’s imagine your company maintains a tool built 25 years ago with the C compiler from that time. It can’t be upgraded safely because the entire world’s banking system would fall apart if it did. Maybe you keep a dedicated laptop as a digital time capsule to maintain this tool. Instead, you can create a Nix shell with whatever version of the tools you need, pinned and never upgraded. This environment is isolated, and plays nice with all your other C versions on your system since it’s just another Nix package. You can share this environment with anybody and they’re one `nix-shell` command away from having the same environment as you.

“We build our computers the way we build our cities–over time, without a plan, on top of ruins.”
– Ellen Ullman

Nix undoubtedly has a steep learning curve, and there are two reasons for that. First, using Nix entails rethinking a lot of accepted wisdom and practices that have become second nature for a developer. Unlearning is hard. Second, Nix surfaces all the complexity involved in building software and forces you to deal with it. In order to make builds truly reproducible, this is largely inevitable. But once the “Nix way” is integrated, and the community tooling leveraged, your system becomes wonderfully transparent. No more chasing down configs and building tooling across your system and various languages. You gain a unified interface for dealing with building, composing, and sharing software artifacts across ecosystems. Nothing is hidden from you, and you can change whatever you like. In that sense, it belongs squarely in the Linux FOSS tradition.

Nix is out of the “science experiment phase,” but still evolving. However, many companies used Nix in production today, for example, Target, Replit, and Shopify. While the complexity of Nix can be daunting, I suspect we’ll look back in 20 years and wonder how we got along without it. In the meantime, enjoy being a few years ahead of the curve.

---

Languages Programming