Nix and NixOS for Cloud Infrastructure: Reproducible Builds, Declarative Configuration, and the End of Environment Drift

Somewhere around year eight of my career I had a moment of clarity about how most infrastructure actually works: it doesn’t. Not reliably, anyway. A developer opens a pull request, CI passes on the build server, the artifact gets deployed to staging, and then staging produces slightly different behavior than production because the two environments have diverged in ways nobody fully tracked. The Chef cookbook that provisions production was last modified two years ago. The apt packages on the staging VM are three minor versions newer than production’s. There’s a Python library pinned to 3.9.7 in the requirements file but the system Python is 3.11.2 because someone ran an unattended upgrade.

We spent years papering over this with increasingly complex tooling. Docker helped: at least the container image pins the userspace. But the Dockerfile itself is often not reproducible, because apt-get install pulls whatever package version is current the day the image is built. Infrastructure as Code tools like Terraform describe what cloud resources to create, but not the software that runs on them. Ansible is better, but a playbook is a sequence of imperative mutations, not a description of a desired state with mathematical guarantees.

Nix solves the reproducibility problem at a fundamental level. After twenty years of working on production infrastructure, I consider it one of the most interesting developments in systems software in the last decade, even if most of the cloud world hasn’t caught up yet.

What Nix Actually Is

Nix is three things that share a name, which causes enormous confusion:

First, Nix is a package manager that works on Linux and macOS. You can install it alongside your existing package manager. It keeps every package version it has ever installed in /nix/store, identified by a cryptographic hash of all its inputs, and never mutates them. Installing a new version of curl doesn’t remove the old one; it builds (or downloads) a new entry in the store alongside it.

Second, Nix is a functional programming language used to describe packages and system configuration. It’s lazy, pure, and somewhat peculiar. Every build is a pure function: given the same inputs, you get the same output, always. The inputs include source code, build scripts, compiler version, and every transitive dependency, all the way down to libc. This is what gives Nix its reproducibility guarantees.

Third, NixOS is a Linux distribution where the entire operating system, including kernel modules, system services, user accounts, and network configuration, is described in Nix. You write a configuration.nix file that looks roughly like this:

{ config, pkgs, ... }:
{
  services.nginx.enable = true;
  services.nginx.virtualHosts."example.com" = {
    forceSSL = true;
    enableACME = true;
    root = "/var/www/example";
  };

  services.postgresql = {
    enable = true;
    package = pkgs.postgresql_16;
    ensureDatabases = [ "myapp" ];
  };

  users.users.deploy = {
    isNormalUser = true;
    extraGroups = [ "wheel" ];
    openssh.authorizedKeys.keys = [ "ssh-ed25519 AAAA..." ];
  };
}

Running nixos-rebuild switch with this file atomically applies the configuration. It doesn’t apply changes sequentially and hope for no errors. It builds the entire new system closure, validates it, switches to it, and if something goes wrong you can roll back to the previous generation in seconds.

The Problem With Everything Else

Let me tell you about a failure mode I’ve seen at three different companies. The CI server builds a Docker image on Monday. The image is built with ubuntu:22.04 as the base. Tuesday, Ubuntu pushes a security update to libssl. Wednesday, you build the image again for a hotfix. The two images have different SSL behavior because they pulled different versions of the base layer. The tests pass because you’re not testing cryptographic behavior. The production deploy on Thursday reveals that the new libssl version changed a cipher default that your payment processor’s SDK was relying on.

With Nix, this cannot happen in the way I just described. When you use Nix to build a container image, you pin every dependency: not just the top-level packages, but every transitive dependency, by hash. If the hash changes, the build fails. If you want to update a dependency, you do it explicitly and the hash in your lockfile changes to reflect it. The build is a function; given the same lockfile, it produces the same output on any machine, on any day.

The existing approach to infrastructure drift detection tries to detect when your running infrastructure diverges from your IaC descriptions and alert you. This is valuable, but it’s reactive. Nix is proactive: it makes drift structurally impossible for the things it manages, because configuration is declarative and immutable.

Nix Flakes: The Modern Interface

For years, Nix had a rough reputation for global state: nix-channel would pin your package set to a channel URL, but channels were mutable, which undermined reproducibility slightly. Flakes, introduced experimentally around 2021 and widely adopted by 2025, solve this. A flake is a Nix project with a flake.nix file and a flake.lock file. The lock file records the exact commit hash of every external input.

Here’s what a development shell flake looks like:

{
  description = "myapp development environment";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
      in {
        devShells.default = pkgs.mkShell {
          buildInputs = with pkgs; [
            go_1_23
            golangci-lint
            kubectl
            helm
            terraform
            jq
          ];
        };
      });
}

Any engineer who checks out this repo and runs nix develop gets exactly those tool versions, bit-for-bit identical, regardless of what their host machine has installed. No asdf, no pyenv, no version managers fighting each other. No .tool-versions file that silently breaks when someone’s asdf doesn’t have the right plugin.

I’ve seen this pay off dramatically in cross-functional teams. A data engineer on a Mac and a platform engineer on NixOS can run the same build pipeline against the same toolchain without a single “it works on my machine” message in Slack.

The cloud development environments space is crowded with tools trying to solve this exact problem. Nix flakes solve it differently: rather than spinning up a remote VM or container, your local shell simply activates an isolated, pinned environment. It’s faster, cheaper, and works offline. The tradeoff is the Nix learning curve, which I’ll address honestly later.

Building Container Images With Nix

One of the most practical uses of Nix in infrastructure is building minimal, reproducible container images without Dockerfiles. The dockerTools.buildImage function in nixpkgs creates OCI images from a Nix closure. The result is a layered image containing exactly and only the packages you specify.

let
  pkgs = import <nixpkgs> {};
in pkgs.dockerTools.buildImage {
  name = "myapp";
  tag = "1.0";
  contents = with pkgs; [
    myapp
    cacert
    tzdata
  ];
  config = {
    Cmd = [ "/bin/myapp" ];
    ExposedPorts = { "8080/tcp" = {}; };
  };
}

The resulting image has no shell, no package manager, no libc beyond what myapp needs. It’s functionally equivalent to a distroless or Chainguard image, but built from first principles rather than maintained by a third party. And critically: given the same flake.lock, this build produces a byte-for-byte identical image every time it runs. The image digest is deterministic. Your CI doesn’t push a new image tag; it verifies that the image it built matches the hash in version control.

This has profound implications for software supply chain security. When your image digest is deterministic and derived from your source code hash, provenance attestation becomes much simpler. You can sign the image with Sigstore and the signature covers everything: not just the image you’re shipping, but every dependency it contains, back to their source code.

NixOS on Virtual Machines and Bare Metal

For teams running on bare metal, colocated hardware, or cloud VMs they manage directly (which remains common in compute-intensive workloads like GPU training and high-frequency data processing), NixOS is genuinely compelling.

The declarative configuration model means you describe your entire server in a .nix file. I’ve worked with teams that keep this file in the same repository as their application code. When you merge a PR that adds a new system service, the CI pipeline doesn’t run an Ansible playbook against the server. It builds the new NixOS system closure, validates it, and can even run it in a VM before the deploy. The actual deploy is a nixos-rebuild switch or a call to a deployment tool like Colmena or deploy-rs.

Colmena is what I use for managing small-to-medium fleets of NixOS machines. Your deployment file looks like this:

{
  meta = {
    nixpkgs = import <nixpkgs> {};
  };

  "web-01.example.com" = { pkgs, ... }: {
    imports = [ ./common.nix ./web.nix ];
    deployment.targetHost = "web-01.example.com";
    deployment.targetUser = "deploy";
  };

  "db-01.example.com" = { pkgs, ... }: {
    imports = [ ./common.nix ./postgres.nix ];
    deployment.targetHost = "db-01.example.com";
    deployment.buildOnTarget = true;
  };
}

Running colmena apply deploys all machines in parallel. Colmena builds each system closure locally, copies the result to the target, and activates it. If the activation fails, the machine remains on its previous generation. No partial deployments. No “I think it applied but I’m not sure.”

For teams managing a handful of servers running specific services, this is dramatically simpler than Ansible + a configuration management database + some drift detection tool. Compare this with the typical infrastructure drift detection setup that requires continuous reconciliation to catch the inevitable manual changes. With NixOS, manual changes to the running system don’t persist across a rebuild. The system converges to its declared configuration.

Nix in CI/CD Pipelines

The combination of Nix’s binary cache and deterministic builds makes CI/CD significantly more efficient. When every build input is content-addressed, Nix knows whether it has already computed a given derivation. If the inputs haven’t changed, it fetches the result from a cache rather than rebuilding.

You can host your own binary cache (using something like Cachix or a simple HTTP server) and share build results across your entire team and CI fleet. If developer A built the test dependencies yesterday, developer B’s machine fetches them from the cache instead of compiling from source. CI runners share the same cache. The result is that most CI runs don’t rebuild anything except the code that actually changed.

I’ve seen this cut CI times from twelve minutes to under three minutes in Go projects where the standard library and dependencies were being recompiled from scratch on every run. The savings compound: fewer compute cycles, shorter feedback loops, lower cost.

This integrates naturally with the CI/CD pipelines you’re already running. Nix doesn’t replace GitHub Actions or GitLab CI; it replaces the ad hoc setup steps inside those pipelines. The nix build command becomes your build step. The nix run command becomes your test step. The same commands work identically on a developer’s laptop and on the CI runner.

The Hard Truth About the Learning Curve

I’m going to be honest about something the Nix documentation often underplays: Nix is genuinely difficult to learn. The Nix language is a lazy, purely functional DSL that looks deceptively similar to JSON but behaves very differently. Error messages were historically cryptic, though they’ve improved significantly with the move to flakes and the flake-improved CLI.

I’ve seen engineers with ten years of infrastructure experience take two to four weeks to become productive with Nix. I’ve seen others abandon it after a week because the cognitive overhead didn’t seem worth the benefits. Both reactions are understandable.

The learning curve is concentrated in three areas. First, understanding the Nix expression language itself, particularly laziness and how attributes and functions compose. Second, understanding the Nix build model, derivations, and fixed-output derivations. Third, understanding how NixOS modules work, which is a separate layer on top of the core Nix concepts.

My recommendation is to adopt Nix incrementally. Start with nix develop for development environments before touching NixOS deployments. Get your team comfortable with flakes and devShells before you try to manage servers with Colmena. The productivity gains from reliable development environments are significant and relatively low-risk. Server management with NixOS is a bigger commitment.

When Nix Makes Sense

Nix is well-suited for teams that have already been burned by environment drift, where multiple engineers work on the same codebase with heterogeneous machines (some on macOS, some on Linux), and where reproducibility matters enough to justify the learning investment. Financial services, healthcare, and security-sensitive environments benefit most from the auditability: you can prove, with cryptographic certainty, exactly what software ran in production.

Nix is a better fit for teams managing a modest fleet of long-lived servers than for teams running everything on managed Kubernetes (where the node OS is someone else’s problem and you mostly care about container images). If you’re fully committed to Kubernetes and containers, the value proposition shifts: use Nix for building reproducible container images, but there’s less reason to adopt NixOS as your node OS when Talos Linux or Bottlerocket provide immutability at the Kubernetes layer without requiring you to learn the Nix ecosystem.

For teams already comfortable with functional programming and type systems, the Nix language feels natural. For teams that write primarily imperative Python or Bash, the adjustment is steeper.

NixOS AMIs and Cloud Integration

AWS has made NixOS easier to adopt with official AMIs in the Marketplace. You can launch a NixOS 25.05 instance, pull your configuration from a git repository, and run nixos-rebuild switch --flake github:yourorg/infra#hostname. The instance converges to its declared state in one command. The same configuration that runs in production can be tested locally in a NixOS VM or in a CI pipeline that spins up a temporary NixOS instance.

Google Cloud and Azure support NixOS through community-maintained images. The deployment story is slightly rougher than AWS but improving rapidly as enterprise adoption increases.

For teams using Infrastructure as Code tools like Terraform to provision cloud resources, Nix fits alongside rather than replacing them. Terraform creates the VM; NixOS configures everything that runs on it. This is a clean separation: Terraform handles cloud API resources (VPCs, security groups, DNS, load balancers), NixOS handles everything inside the compute boundary. You get reproducibility and auditability at both layers.

Rollbacks That Actually Work

The feature that converted me was the rollback model. On a traditional Linux system, rolling back a bad deployment means either restoring from a snapshot (slow, lossy) or undoing your Ansible playbook (error-prone, requires knowing exactly what changed). With NixOS, every nixos-rebuild switch creates a new boot entry in the bootloader. If the new configuration breaks networking or fails to start a critical service, you reboot, select the previous generation from the GRUB menu, and you’re back. The old system state is completely intact; nothing was mutated.

I’ve used this three times in production. Once when a PostgreSQL configuration change caused the service to fail to start. Once when an nginx module upgrade broke SSL renegotiation. Once when a custom kernel module for a proprietary storage card turned out to be incompatible with a new kernel version. In all three cases, the rollback was under five minutes from “this is broken” to “we’re running the previous version.” No restore from snapshot. No emergency Ansible run.

On a traditional system with zero-downtime deployment strategies you’d handle this at the application layer with blue-green or canary releases. NixOS complements those patterns at the OS layer: the infrastructure itself becomes as rollback-friendly as a containerized application.

Secrets and NixOS

One area where NixOS needs careful handling is secrets management. The naive approach of putting secrets in configuration.nix doesn’t work: since NixOS configurations are built into world-readable store paths, any secret you put in the config becomes visible to anyone who can read /nix/store. The community has converged on a few solutions.

sops-nix integrates with Mozilla’s SOPS (Secrets OPerationS) tool and PGP or age key encryption. Secrets are stored encrypted in your git repository and decrypted by sops-nix at activation time, placed in paths that are ephemeral and not in the store.

agenix uses age encryption with the machine’s SSH host key as the recipient. Simpler than sops-nix, works well for small teams.

For larger deployments, integrating with HashiCorp Vault or AWS Secrets Manager via a systemd service that fetches secrets at startup is more robust. The NixOS activation system can call arbitrary scripts during nixos-rebuild switch, so pulling secrets from your secrets manager before starting services is straightforward to implement.

Getting Started Without Going All-In

If you want to experiment with Nix without committing to NixOS, start here. Install the Nix package manager on your existing macOS or Linux system. Enable flakes in your Nix configuration (experimental-features = nix-command flakes in /etc/nix/nix.conf). Write a flake.nix for one of your projects that defines a devShell with the tools your project needs. Share it with your team. Run it for a month and see whether “it works differently on my machine” appears in your incident retrospectives.

If your team finds value in the development environment story, extend to building your container images with dockerTools.buildImage. Then, if you’re managing cloud VMs, consider converting one non-critical server to NixOS and managing it with Colmena. By the time you’ve done all three, you’ll have a solid intuition for when Nix’s reproducibility guarantees are worth the configuration investment.

The Bigger Picture

What I find most interesting about Nix is what it implies about how we think about infrastructure correctness. Most monitoring and observability practices assume that the running system is the source of truth and you need to continuously check it. Nix inverts this: the Nix expression is the source of truth, and the running system is guaranteed to match it (barring hardware failure). Observability becomes about understanding application behavior, not about detecting whether sshd_config has been modified.

This doesn’t eliminate the need for monitoring. You still need to know when your application throws 500 errors, when latency spikes, when a disk fills up. But it eliminates a whole class of infrastructure toil: the endless investigation of “why does this server behave differently from the others?” When every server runs from the same Nix expression, they’re identical by construction.

Twenty years of infrastructure work has given me a lot of opinions about tooling. Most tools improve at the margins. Nix changes the model. Whether that change is worth the learning curve depends on your team, your infrastructure, and how much pain environment drift has caused you. But if you’ve ever spent a Friday afternoon debugging a production incident that turned out to be a library version mismatch between staging and production, you already understand the problem Nix is solving. It’s just solving it in a way most of the industry hasn’t caught up to yet.