Nix: Yet Another Package Manager?

Nicolas B. Pierron

Nix: Yet Another Package Manager?

Nicolas B. Pierron
#NixOSParis 2015-06-04

Yet Another?

Xkcd
Creative Commons Attribution-NonCommercial 2.5 License

Managing dependencies

Tools:

  1. Package Manager: apt-get, yum, pacman, portage, …
  2. Build system: make, cmake, setup.py, …

Method:

  1. Read & Create a dependency graph
  2. Check for updates
  3. Download / Build the result
  4. Replace existing files

Program Analogy?

Package Manager Program
File System
Path
Package
Package Manager
Install
Remove
no such file or directory

Program Analogy

Package Manager Program
File System Memory
Path Pointer
Package Object
(mutable / immutable)
Package ManagerGarbage Collector
Install Allocate & Root
Remove Free / Lose Reference
no such file or directory Segmentation Fault

Nix Daemon: Properties

Package Manager Program
File System Memory
Path Pointer
Package Object
(mutable / immutable)
Package ManagerGarbage Collector
Install Allocate & Root
Remove Free / Lose Reference
no such file or directory Segmentation Fault

Nix Daemon: Properties

Some garbage collector properties:

  1. Conservative
  2. Language Independent

Conservative

Scan packages:

  1. Look for paths (hash)
  2. If a path is present, assumes a strong reference.
  3. Immutable packages, implies that we can cache references.

Language Independent


Nix

Guix

Listen on an UNIX socket:

  1. Add files
  2. Add recipes
  3. Build recipes

With languages, such as:

  1. The Nix Language
  2. Scheme (Guile VM)

Generated Build Recipes (.DRV files)

Derive([ ("out","/nix/store/hlf8kajdx1zb387d8z6a4811759jn3kg-python3-3.4.2","","")],
       [ ("/nix/store/2amb6nsgbjbimgdmclhn7xvwdppvb8sp-openssl-1.0.1m.drv",["out"]),
         ("/nix/store/3wffffp6k6j0scpxf58c04vy1kby8jx5-bzip2-1.0.6.drv",["out"]),
         ("/nix/store/6lzz188bxh3zvab1wrimhp0i465951kx-bash-4.3-p30.drv",["out"]),
         ("/nix/store/kdayv5zfvy5rp6q4zlc011bfpyfk6ifa-stdenv.drv",["out"]),
         ("/nix/store/njyjybqq70293y3i05yz0m257qhlgg4b-zlib-1.2.8.drv",["out"]),
         ("/nix/store/y0ypz4fc97yz0r22mbkp54vl3k29w0vg-Python-3.4.2.tar.xz.drv",["out"])],
       [ "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"],
       "x86_64-linux",
       "/nix/store/r5sxfcwq9324xvcd1z312kb9kkddqvld-bash-4.3-p30/bin/bash",
       [ "-e","/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"],
       [ ("NIX_LDFLAGS","-lgcc_s"),
         … … … … … … … … … … … … … …
         ("src","/nix/store/a8zb3r363n29lgl2mlmkpikqa4nf7x5w-Python-3.4.2.tar.xz"),
         ("stdenv","/nix/store/2xppxfdg15ra07izp3ww1ddqfywiwf05-stdenv")])

Output Path

Derive([ ("out","/nix/store/hlf8kajdx1zb387d8z6a4811759jn3kg-python3-3.4.2","","")],
       [ ("/nix/store/2amb6nsgbjbimgdmclhn7xvwdppvb8sp-openssl-1.0.1m.drv",["out"]),
         ("/nix/store/3wffffp6k6j0scpxf58c04vy1kby8jx5-bzip2-1.0.6.drv",["out"]),
         ("/nix/store/6lzz188bxh3zvab1wrimhp0i465951kx-bash-4.3-p30.drv",["out"]),
         ("/nix/store/kdayv5zfvy5rp6q4zlc011bfpyfk6ifa-stdenv.drv",["out"]),
         ("/nix/store/njyjybqq70293y3i05yz0m257qhlgg4b-zlib-1.2.8.drv",["out"]),
         ("/nix/store/y0ypz4fc97yz0r22mbkp54vl3k29w0vg-Python-3.4.2.tar.xz.drv",["out"])],
       [ "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"],
       "x86_64-linux",
       "/nix/store/r5sxfcwq9324xvcd1z312kb9kkddqvld-bash-4.3-p30/bin/bash",
       [ "-e","/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"],
       [ ("NIX_LDFLAGS","-lgcc_s"),
         … … … … … … … … … … … … … …
         ("src","/nix/store/a8zb3r363n29lgl2mlmkpikqa4nf7x5w-Python-3.4.2.tar.xz"),
         ("stdenv","/nix/store/2xppxfdg15ra07izp3ww1ddqfywiwf05-stdenv")])

Dependencies Substitution

Derive([ ("out","/nix/store/hlf8kajdx1zb387d8z6a4811759jn3kg-python3-3.4.2","","")],
       [ ("/nix/store/2amb6nsgbjbimgdmclhn7xvwdppvb8sp-openssl-1.0.1m.drv",["out"]),
         ("/nix/store/3wffffp6k6j0scpxf58c04vy1kby8jx5-bzip2-1.0.6.drv",["out"]),
         ("/nix/store/6lzz188bxh3zvab1wrimhp0i465951kx-bash-4.3-p30.drv",["out"]),
         ("/nix/store/kdayv5zfvy5rp6q4zlc011bfpyfk6ifa-stdenv.drv",["out"]),
         ("/nix/store/njyjybqq70293y3i05yz0m257qhlgg4b-zlib-1.2.8.drv",["out"]),
         ("/nix/store/y0ypz4fc97yz0r22mbkp54vl3k29w0vg-Python-3.4.2.tar.xz.drv",["out"])],
       [ "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"],
       "x86_64-linux",
       "/nix/store/r5sxfcwq9324xvcd1z312kb9kkddqvld-bash-4.3-p30/bin/bash",
       [ "-e","/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"],
       [ ("NIX_LDFLAGS","-lgcc_s"),
         … … … … … … … … … … … … … …
         ("src","/nix/store/a8zb3r363n29lgl2mlmkpikqa4nf7x5w-Python-3.4.2.tar.xz"),
         ("stdenv","/nix/store/2xppxfdg15ra07izp3ww1ddqfywiwf05-stdenv")])

Builder Independent

Derive([ ("out","/nix/store/hlf8kajdx1zb387d8z6a4811759jn3kg-python3-3.4.2","","")],
       [ ("/nix/store/2amb6nsgbjbimgdmclhn7xvwdppvb8sp-openssl-1.0.1m.drv",["out"]),
         ("/nix/store/3wffffp6k6j0scpxf58c04vy1kby8jx5-bzip2-1.0.6.drv",["out"]),
         ("/nix/store/6lzz188bxh3zvab1wrimhp0i465951kx-bash-4.3-p30.drv",["out"]),
         ("/nix/store/kdayv5zfvy5rp6q4zlc011bfpyfk6ifa-stdenv.drv",["out"]),
         ("/nix/store/njyjybqq70293y3i05yz0m257qhlgg4b-zlib-1.2.8.drv",["out"]),
         ("/nix/store/y0ypz4fc97yz0r22mbkp54vl3k29w0vg-Python-3.4.2.tar.xz.drv",["out"])],
       [ "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"],
       "x86_64-linux",
       "/nix/store/r5sxfcwq9324xvcd1z312kb9kkddqvld-bash-4.3-p30/bin/bash",
       [ "-e","/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"],
       [ ("NIX_LDFLAGS","-lgcc_s"),
         … … … … … … … … … … … … … …
         ("src","/nix/store/a8zb3r363n29lgl2mlmkpikqa4nf7x5w-Python-3.4.2.tar.xz"),
         ("stdenv","/nix/store/2xppxfdg15ra07izp3ww1ddqfywiwf05-stdenv")])

Nix: Managing dependencies

Method:

  1. Read & Create a dependency graph
  2. Check for updates
  3. Download / Build the result
  4. Replace existing files

Check for Updates

Immutable Paths:

  1. Is the output present?

Nix: Managing dependencies

Method:

  1. Read & Create a dependency graph
  2. Check for updates
  3. Download / Build the result
  4. Replace existing files

Substitute (Download)

Immutable Path & Predictable Hash:

  1. Collect output checksums (trusted remote)
  2. Download (untrusted remote / network)
  3. Verify checksums & signature

Download from:

  • Another computer
  • The web
  • A Peer-2-Peer network (maybe one day?)

Build

  1. Delegate to a remote
  2. Create a chroot
  3. Mount dependencies in the chroot
  4. Run the builder
  5. Scan for references

Nix: Managing dependencies

Method:

  1. Read & Create a dependency graph
  2. Check for updates
  3. Download / Build the result
  4. Replace existing files

Install

  1. Root the build result
  2. Update a symbolic-link (atomic)

Nix: Managing dependencies

Method:

  1. Read & Create a dependency graph
  2. Check for updates
  3. Download / Build the result
  4. Install new files