Table of Contents
Below are notes for my Nix workshop at Installfest. https://pretalx.installfest.cz/installfest-2022/talk/QDTPQL/
1. Why Nix?
- Reproducible
- Build exactly the same software on different systems.
- Declarative
- Store your development environment in a simple file/git.
- “Reliable”
- Nix ensures that installing or upgrading one package cannot break other packages.
2. Basics
2.1. Commands
- Classic:
nix-build
,nix-shell
,nix-env
- older, incoherent, …
- Experimental:
nix
nix build
,nix develop
,nix profile
- needs to be explicitly enabled:
nix --experimental-features nix-command flake
- config file:
~/.config/nix/nix.conf
2.1.1. Package management
apt | Nix classic | Nix (flakes see later) |
---|---|---|
apt install mc | nix-env -iA nixpkgs.mc | nix profile install nixpkgs#mc |
dpkg -l | nix-env -q | nix profile list |
apt remove mc | nix-env -e mc | nix profile remove … |
apt upgrade mc | nix-env -uA nixpkgs.mc | nix profile upgrade .. |
- Features:
- No root required
- Installed into
~./nix-profile/bin
ls -l ~/.nix-profile ls -lH ~/.nix-profile
Updating
nix-channel --list # channel = package repository nix-channel --update # similar to apt update nix-env --upgrade # upgrade installed packages to channel version
2.1.2. Searching
apt | Nix classic | Nix (flakes see later) |
---|---|---|
apt-cache search … | nix-index + nix-locate | nix search nixpkgs … |
(3rd party tool) |
3. Under the hood
3.1. Nix store
ls -l /nix/store | head ls -lh $(which bash) tree $(dirname $(readlink $(which bash)))/.. | head
3.2. Profiles
https://nixos.org/manual/nix/stable/package-management/profiles.html
ls -l /nix/var/nix/profiles/per-user/$USER
- Easy to switch to older versions
nix-env --rollback nix-env --list-generations nix-env --switch-generation 43
Comparing generations (experimental nix)
nix profile diff-closures
4. Nixpkgs
- Collection of Nix expressions for building software
- Whole Linux distribution in one repository
- Ranks well in https://repology.org/ top repositories
- https://github.com/NixOS/nixpkgs/
- https://nixos.org/manual/nixpkgs/unstable/
4.1. Structure
pkgs
– packages (programs/libraries/…)nixos
– Linux distribution based on nixpkgslib
– helper functionspkgs/top-level/all-packages.nix
– list of top-level “attributes”Searching: search for “attrname<space>=”
cd nixpkgs grep -r 'gnuplot =' .
4.2. Building/Modifying packages
nix-build '<nixpkgs>' -A gnuplot git clone https://github.com/NixOS/nixpkgs/ nix-build ./nixpkgs -A gnuplot edit ./nixpkgs/pkgs/tools/graphics/gnuplot/default.nix nix-build ./nixpkgs -A gnuplot ls -l result nix log result/ # show build log nix log /nix/store/mr9l6qjimaarcak7s5rk6j1h4b16rpw9-gnuplot-5.4.3 # See the log from “distro” build nix log $(dirname $(readlink $(which bash)))/.. cd nixpkgs # Rebuild all packages depending on nixpkgs-review wip
4.2.1. Nix output monitor
Let’s have better understanding what happens during building:
NIXPKGS_ALLOW_UNFREE=1 \ nix-build $HOME/.cache/nixpkgs-review/rev-877f0e97eb2eff62e675b75dfe19940400b2ed09-dirty/build.nix \ --arg pkgs 'import <nixpkgs> {}' |& nom
Unfortunately does not work with experimental nix
command, but will
probably implement this in the future.
4.2.2. Examining dependencies
nix-tree ./result/
5. Development environments
Task | Nix classic | Nix |
---|---|---|
Temporary package installation | nix-shell -p | nix shell |
Enter build environment of a package | nix-shell | nix develop |
5.1. Install packages temporarily
mc --version echo $PATH echo $PATH | tr : \\n nix-shell -p mc mc --version echo $PATH | tr : \\n exit nix-shell --pure -p mc # only explicitly mentioned dependencies are available echo $PATH | tr : \\n exit
nix-shell -p mc -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/d1c3fea7ecbed758168787fe4e4a3157e52bc808.tar.gz
5.1.1. Experimental nix, flakes
- Discoverability, CLI completion
nix shell nixpkgs#gnuplot nix run nixpkgs#gnuplot
5.2. Existing package development/hacking
Traditionally:
./configure sudo apt install ... ./configure sudo apt install ... ./configure sudo apt install ...
With nix:
git clone https://github.com/MidnightCommander/mc.git cd mc ./autogen.sh nix-shell '<nixpkgs>' -A mc ./autogen.sh ./configure make -j$(nproc)
5.3. Per-project development environments
- One does not need to relay only on packages in nixpkgs
- Source can contain Nix files (e.g.
default.nix
,shell.nix
,flake.nix
)
git clone https://github.com/wentasah/boardproxy cd boardproxy make # fails due to missing dependencies nix-shell # make deps from shell.nix or default.nix available make # success
Just building the package:
nix-build
- Drawback: every single change in the source code rebuilds everything
5.3.1. Entering the environment automatically
- direnv, lorri, nix-direnv
Needs to be installed and configured (simpler on NixOS)
nix-env -i -A direnv -A lorri cd my-project lorri init # creates shell.nix template unless it exists direnv allow cd .. cd - edit shell.nix
6. How are Nix derivations built?
Nix expression is evaluated → low-level derivation
nix-build '<nixpkgs>' -A gnuplot ls -l result nix-store --query --deriver result/ nix show-derivation result/ # pretty format cat "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"
- Important attributes:
builder
,args
- Most attributes (and their values) are propagated to environment variables
nix-shell '<nixpkgs>' -A gnuplot printenv type genericBuild type buildPhase type configurePhase unpackPhase cd gnuplot-5.4.3 configurePhase buildPhase
- invalidace hash (aby se projevil update)
7. Generating Docker images from Nix packages
- https://nix.dev/tutorials/building-and-running-docker-images
https://nixos.org/manual/nixpkgs/unstable/#sec-pkgs-dockerTools
{ pkgs ? import <nixpkgs> {} }: pkgs.dockerTools.buildImage { name = "gnuplot"; config = { Cmd = [ "${pkgs.gnuplot}/bin/gnuplot" ]; }; }
7.1. Running
nix-build docker-image.nix docker load < result docker run -it gnuplot:nmg0lnkf1dl7a0bsv152a6xphkxm9sq9
7.2. What’s inside?
mkdir tmp && cd tmp
tar xf ../result
ls -lR
tar tf */layer.tar
7.3. Image from custom package
{ pkgs ? import <nixpkgs> {} , boardproxy ? import ./boardproxy/default.nix { inherit pkgs; } }: pkgs.dockerTools.buildImage { name = "boardproxy"; config = { Entrypoint = "${boardproxy}/bin/boardproxy"; }; }
nix-build docker-boardproxy.nix docker load < result docker run -it boardproxy:60j25jpwggspx1mg694143ddchsynq1q --help
7.4. Smaller image needed?
https://nixos.org/#asciinema-demo-cover
{ pkgs ? import <nixpkgs> {} }: pkgs.redis.overrideAttrs (old: { # no need for systemd support in our docker image makeFlags = old.makeFlags ++ ["USE_SYSTEMD=no"]; # build static binary with musl preBuild = '' makeFlagsArray=(PREFIX="$out" CC="${pkgs.musl.dev}/bin/musl-gcc -static" CFLAGS="-I${pkgs.musl.dev}/include" LDFLAGS="-L${pkgs.musl.dev}/lib") ''; # Let's remove some binaries which we don't need postInstall = "rm -f $out/bin/redis-{benchmark,check-*,cli}"; })
8. Flakes intro
8.1. Problems of clasic Nix
- What is inside a repo? Difficult to figure out without reading all *.nix files.
- No enforcment of naming .nix files (default.nix, shell.nix, …)
- What does
<nixpkgs>
mean? Depends on the value of $NIXPATH env. var. nix-env
operation depends on the content of~/.nix-defexpr/
.- Evaluation of Nix expressions can read arbitrary files on your disk. Harms reproducibility.
8.2. How do flakes solve it?
- Hermetic isolation of Nix evaluation (no NIXPATH, no arbitrary files).
- Only files from the same git repo can be read.
- => Makes it possible to cache results of evaluation.
- Defines a structure of repository “entry point” (flake.nix).
- Predefined attribute names for packages, overlays, tests, …
- Explicit references with versions to other sources, e.g. nixpkgs.
- lock files
8.3. Drawbacks of flakes
- One more thing to learn.
- Lower motivation to include software into nixpkgs (can be separate flake)
9. Other notes
- nixGL – running OpenGL apps from nixpkgs on other distros
- Remote builds
- slow laptop?
nix --builders ssh://server --max-jobs 0
- build on
server
, no local builds (except simple ones, i.e. runCommdnLocal)
- build on
10. Pros and cons
- Nix vs. Docker
- Docker solves problems of traditional package managers by building on top of them
- Nix changes (fixes) the core of package management
- Nix is (largely) reproducible
- Declarative specifications of build/development/production environments
- Nix is difficult to learn
- Using unpackaged applications (especially proprietary) is more difficult
- buildFHSUserEnv
- nix-ld
- autopatchelf-hook