This is the multi-page printable view of this section. Click here to print.
nspawn manages systemd-nspawn machines the way docker manages containers: images come from an OCI registry (the hub), are stored as shared layers and are started, inspected and stopped through the D-Bus APIs of systemd-machined and systemd. The work is done by a service on the system bus and the command line is its client. This documentation describes nspawn 1.9.1.
If you are new, start with the overview and then get started. The command reference lists every command and option.
nspawn is a command line tool that manages systemd-nspawn machines with a docker-like workflow:
hub.nspawn.org,
where the nspawn.org team publishes images of common distributions built with
mkosi. Docker Hub and any other registry
work as well: search looks on the hub and on Docker Hub at once, and
login keeps credentials per registry, the way docker login does.systemd.mstack directory, or a flat copy. create makes more machines from
an image that is already local, without touching the registry.machinectl and importctl are never called, and
every machine is an ordinary systemd-nspawn@NAME.service unit that
machinectl, systemctl and journalctl see like any other. A drop-in makes that unit call nspawn before
the machine starts, once it runs and after it ends, so machinectl start, a
unit enabled at boot or a crash get the same network setup and cleanup as
nspawn start and nspawn stop.-p publishes
ports, -e sets variables, -v mounts host directories or named volumes,
--entrypoint and the arguments after -- change what an app runs, -l
labels it, --restart gives it a restart policy, -m, --cpus and
--pids-limit bound its resources, --health-cmd probes it, --secret
hands it a secret, and --hostname, -u, --cap-drop, --read-only,
--tmpfs, --device, --dns, --add-host, --ulimit and the rest of
docker run’s flags mean what they mean there.build runs mkosi on a directory with a mkosi.conf and imports the result
as a local image; push uploads an image to a registry, skipping the layers
that are already there.The work is done by a service on the system bus, org.nspawn, and the command
line is one of its clients, the way machinectl and systemctl are clients of
machined and systemd. The service is started by the bus when a command arrives
and exits when it has been idle for a while; nothing runs in the background
otherwise. See The service.
It is one binary written in Rust. Its own state lives under /var/lib/nspawn,
the assembled machines under /var/lib/machines, and everything it generates on
the host is a plain systemd unit, drop-in or .nspawn settings file that you can
read.
Every command is a method call on org.nspawn.Manager: images, machines, the
network and credentials are methods, the long operations (pull, push, build,
create, rm, images rm, cp, volume rm and prune, network rm and prune) come back
as job objects that report their output and result, and exec, logs,
events and an attached run hand their streams or terminal over the bus,
with a process object that says how they ended. Scripts that would rather not speak
D-Bus get the same dictionaries as JSON from inspect and --json on the
listings.
Who may call what is polkit’s answer, the way it is for machined and systemd. The bus lets everyone in and the service asks polkit about the caller, under two actions:
| Action | Methods |
|---|---|
org.nspawn.inspect | The ones that only read: images ls, ps and machines ls, inspect, top, stats, events, network ls and network inspect, volume ls, secret ls and secret inspect. |
org.nspawn.manage | Everything else: pulling, building, starting, stopping, restarting, pausing, removing, exec, shell, cp, logs, volumes, secrets, the registry commands and the credentials. |
Both are for administrators by default, so sudo nspawn ... works as it always
did, and a desktop session (or a terminal where you started pkttyagent) is
asked for a password instead. Root is never asked, which is also how the
service keeps working where polkit is not installed: there, nobody but root can
call it. One interface asks nothing: org.nspawn.Names hands out the names of
machines, images, networks and volumes, and nothing else about them, to any
user.
To drive nspawn without a password, an administrator hands an action to a group
in a rule of their own. The packages ship one as an example in their
documentation directory; copy it to /etc/polkit-1/rules.d/50-nspawn.rules:
polkit.addRule(function (action, subject) {
if (action.id.startsWith("org.nspawn.") && subject.isInGroup("wheel")) {
return polkit.Result.YES;
}
});
Everyone in that group can then run commands as root inside a machine and mount
any path of the host into one, which is to say they are administrators of the
host. Granting org.nspawn.inspect alone is the mild version: looking, without
touching.
A package installs the service; for a binary you built yourself,
sudo nspawn daemon --install writes the bus policy, the polkit actions, the
activation file and the unit, and tells systemd and the bus about them. Its journal is the usual one:
journalctl -u nspawn.service
nspawn tells two kinds of images apart when it installs them:
| Kind | What the image contains | How it runs |
|---|---|---|
| boot | An init system (systemd) and no other entrypoint, like the hub images | Booted with --boot, as machinectl start does. shell opens machined’s login session inside; exec enters the machine’s namespaces and needs nothing from it. |
| app | Anything else, for example an image from Docker Hub | Its entrypoint runs under nspawn’s stub init, with the environment, working directory, user and stop signal from the OCI config. Its network namespace is prepared by nspawn before the program starts, so the network is there from the first instruction. exec and shell enter the namespaces. |
The kind is detected at pull or build time and can be forced with
--mode boot or --mode app. Both kinds join the bridge network, are listed
by ps, stopped by stop and read by logs; the differences are described
in Machines.
Every machine joins nspawn0, a docker0 style bridge that nspawn manages
itself: fixed addresses, NAT, published ports (-p 8080:80) and a generated
/etc/hosts with the names of the other machines. network create makes
more networks like docker’s user-defined ones, a machine may join several of
them with names of its own on each, --network host shares the host’s network
instead, --network none gives a machine no network, and --network veth
gives booted machines the classic systemd-nspawn virtual ethernet pair
configured by systemd-networkd. See Networking.
| Piece | Where | Role |
|---|---|---|
nspawn | github.com/nspawn/nspawn | The tool this documentation is about: the service and its command line. |
| The hub | hub.nspawn.org | An OCI registry with the images the team publishes. It is the default registry of the tool. |
| mkosi definitions | github.com/nspawn/mkosi-definitions | The public mkosi configuration the hub images are built from. |
| Blog | blog.nspawn.org | Release notes and news. |
build needs
mkosi installed on the host.pull verifies one of those signatures before it downloads
anything (see Signed images). Every blob is
then checked against the sha256 digest in the manifest while it downloads,
and registries are reached over HTTPS only./run). cgroup v2 is
required.overlay backend, which is
the default and what app images always use (the layers are mounted with
lowerdir+=). Without it images are extracted as flat directories.ip and nft) for the bridge network. Nothing
else: the bridge does not need systemd-networkd or NetworkManager on the host.
Only --network veth needs systemd-networkd, and only --interface with a
wifi adapter needs iw (for an app on the bridge) or systemd 256 (for a
booted machine).sudo, which works everywhere, including hosts without polkit.shell, which uses machined’s login
session; the hub images have it. exec enters the machine’s namespaces and
needs nothing inside.A package brings the binary and the service it runs as. Every
release carries an RPM for Fedora,
a package for Arch, a .deb built on Ubuntu 24.04 that also fits later Debian
and Ubuntu releases, the plain binary in a tarball, and a SHA256SUMS over all
of them. On Arch Linux the AUR has two packages as well:
nspawn, built from the latest
release, and nspawn-git, from
the latest commit on master.
On Fedora or RHEL with SELinux enforcing, the nspawn-selinux package carries
the domain the service runs in; the RPM recommends it, so a plain dnf install
of the package brings it along. Without it the service runs unconfined and the
bus drops it when it passes a descriptor, which is what exec, shell,
logs and cp do: the rest of the commands work either way.
To build it yourself you need a Rust toolchain of version 1.85 or newer:
git clone https://github.com/nspawn/nspawn.git
cd nspawn
cargo build --release
sudo install -Dm755 target/release/nspawn /usr/local/bin/nspawn
sudo nspawn daemon --install
nspawn --version
daemon --install writes the bus policy, the polkit actions, the activation
file and the unit that let the bus start the service on demand, naming the
binary you just installed; a package does the same for you. Install the binary
in /usr/local/bin or /usr/bin, not below a home directory: the unit of every
machine calls nspawn by the path it was installed from, and on SELinux hosts a
system service is refused a binary under /home.
On a host with SELinux enforcing, a binary installed by hand keeps the label of
where it sits, and a service that is not nspawn_exec_t runs unconfined, so
exec, shell, logs and cp fail with the client disconnected. daemon --install
says so, and this is the fix:
sudo semanage fcontext -a -t nspawn_exec_t /usr/local/bin/nspawn
sudo restorecon -v /usr/local/bin/nspawn
The packages install into /usr/bin, which their policy already labels.
Find an image. search asks the hub and Docker Hub and prints, for every hit,
the reference pull takes:
sudo nspawn search fedora
SOURCE NAME DESCRIPTION STARS OFFICIAL
hub.nspawn.org fedora tags: 43, 44, latest, rawhide - -
hub.nspawn.org fedora-devel tags: 44, latest - -
Docker Hub docker.io/library/fedora Official Docker builds of Fedora 1300 yes
...
Pull one. References without a registry part go to the hub, and the local name
is derived from the reference unless you pass --name:
sudo nspawn pull fedora:44
hub.nspawn.org/fedora:44: signature verified (key 6wiWMtJZCUkV, keyless https://github.com/nspawn/mkosi-definitions/.github/workflows/mkosi.yml@refs/heads/master)
hub.nspawn.org/fedora:44: manifest 9240778b2c77 with 1 layer(s), assembling as overlay
blob f0c1e88e32b3: downloading
blob 28edf9a59c17: downloading
blob f0c1e88e32b3: downloaded
blob 28edf9a59c17: downloaded
image fedora-44 (boot image) is ready: nspawn start fedora-44
The image’s signature was checked first (every image on the hub is signed by
the workflow that builds it; see Signed images),
then the blobs went to /var/lib/nspawn, the root file system of the machine is
mounted at /var/lib/machines/fedora-44, /etc/systemd/nspawn/fedora-44.nspawn
holds the settings nspawn boots it with, and a drop-in of
systemd-nspawn@fedora-44.service makes the unit call nspawn around its life.
sudo nspawn start fedora-44
sudo nspawn ps
MACHINE IMAGE MODE COMMAND STATE UP PID NETWORK OS
fedora-44 hub.nspawn.org/fedora:44 boot init running 12s 48213 10.99.0.2 Fedora Linux 44 (Forty Four)
start returns once the machine’s own systemd is up, so a command can follow
right away. exec runs it inside and brings back its exit code; shell opens
a login session as root:
sudo nspawn exec fedora-44 -- systemctl is-system-running
sudo nspawn shell fedora-44
The hub images log in as root without a password on the console. What the
machine printed is in logs, and --inside reads the journal of the machine
itself:
sudo nspawn logs fedora-44
sudo nspawn logs fedora-44 --inside -n 50
Stop it and, when you no longer need it, remove it. Removing an image also frees the layers and blobs that no other image references:
sudo nspawn stop fedora-44
sudo nspawn images rm fedora-44
Images without an init system run as apps: the entrypoint from the image runs
under a stub init, on the bridge like any other machine, so -p publishes its
ports on the host:
sudo nspawn search nginx --source dockerhub
sudo nspawn pull docker.io/library/nginx:latest --name web
sudo nspawn start web -p 8080:80
curl -sI http://localhost:8080/ | head -1
sudo nspawn logs web -f
sudo nspawn stop web
sudo nspawn run -d docker.io/library/nginx:latest --name web -p 8080:80 does
the pull and the start in one step, like docker run -d. Without -d, run
stays attached as docker run does: sudo nspawn run -it --rm docker.io/library/alpine:3 sh gives you a shell in a machine that goes away
when you leave it. The port and any other flag given to start or run are
remembered, so the next sudo nspawn start web publishes it again. stop sends the image’s stop
signal (SIGQUIT for nginx) to the program and kills the machine after ten
seconds if it is still there; -t changes the grace period.
Docker Hub limits anonymous pulls per address. sudo nspawn login docker.io -u USER
keeps your credentials for that registry only; see
Registries and credentials.
create, where things are stored.start, run, stop, kill, exec,
shell, logs, ps, stats, events, entrypoints, environment and
volumes.build and push./etc/nspawn/nspawn.toml, environment
variables and flags.An image reference has the form [registry/]repository[:tag|@digest]:
hub.nspawn.org; the configuration page
shows how to change it. The first path component counts as a registry when it
looks like a host, for example docker.io/library/nginx or
registry.example:5000/team/app, or when it is the configured registry’s own
name, so a registry of the local network without a dot or a port (myhub)
reads back as itself.latest.., _, -
and /. Official Docker Hub images live under library/, so nginx is
docker.io/library/nginx; search prints references in that form.The local name of an image, the one start, stop, exec and friends
use, is derived from the reference (fedora:44 becomes fedora-44) unless
--name says otherwise. It has to be a valid machine name: letters, digits,
., _ and -, at most 64 characters, not starting with a dot.
The hub is a plain OCI registry. nspawn hub ls lists its repositories with
their tags, and nspawn hub tags REPOSITORY the tags of one of them:
sudo nspawn hub ls # everything
sudo nspawn hub ls deb # repositories whose name contains "deb"
sudo nspawn hub ls --no-tags # faster on big registries
sudo nspawn hub tags fedora
The same commands work against any registry that implements the catalog
endpoint; point them somewhere else with --registry, the NSPAWN_REGISTRY
variable or the configuration file. A private CA is trusted with --ca-cert
or NSPAWN_CA_CERT.
The images on the hub are built with mkosi from the public definitions in nspawn/mkosi-definitions, with the package manager of each distribution, and rebuilt every week so that they carry the current security updates. They come in three kinds:
| Kind | Images |
|---|---|
| Distributions | debian, ubuntu, fedora, centos, almalinux, rockylinux, opensuse, archlinux, kali, several releases each |
| Services | nginx, apache, caddy, haproxy, postgresql, mariadb, redis, valkey, memcached, rabbitmq, mosquitto, unbound, dnsmasq, bind9, samba, openssh, prometheus, node-exporter, grafana, wordpress, syncthing, forgejo |
| Machines to work in | debian-devel, ubuntu-devel, fedora-devel, archlinux-devel, python, nodejs, golang, rust, openjdk |
A service image is a whole Debian machine with that service installed and
enabled, so shell gets you a root shell and systemctl status nginx inside
tells you what it is doing. Its tags carry the version of the service
(nginx:1.26.3, nginx:1.26, nginx:latest), and every build also gets a
dated tag (nginx:1.26.3-20260923) that never moves, so a machine can be
pinned to one build. https://hub.nspawn.org/ browses the lot.
sudo nspawn search TERM [--source hub|dockerhub] [-n LIMIT]
search is docker search across the sources nspawn knows: the configured
hub (its catalog, case-insensitive substring match, with the tags of every
match) and Docker Hub (its search API). Every hit names its source and the
reference pull takes:
SOURCE NAME DESCRIPTION STARS OFFICIAL
hub.nspawn.org debian tags: 12, 13, bookworm, trixie - -
Docker Hub docker.io/library/debian Debian is a Linux distribution that's... 5100 yes
Docker Hub docker.io/someone/debian-x Debian with extras 3 -
--source asks one side only and -n limits the results per source (25 by
default). A source that cannot be reached is reported as a warning, not as an
error, so the other one still answers.
Registries are used anonymously until you log in:
sudo nspawn login # the hub
sudo nspawn login docker.io -u USER # Docker Hub
echo "$TOKEN" | sudo nspawn login registry.example -u ci --password-stdin
sudo nspawn logout docker.io
login asks for the user name and the password on the terminal (without
echo), or reads the password from standard input with --password-stdin, and
checks them the way docker login does: GET /v2/ on the registry, then basic
authentication or a token request at the realm the registry announces. Bad
credentials are refused with a clear message. A registry that never asks for
credentials gets them stored anyway, with a note saying so.
Credentials are kept in /etc/nspawn/auth.json, mode 0600, in the auth.json
format that podman and skopeo use. That file is the only one consulted: the
work happens in the service, which has no home directory of yours to look into,
so what docker login or podman login left behind does not carry over. Every
operation chooses the credentials of the registry it talks to, so the hub’s
never travel to Docker Hub; the many names of Docker Hub (docker.io,
index.docker.io, registry-1.docker.io) are one entry.
Docker Hub limits anonymous pulls per address; logging in lifts that. push
authenticates before it uploads anything and, when the registry wants
credentials it does not have, says which login to run.
Every image on the hub is signed twice by the workflow that builds it
(nspawn/mkosi-definitions) with
cosign: with the project’s key, whose public half (cosign.pub in that
repository) is built into nspawn, and keyless, with the workflow’s own identity
through Sigstore
(https://github.com/nspawn/mkosi-definitions/.github/workflows/mkosi.yml@refs/heads/master,
issued by GitHub). The signatures are referrers of the image on the registry:
Sigstore bundles that carry the certificate or the key’s hint, the transparency
log entry and a timestamp, so pull verifies them offline, with nothing but the
registry consulted, against the Sigstore trusted root the binary embeds. One of
the two has to verify, and the check comes before a single layer is downloaded:
$ sudo nspawn pull fedora:44
hub.nspawn.org/fedora:44: signature verified (key 6wiWMtJZCUkV, keyless https://github.com/nspawn/mkosi-definitions/.github/workflows/mkosi.yml@refs/heads/master)
hub.nspawn.org/fedora:44: manifest 9240778b2c77 with 1 layer(s), assembling as overlay
...
An image without a signature (a tag dated before the signing began, or one
pushed by hand with nspawn push) is refused with carries no signature, and
so is one whose signatures do not verify, with the reason of each. --no-verify
on pull, run and create skips the check for that command, like docker’s
--disable-content-trust; create checks only an image it has to pull, never
a local source. inspect shows who signed (signed_by: key <hint>, keyless <identity>) and when (signed_at), which a machine made with create
inherits from its source.
Images of other registries are pulled unverified until the configuration file says what they must carry, one table per registry; the same table can require other keys of the hub, or turn its check off:
[registries."hub.nspawn.org"]
verify = false
The policies belong to the service: --registry on the command line chooses
the registry, never the policy.
sudo nspawn pull REFERENCE [--name NAME] [--backend BACKEND] [--mode MODE] [--force] [--no-verify]
It resolves the reference to the manifest for the host’s platform (image indexes are followed), checks the image’s signatures where its registry has a policy (the hub’s is built in; see Signed images) before anything is downloaded, downloads every layer and the config blob that is not already in the store, three at a time as docker does, checking each one against its sha256 digest while it streams (a blob is written next to its final name and renamed only once verified, so an interrupted download never passes for a complete one, and a download cut short takes its part file with it), says of each blob when it is downloaded, and then:
/etc/systemd/nspawn/NAME.nspawn, the settings the machine boots
with, and the drop-in that makes systemd-nspawn@NAME.service call nspawn
around its life;/var/lib/nspawn, and keeps the manifest and blobs so that
the image can be pushed or cloned later.An image with the same name is not replaced unless you pass --force, and
never while its machine is running, starting or being restarted by its policy. The old image only goes once the new one
is downloaded, and the store is locked only for that last step, so a long
download holds up neither other commands nor the unit hooks.
The backend decides how the layers become a directory systemd-nspawn can boot:
| Backend | What it does | Needs |
|---|---|---|
overlay | Layers are extracted once under /var/lib/nspawn and shared. Each machine gets an overlayfs mount at /var/lib/machines/NAME, defined by a generated mount unit, with a private upper directory for its writes. | Any systemd with overlayfs. |
mstack (experimental) | A native systemd.mstack directory at /var/lib/machines/NAME.mstack that points at the shared layers; the machine boots with managed user namespaces (PrivateUsers=managed) and the layers are shifted into the foreign UID range. Only when asked for by name, with a note that it is experimental: systemd-nsresourced and systemd-mountfsd are still settling, and what the host’s systemd makes of a managed user namespace is its business (the 262 release cannot boot one at all until the fix for systemd/systemd#43899 lands in a later version). | systemd 261 or newer with the systemd-nsresourced and systemd-mountfsd sockets available (nspawn starts them). |
flat | The layers are extracted into a plain directory at /var/lib/machines/NAME. Nothing is shared, maximum compatibility. | Nothing. |
auto (default) | overlay when overlayfs is available, otherwise flat. Never mstack. |
Pick one per image with --backend on pull, build or create, or set a
default in the configuration file. nspawn images ls shows which backend each
image uses. On create, auto means the backend of the image it comes from,
not the chain above.
App images are always assembled as overlay (or flat), even where mstack
is available: a machine under managed user namespaces cannot join the network
namespace nspawn prepares for apps on the bridge. pull says so with a note
when it makes that choice.
After assembling, nspawn looks at the root file system and at the OCI config:
usr/lib/systemd/systemd,
lib/systemd/systemd, sbin/init or usr/sbin/init, as the topmost layer
leaves it) and whose entrypoint is that init, or has no entrypoint at all, is
a boot image. It is started with Boot=yes, as machinectl start would.Boot=no, ProcessTwo=yes), with the
config’s environment, working directory, user and stop signal.--mode boot or --mode app overrides the detection. The mode is recorded with
the image; Machines explains how it changes start, exec,
shell and stop.
sudo nspawn create SOURCE NAME [--backend BACKEND] [--network NETWORK]... [-p [IP:]HOST:CONTAINER[/udp]]...
[--entrypoint PROGRAM] [-e VAR[=VALUE]]... [-v SOURCE:TARGET[:ro]]... [-l KEY=VALUE]...
[--restart POLICY] [-m SIZE] [--cpus N] [--pids-limit N] [OPTIONS OF start] [-f] [-- ARGUMENTS...]
create is docker create: another machine from an image that is already
local, given by its name or by the reference it was pulled from, without
touching the registry. A reference that is not local is pulled first, under
the image’s own name, as run does. The machine shares the source’s layers
and gets a writable layer, an address, settings and a record of its own, with
create as its origin; images rm of one never affects the others.
The network kind is inherited from the source unless --network says
otherwise; published ports are not, since two machines cannot publish the same
one, and neither are the source’s labels, restart policy, limits, variables,
volumes, healthcheck, secrets or the other flags: a new machine only gets what
its own command line gives. Every option of start is accepted and means the
same; --entrypoint, -e and the arguments after -- apply to app images
only, -v works for both kinds. Everything on the command line is checked
before anything is made, so a refused create leaves nothing behind.
pull --name ends up the same way, but resolves the manifest through the
registry first.
sudo nspawn images ls
NAME TYPE BACKEND ORIGIN SOURCE SIZE RO
fedora-44 directory overlay pull hub.nspawn.org/fedora:44 612 MiB no
db directory overlay create hub.nspawn.org/fedora:44 612 MiB no
web directory overlay pull docker.io/library/nginx:latest 190 MiB no
old-arch directory - - - 1.2 GiB no
The list comes from systemd-machined, so it includes images nspawn did not
create (old-arch above, with - in the nspawn columns). ORIGIN is pull,
build or create.
sudo nspawn images rm NAME...
images rm refuses to remove the image of a running machine, or of one its
restart policy is bringing back; nspawn rm -f stops it first. A pulled image
is a machine too, so nspawn rm NAME removes the same thing.
For an image nspawn manages it unmounts and deletes the assembled root, the
settings file, the generated units and drop-ins, the network namespace of an
app and the record; for any other image, or for the leftovers of a failed
install, it removes what it finds and asks machined to forget the rest.
Afterwards the layers and blobs that no remaining image references are
deleted, the /etc/hosts files of the machines on the bridge are regenerated
and the published ports are brought in line with the machines that still run.
Named volumes are not deleted: they may belong to another machine. nspawn volume ls shows them and volume rm or volume prune removes the unused ones.
images ls --json prints the list as the service returns it.
| Path | Contents |
|---|---|
/var/lib/machines/NAME | The root of a machine with the overlay or flat backend (a mount point in the first case). |
/var/lib/machines/NAME.mstack | The systemd.mstack directory of a machine with the mstack backend. |
/var/lib/nspawn/ | nspawn’s own state: blobs as downloaded, extracted layers, image records, stored manifests, the private upper directories of overlay machines, the generated network files and units of each machine, mkosi build output and cache, and the store lock. Root’s alone (0700), but for the generated files an mstack machine binds from inside its user namespace; the directory itself can only be passed through (0711). |
/var/lib/nspawn/volumes/NAME | A named volume (-v NAME:/inside). |
/etc/systemd/nspawn/NAME.nspawn | The settings nspawn generates for a machine; regenerated at every start, mode 0600 since it carries the -e variables. |
/etc/systemd/system/systemd-nspawn@NAME.service.d/ | The drop-in with the unit hooks (and the machine’s Restart=, MemoryMax=, MemorySwapMax=, CPUQuota= and TasksMax= when it has them) and, for an overlay machine, the one that requires its mount unit. The mount unit itself is next to them in /etc/systemd/system/. |
/etc/systemd/system/machines.target.wants/systemd-nspawn@NAME.service | The boot link of a machine with --restart always or unless-stopped; rm removes it. |
/etc/nspawn/nspawn.toml | The configuration file, optional. |
/usr/lib/systemd/system/nspawn.service, /usr/share/dbus-1/system.d/org.nspawn.conf, /usr/share/polkit-1/actions/org.nspawn.policy | The service, its bus policy and its polkit actions, as a package installs them. nspawn daemon --install writes the unit and the bus policy under /etc instead, where they take precedence, and the actions in the same place as the packages. |
/etc/nspawn/auth.json | The credentials login stored, mode 0600. |
/run/netns/nspawn-NAME | The network namespace of a running app machine on the bridge. |
The state is kept outside /var/lib/machines on purpose: machined would list
a directory there as an image, and machinectl clean would delete it.
Every one of these commands is a call to the
service on the system bus, which asks polkit
whether you may: sudo always works, and a rule can let your group through
without a password. The examples here use sudo.
Every machine nspawn starts is the systemd unit systemd-nspawn@NAME.service,
registered with systemd-machined under its name. machinectl list,
machinectl status NAME, systemctl status systemd-nspawn@NAME and
journalctl -u systemd-nspawn@NAME (with --namespace=nspawn for what the
machine prints, see logs) all work on it; nspawn adds the
docker-like commands on top.
The unit also carries a drop-in, nspawn-hooks.conf, that calls nspawn around
the machine’s life: ExecStartPre prepares its network and settings,
ExecStartPost publishes its ports once it is registered, and ExecStopPost
releases everything however the machine ended. So machinectl start NAME,
systemctl enable systemd-nspawn@NAME for a machine that comes up at boot, a
program that exits on its own or a crash all behave like nspawn start and
nspawn stop. The drop-in names the nspawn binary that wrote it, which is why
nspawn belongs in /usr/local/bin or /usr/bin. It also carries the machine’s
restart policy and resource limits, when it has them, and for an app machine
the ExecStart= that runs systemd-nspawn through nspawn attach-exec, which
is how run -i and run -t hand the program an input or a terminal.
sudo nspawn start NAME [--network NETWORK]... [--network-alias [NETWORK=]NAME]...
[-p [IP:]HOST:CONTAINER[/udp]]... [--entrypoint PROGRAM] [-e VAR[=VALUE]]...
[-v SOURCE:TARGET[:ro]]... [-l KEY=VALUE]... [--restart POLICY] [-m SIZE]
[--cpus N] [--pids-limit N] [HEALTHCHECK OPTIONS] [OTHER OPTIONS]
[--secret NAME[:TARGET[:MODE[:UID:GID]]]]... [--image-command] [--no-wait]
[-- ARGUMENTS...]
start regenerates /etc/systemd/nspawn/NAME.nspawn from the machine’s
record, sets up the network it is configured for, starts the
unit through systemd and waits up to 30 seconds for the machine to register
with machined. For a booted machine it then waits, up to 20 more seconds, until
the machine’s own systemd is listening, so that a command can follow right
away; --no-wait skips that last wait. A program that runs and returns at once
is reported as such, not as a failure.
Everything given to start is remembered for the machine, so a plain
nspawn start NAME next time reuses the network, the ports, the command, the
variables and the volumes of the last run:
--network switches the machine between the bridge, networks of its own
made with network create (several at once, --network-alias giving it
more names on them), a veth pair, the host’s network and none; see
Networking.-p [IP:]HOST:CONTAINER[/udp] publishes a port on the host, on every
address or on one, like docker. It needs a bridge network; -p none forgets
all published ports.--entrypoint, -e and the arguments after -- change what an app
image runs; see Command, entrypoint and environment.
Boot images refuse them.-v mounts a host directory or a named volume into any kind of machine; see
Volumes.-l/--label, --restart, -m, --cpus and --pids-limit label the
machine, give it a restart policy and bound its resources; see
Labels and Restart policies and limits.--health-cmd and the other --health-* options give it a healthcheck, or
replace the image’s; see Healthchecks.--hostname, -u, -w, --cap-add, --cap-drop, --privileged,
--read-only, --tmpfs, --shm-size, --device, --dns, --dns-search,
--add-host, --ulimit, --oom-score-adj, --stop-signal,
--stop-timeout, --init and --sysctl are the other flags of
docker run, --log-driver where the program’s output goes, and
--timezone how systemd-nspawn sets
/etc/localtime; see The other flags of docker run.--secret hands it a secret as a file; see Secrets.Machines are started by name. A reference (fedora:44, docker.io/x) is
refused with a hint to pull it or to create a machine from a local image.
Images that were not installed by nspawn, for example something created with
machinectl import-tar, can be started too: they get the stock template’s veth
networking, and nspawn makes sure systemd-networkd runs on the host so that the
machine actually gets an address, but none of the flags above apply to them.
sudo nspawn run [-d] [--rm] [-i] [-t] [OPTIONS] REFERENCE [COMMAND [ARGUMENT...]]
run makes a machine from an image and starts it, like docker run: pull,
or create from a local image with the same reference, then start, with the
options of both. What follows the image replaces an app’s command, as with
docker. Without -d it stays with the machine:
nspawn logs NAME shows
it later, and the machine goes on should run be interrupted. With
--log-driver none it comes straight from the program and nothing is kept.run exits with the program’s exit code, or 128 plus the signal it died of:
130 after Ctrl-C, 137 after kill. Ctrl-C, SIGTERM, SIGHUP and SIGQUIT go to
the program; a third Ctrl-C within a second leaves the machine running and
returns.-i gives the program this standard input, -t a terminal; closing that
terminal stops the machine, since nothing would read it any more.--rm removes the machine once it ends, and leaves nothing behind when the
start fails. Named volumes stay, and so does an image the run had to pull,
under the image’s local name, as docker keeps images; without --name the
machine gets a name of its own.sudo nspawn run --rm docker.io/library/busybox:latest sh -c 'echo hi; exit 3'; echo $?
echo abc | sudo nspawn run -i --rm docker.io/library/busybox:latest wc -c
sudo nspawn run -it --rm docker.io/library/alpine:3 sh
sudo nspawn run -d --name web -p 8080:80 docker.io/library/nginx:latest
A booted image shows its console until it powers off (Ctrl-C powers it off).
run -it on one waits for its boot, opens a root shell and powers the machine
off when the shell ends, with the shell’s exit code: sudo nspawn run -it --rm fedora:44 is a throwaway Fedora with its own systemd.
A boot image is started with Boot=yes: systemd-nspawn runs the image’s init
as PID 1, like machinectl start does, and the machine joins the bridge with
an address that its own systemd-networkd configures.
exec enters the machine’s namespaces directly, like docker exec, so
nothing is needed inside: no D-Bus, no PAM. The exit code comes back and the
program is found on the machine’s PATH.shell opens machined’s login session (OpenMachineShell), which needs
D-Bus inside the machine; the hub images have it. Right after start the
machine’s D-Bus may not be up yet, so shell retries for up to 20 seconds
instead of failing.stop asks machined to power the machine off and repeats the request every
two seconds until the machine is gone, for up to a minute; the repetition
covers the window right after boot in which the init has not installed its
signal handlers yet. --force kills every process at once.nspawn-volumes.service,
holds local-fs.target until all of them are mounted, so services find their
data in place whatever the backend.An app image is started with Boot=no and ProcessTwo=yes: nspawn’s stub init
is PID 1 and the image’s entrypoint runs under it, as its child, with the
environment, working directory, user and stop signal from the OCI config. Its
network namespace is built on the host before the program starts, so the
network is there from the first instruction, as in docker. For the same reason
an app on the bridge runs without a user namespace (PrivateUsers=no), which
is also docker’s default; capabilities, seccomp and the other namespaces still
apply, and such an app keeps docker’s default capabilities and SYS_BOOT,
with the kexec system calls filtered out, rather than systemd-nspawn’s, whose CAP_SYS_ADMIN would be root on the host
outside a user namespace. --cap-add puts one back, --privileged all of
them.
exec and shell enter the namespaces of the machine’s leader process, on a
pseudo terminal, with the image’s environment. shell runs /bin/sh, and
-u USER switches user for both.stop sends the image’s stop signal (StopSignal in the config, SIGTERM
by default, --stop-signal over it) to the program itself, waits --timeout
seconds (the machine’s --stop-timeout, else 10) and kills the machine if it
is still there, like docker stop. A stop
through systemd (systemctl stop, shutdown) reaches the stub init as a
poweroff request, which it answers with SIGTERM to the program.nspawn stop on it afterwards only clears
what it left behind and says NAME was not running.What an app runs is decided exactly as with docker:
-- replace the image’s cmd and follow its
entrypoint. nspawn start web -- nginx -T still runs
/docker-entrypoint.sh first, as docker run nginx nginx -T would.--entrypoint PROGRAM replaces the entrypoint with one program; the
arguments after -- follow it. --entrypoint "" drops the entrypoint, so
the arguments run alone.--image-command forgets them and runs the image’s own entrypoint and cmd
again.-e VAR=value adds a variable on top of the image’s, and -e VAR copies it
from the shell that runs nspawn. The later value of a variable wins, -e none
forgets them all, and exec sees the same environment as the program. Names
follow the usual rules (letters, digits and _, not starting with a digit).-w, -u and --stop-signal say otherwise. docker’s USER:GROUP
form is taken as docker takes it, from the image’s config or from -u: the
group, a name or a number, is the program’s primary and only group. The user is resolved as
docker resolves it, from the image’s passwd and group files: systemd-nspawn
asks getent inside the machine, which busybox lacks and musl’s (alpine)
cannot answer, so nspawn binds a stand-in that answers those lookups and
hands any other to the image’s own getent. A uid the passwd does not list
runs all the same.ps shows the effective command; nspawn start NAME -- true is a quick way to
check that an image runs at all.
sudo nspawn start web -v /srv/www:/usr/share/nginx/html:ro -v pgdata:/var/lib/postgresql
-v SOURCE:TARGET[:ro] is docker’s syntax:
SOURCE is a host directory (or file) that has to exist already,
as with podman: start refuses a path that is not there rather than making
one. A SOURCE without a leading / is a named volume that nspawn
keeps under /var/lib/nspawn/volumes/NAME, created on first use with what
the image has at TARGET, owner and files included, as docker seeds one
(a program that runs as a user finds its data directory its own), and
never deleted by rm or images rm. Names may contain letters, digits,
_, . and -, and may not start with a dot.TARGET is an absolute path inside the machine, other than /. The same
target cannot be mounted twice.:ro mounts it read-only; :rw is the default. Paths with whitespace are
not supported.-v is repeatable and remembered; -v none forgets every volume.In machines that run with private users (booted machines on overlay and flat,
which is the default), the mount is idmapped, so root inside owns what it
writes on the host, as docker users expect. On mstack machines nspawn attaches
the volumes from the host right after the machine’s init starts, since
systemd-nspawn cannot idmap binds under managed user namespaces; that is what
nspawn-volumes.service waits for, for up to two minutes, failing visibly
otherwise. On overlay and flat the volumes come from the settings file and are
there before the init even runs.
Named volumes outlive the machines that use them. nspawn volume ls lists them
with the machines whose records mount them, volume create NAME makes one
ahead of its first use, and volume rm and volume prune remove the ones no
machine uses; a volume a machine still names is refused until that machine is
started with other volumes (or -v none) or removed.
sudo nspawn start web --label caddy=web.example --label tier=front
Labels work as in docker. The image’s own labels (LABEL in a Containerfile,
OciLabels= in a mkosi.conf) are read from its configuration when it is
pulled or built, and -l/--label KEY=VALUE on start or create adds the
machine’s own on top, remembered like the ports; --label none forgets them.
nspawn does nothing with them itself: nspawn inspect and ps --json show the
merged labels and the image’s image_labels, for tools that configure
themselves from them, a reverse proxy say.
sudo nspawn start web --restart unless-stopped -m 512m --cpus 1 --pids-limit 500
--restart takes docker’s policies:
no, the default: a machine that ends stays down.on-failure: restarted when its program, or its init, dies with an error.always: restarted whenever it ends, and started at boot.unless-stopped: like always, until nspawn stop, which also takes it
off the boot list until the next nspawn start.A machine that ends is started again after a second, then later and later, up
to half a minute, for as long as it keeps failing; ps shows it as
restarting meanwhile, and nspawn stop ends that. The ports, the address and
the volumes come back with it, since the unit’s own hooks prepare every run.
always and unless-stopped enable the unit the way machinectl enable does,
and removing the machine takes that back. machinectl stop stops a machine for
good too, but does not take an unless-stopped one off the boot list, and a
poweroff from inside counts as ending under always.
-m/--memory (512m, 2g), --cpus (0.5, 2) and --pids-limit bound
the whole machine: they are the MemoryMax=, CPUQuota= and TasksMax= of its
unit, which is why nothing inside shows them. As with docker, --memory also
lets the machine use as much swap again (MemorySwapMax=), and no more, unless
--memory-swap gives memory and swap together, as docker takes it: equal to
--memory for no swap, larger for the difference, -1 for swap without a
bound. 0 removes a limit, and brings --memory-swap back to its default.
Both are remembered like the ports and apply at the next start, and
nspawn update changes them without one, like docker update: a running
machine gets the new limits in its cgroup at once.
sudo nspawn update web -m 1g --cpus 2 --restart always
With a policy,
stop --no-wait of an app also lets systemd-nspawn’s stub init send the
program SIGTERM and SIGHUP, since nobody stays behind to stop the unit later.
sudo nspawn start web --health-cmd "curl -fsS http://localhost/ || exit 1" --health-interval 10s --health-retries 3
Healthchecks are docker’s. An image’s HEALTHCHECK applies as it is, and
--health-cmd with --health-interval, --health-timeout,
--health-retries, --health-start-period and --health-start-interval on
start, run, create and update replace it; --no-healthcheck turns it
off. Durations are docker’s (10s, 1m30s, 500ms), and so are the
defaults: 30 seconds between probes, 30 seconds for a probe, 3 failures in a
row before the machine is unhealthy, no start period, and 5 seconds between
probes during one.
The probe runs inside the machine through /bin/sh -c (exit 0 is healthy),
from a unit of its own, nspawn-health-NAME.service, bound to the machine’s
so that it goes with it. The machine is starting until a probe succeeds or
the start period ends, healthy after a success and unhealthy after the
retries fail in a row; a success brings it back.
ps shows the health next to the state: running (healthy),
running (unhealthy), running (health: starting).
inspect shows health with the status, the failing streak and the last
five probes, each with its times, exit code and output, and healthcheck
as it applies:
"health": "healthy",
"healthcheck": {
"interval": 2000000,
"retries": 2,
"start_interval": 0,
"start_period": 0,
"test": [
"CMD-SHELL",
"test -f /ok"
],
"timeout": 0
},
events reports health_status with the new status on every change.
update changes the healthcheck of a running machine at once, and the
probes start afresh.
A stopped machine has no health, and neither has one without a healthcheck.
sudo nspawn start web --hostname web-1 -u nobody -w /tmp --cap-drop ALL --cap-add NET_BIND_SERVICE \
--read-only --tmpfs /run:size=64m --tmpfs /tmp --shm-size 64m --device /dev/null:/dev/nullo \
--dns 10.99.0.1 --dns-search example.test --add-host db:10.99.1.3 --add-host gw:host-gateway \
--ulimit nofile=1024:4096 --oom-score-adj 500 --stop-signal SIGQUIT --stop-timeout 30 \
--sysctl net.ipv4.icmp_echo_ignore_all=1
They mean what they mean to docker, are remembered like the rest and shown by
inspect. Each becomes a line of the machine’s .nspawn settings file or of
its unit:
--hostname NAME: the hostname inside, the machine’s name by default. A
booted machine gets it as its /etc/hostname, over the image’s, since its
systemd sets the hostname from that file.-u USER[:GROUP] and -w DIR: the user (a name or a uid, listed in the
image’s passwd or not, with a group of the image’s group file or a gid
after a colon, as docker’s --user takes it; a group name the image lacks
is refused) and the working directory the program runs with, instead of the
image’s. App images only.--cap-add, --cap-drop and --privileged: capabilities on top of, or
out of, the default set (NET_ADMIN or CAP_NET_ADMIN; ALL): docker’s
(CHOWN, DAC_OVERRIDE, FOWNER, FSETID, KILL, MKNOD, NET_BIND_SERVICE,
NET_RAW, SETFCAP, SETGID, SETPCAP, SETUID, SYS_CHROOT, AUDIT_WRITE) plus
SYS_BOOT, so that a reboot from inside ends the machine, for an app on a
bridge network, which runs without a user namespace, and systemd-nspawn’s
for a machine in one. An interface given with
--interface keeps NET_ADMIN on its own. --cap-drop ALL --cap-add NET_BIND_SERVICE keeps that one, as with docker, and --privileged keeps
every one.--read-only: the root read-only. --tmpfs PATH[:OPTIONS]: an empty tmpfs
at a path (size=64m, mode=1777), which is how a read-only machine still
writes /tmp or /var/cache. /run is a tmpfs of every machine already,
so a --tmpfs that lands on it (/var/run in most images, docker’s habit
for a read-only nginx) is left out with a note. --shm-size: the size of
/dev/shm.--device HOST[:CONTAINER[:PERMISSIONS]]: a device node of the host, bound
into the machine and allowed to its cgroup (r, w, m; rwm by
default), or a directory such as /dev/dri, whose nodes are allowed one by
one. In a machine with private users the node keeps the host’s
ownership.--dns and --dns-search: the machine’s resolv.conf, instead of the
host’s resolvers. --add-host HOST:IP (or HOST=IP, for an IPv6 address):
lines for its /etc/hosts,
host-gateway standing for the host’s address on the machine’s network.--ulimit NAME=SOFT[:HARD]: resource limits of the program (nofile,
nproc, core, …; unlimited is a value). --oom-score-adj: the
machine’s, -1000 to 1000.--stop-signal and --stop-timeout: what stop, restart and kill use
for the machine instead of the image’s stop signal and the default 10
seconds; -t on stop still wins.--timezone MODE: how systemd-nspawn sets the machine’s /etc/localtime at
each start. Its default, auto, points it at the host’s zone every time, so
a zone set inside with timedatectl does not survive a restart; off leaves
the machine’s own, and copy, bind, symlink and delete are its other
modes. An app also takes -e TZ=.--log-driver DRIVER: where the program’s output goes. local, the
default, sends it to journald’s nspawn namespace, a journal of nspawn’s
own apart from the system’s, so that a program that writes a lot never
floods the host’s logs; logs and an attached run read it, and
journalctl --namespace=nspawn shows every machine’s. It holds 1 GiB at
most, shared by all machines, the oldest lines going first
(/usr/lib/systemd/journald@nspawn.conf, which
/etc/systemd/journald@nspawn.conf overrides). journal keeps the output
in the system’s journal, and none drops it: logs then refuses the
machine, an attached run of an app still shows the output straight from
the program, and a booted machine, whose console goes nowhere, runs with
-d.--init: accepted for docker’s sake; nspawn’s stub init reaps orphans
anyway. App images only.--sysctl KEY=VALUE: net.* keys, set in the network namespace nspawn
makes for an app machine on a bridge network. Nothing else is accepted, and
a booted machine sets its own.--interface IFACE: a network interface of the host, moved into the
machine while it runs and back on the host when it stops, a wifi adapter
with its whole phy included; no docker counterpart. See Physical
interfaces.A list takes none to forget it (--cap-drop none, --tmpfs none), a value
an empty string (--hostname "") or 0 (--oom-score-adj 0), and
--privileged=false and --read-only=false take those back. A path inside the
machine, the target of a volume or a secret, a --tmpfs or a --device path,
must be plain: a . or .. component is refused, since it would land elsewhere
once mounted. A path the image
declares as a volume with nothing mounted over it gets a note at start: nspawn
has no anonymous volumes, so what is written there goes with the machine.
printf '%s' 'hunter2' | sudo nspawn secret create db-password
sudo nspawn start db --secret db-password -e POSTGRES_PASSWORD_FILE=/run/secrets/db-password
Secrets are docker’s, without a swarm. secret create NAME reads the content
from standard input, or from --file, and keeps it encrypted with
systemd-creds, bound to the host’s TPM2 where there is one and to its
credential key otherwise, under /var/lib/nspawn/secrets; -l KEY=VALUE
labels it. --secret NAME on start, run or create gives the machine the
plaintext at /run/secrets/NAME, a read-only file of root’s with mode 0444;
NAME:TARGET:MODE:UID:GID chooses the path, mode and owner, as docker’s long
form does. The file is decrypted when the machine starts, into a tmpfs of
root’s alone, and gone when it stops.
secret ls lists the secrets with the machines that take them:
SECRET SIZE CREATED USED BY
db-password 7 B 9s ago api
secret inspect prints what is known about them, never the content, and secret rm
removes the ones no machine takes; one a machine’s record names is refused
until that machine is started with other secrets (or --secret none) or
removed. Secrets are not attached to mstack machines yet.
sudo nspawn ps [-a] [--json] # same as: nspawn machines ls [-a] [--json]
MACHINE IMAGE MODE COMMAND STATE UP PID NETWORK OS
api docker.io/library/busybox:latest app /bin/sh -c while true; do sleep 1; done running 6s 895101 back:10.99.2.4 -
db docker.io/library/busybox:latest app /bin/sh -c while true; do sleep 1; done running 8s 894865 back:10.99.2.2 -
web docker.io/library/busybox:latest app /bin/sh -c touch /ok; while true; do ... running (healthy) 7s 894994 front:10.99.1.2 back:10.99.2.3 127.0.0.1:8080->80/tcp -
ps lists every container machined knows about (the virtual machines it also
registers, libvirt’s among them, are left out); machines that nspawn did not
install show - in the image columns. -a adds the nspawn machines that are
not running. COMMAND is the effective entrypoint and arguments of an app,
and NETWORK the machine’s address on the default network, its other networks
as NAME:ADDRESS, and the published ports, or host, veth, none or
container:NAME. A
machine between two runs of its restart policy is listed as restarting even
without -a, a frozen one as paused, and a healthcheck follows the state:
running (healthy).
ps --json prints the same machines as the service returns them, each with its
whole record, and nspawn inspect NAME... prints one or more machines, running
or not, as a JSON array, like docker inspect: what a script or an agent
reads.
sudo nspawn exec MACHINE [-u USER] [-e VAR[=VALUE]]... [-w DIR] [-T] [-t] [-i] [-d] COMMAND...
sudo nspawn shell MACHINE [-u USER]
exec runs one command inside a running machine of either kind, on a terminal
when standard input and output are both one, and exits with the command’s status, so it works in scripts and
pipelines (what goes through stdin and stdout is byte exact). The program is
looked up on the machine’s PATH, the image’s environment and the -e
variables of the machine apply, and the working directory is the machine’s.
Neither D-Bus nor anything else is needed inside. The command runs with the
machine’s capabilities and resource limits, like the machine’s own processes,
except CAP_SYS_BOOT in an app that shares the host’s user namespace (unless
--privileged): the machine keeps it beside a seccomp filter that keeps kexec
out, which the command does not carry. Right after a start it waits, a few seconds at most, until systemd-nspawn has
finished confining the machine. The flags are docker exec’s: -u USER[:GROUP]
takes names or numbers of the image’s passwd and group files (with a group, that
one is the only group), -e adds variables for this command, -w a working directory, -T refuses a
terminal even from one (pipes, as in a script) and -t asks for one even
without, -i is accepted for docker’s sake, and -d leaves the command
running in the background and returns at once.
shell opens an interactive shell as root (or -u USER): machined’s login
session for booted machines, /bin/sh in the machine’s namespaces for apps.
sudo nspawn cp ./nginx.conf web:/etc/nginx/
sudo nspawn cp web:/var/log/nginx ./nginx-logs
sudo nspawn cp ./site/. web:/usr/share/nginx/html
cp copies files and directories between the host and a machine, running or
not, with docker cp’s rules:
/
has to be a directory.DIR/. copies the contents of DIR rather than DIR.cp. Modes and modification
times are kept./proc, /sys and the like).A stopped overlay or flat machine can be copied into and out of; a stopped
mstack machine cannot, since its tree only exists while it runs. Where
SELinux enforces, a host directory mounted with -v keeps its own label, which
the service may not be allowed to write to (docker needs :z for the same);
named volumes are nspawn’s own and always work.
sudo nspawn logs MACHINE... [-f] [-n N] [--since WHEN] [--until WHEN] [-t] [--all] [--inside]
What the machine writes to its console goes, by default, to journald’s
nspawn namespace under systemd-nspawn@MACHINE.service, apart from the
system’s journal (see --log-driver above), and logs reads it with
journalctl, together with the system’s journal, where systemd’s lines and
what a machine wrote before 1.8.0 or with --log-driver journal are. By
default only the machine’s own output is shown, from every run of the unit,
earlier ones included:
-f keeps printing new output, starting from the last 10 lines unless -n
says otherwise. -n N shows the last N lines; --since "10 min ago" and
--until accept anything journalctl does, and nothing is followed past
--until; -t prefixes each line with its timestamp.web | ...), the way docker compose shows them.--all also shows what systemd logged about the unit: start, stop,
failures.--inside reads the journal of a booted machine itself
(journalctl --machine), which is where the services running inside log.logs works for stopped machines too, since the journal keeps what they wrote.
sudo nspawn stats [NAME...] [--no-stream] [--json]
stats shows what each running machine uses, like docker stats, drawn again
every second: CPU (100% is one CPU busy), memory in use against the limit (the
host’s memory without one), network and disk traffic, and processes. The
numbers are those of the cgroup of the machine’s unit, which holds the whole
machine, and of its interfaces as the machine sees them. --no-stream prints
one table, --json one object per machine and reading.
NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
api 0.00% 2.4 MiB / 3.8 GiB 0.06% 372 B / 0 B 0 B / 4.0 KiB 3
db 0.00% 2.4 MiB / 3.8 GiB 0.06% 892 B / 0 B 0 B / 4.0 KiB 3
web 0.00% 2.4 MiB / 3.8 GiB 0.06% 522 B / 0 B 0 B / 4.0 KiB 3
sudo nspawn events [--since WHEN] [--until WHEN] [-f KEY=VALUE]... [--json]
events reports what happens, like docker events: machines that start, die
(with their exit code), stop, are restarted, run out of memory, fail or change
health (health_status), and what nspawn does: pulls, builds, creations,
pushes, kills, updates and removals, of networks, volumes and secrets too. It
reads the journal, where systemd logs every start and end of a machine’s unit
however it was started, and nspawn logs what it does, so --since reads past
events back. Filters take name=, type= (machine, network, volume,
secret), event= and label=KEY or label=KEY=VALUE: the same key given
twice matches either value, different keys must all match. --json prints one
object per event.
2026-09-24T20:44:51.767425Z network create backend (interface=nsbr-backend, internal=false, subnet=10.99.1.0/24)
2026-09-24T20:44:53.691583Z machine start db (image=docker.io/library/busybox:latest)
2026-09-24T20:44:56.730139Z machine create busybox (from=api, reference=docker.io/library/busybox:latest)
2026-09-24T20:44:57.105739Z machine start busybox
2026-09-24T20:44:57.125181Z machine die busybox (code=exited, exit_code=3)
2026-09-24T20:44:57.154522Z machine fail busybox (result=exit-code)
2026-09-24T20:44:57.381848Z machine remove busybox (image=docker.io/library/busybox:latest)
2026-09-24T20:44:57.645267Z machine kill api (image=docker.io/library/busybox:latest, signal=1)
2026-09-25T10:20:16.502186Z machine health_status web (image=docker.io/library/busybox:latest, status=healthy)
Times are in UTC. A machine’s die carries the exit code systemd-nspawn gave:
255 for an app whose program died of a signal, as stop makes it. Every event
of a kind carries the same metadata whatever the action, so a remove says as
much as a create: image= on a machine’s, subnet=, interface= and
internal= on a network’s, path= on a volume’s, size= on a secret’s.
sudo nspawn stop NAME... [-f] [-t SECONDS] [--no-wait]
Stops the machines as described above for boot and app machines, one after
the other, waits until each is gone (a minute at most for a booted machine),
stops the unit so that the image can be removed right away, clears the failure
a signal-killed program leaves on the unit, releases the firewall exceptions
of a veth machine and removes the machine’s published ports and network
namespace. --no-wait returns right after the request, without the kill after
--timeout (the machine’s --stop-timeout, else 10 seconds); the unit hooks
release the network when the machine ends. -f kills every process at once,
like docker kill. Stopping a machine that already ended is not an error, and
a paused machine is thawed first.
With a restart policy the machine stays stopped for now: an unless-stopped
machine is also taken off the boot list until the next start, while an
always one still starts at the next boot.
sudo nspawn restart NAME... [-t SECONDS]
restart is a stop followed by a start with everything the machine
remembers, like docker restart; a machine that is not running is started.
-t is stop’s.
sudo nspawn pause NAME...
sudo nspawn unpause NAME...
pause freezes every process of a machine through the cgroup freezer of its
unit, like docker pause, and unpause thaws them. ps shows a frozen
machine as paused; stop and kill thaw it first.
sudo nspawn top MACHINE
PID USER TIME COMMAND
894994 root 00:00:00 (sd-stubinit)
895009 root 00:00:00 /bin/sh -c touch /ok; while true; do sleep 1; done
895198 root 00:00:00 sleep 1
top lists the processes of a running machine, like docker top: the PID as
the host sees it, the user as the machine sees it, CPU time and command, read
from the machine’s cgroup and PID namespace.
sudo nspawn kill web # SIGKILL: stopped for good
sudo nspawn kill -s HUP web # a signal for the program
kill sends a signal to machines, like docker kill. SIGKILL, the default,
stops the machine for good, as stop --force does. Any other signal goes to
the program of an app, or to the init of a booted machine, and the machine
lives on unless it ends of it; then its restart policy applies, except when the
signal was the machine’s own stop signal, which counts as a stop, as with
docker.
sudo nspawn rm web2
sudo nspawn rm -f web # stop it first, like docker rm -f
rm removes a machine: its record, its tree, its unit files, the boot link a
restart policy made, and the layers nobody else uses. A pulled image is a
machine too, so rm and images rm remove the same thing; rm -f stops a
running machine first where both refuse otherwise. Named volumes are kept, and
rm says which.
Every machine records which networks it uses. start --network changes them,
and the choice sticks for the next start.
| Network | Default for | What the machine gets |
|---|---|---|
bridge | Every image nspawn installs | A fixed address on the nspawn0 bridge, NAT to the outside, published ports, the names of the other machines and of the host. Needs nothing from the host’s own network manager. |
| a network’s name | The same on a bridge of its own, made with network create; several at once, with --network repeated: see Networks of your own. | |
host | The host’s network namespace, like docker run --network host: the machine sees the host’s interfaces and binds to the host’s ports. Works for both kinds of image. | |
none | No network at all, like docker run --network none: lo and nothing else. Works for both kinds of image. | |
container:NAME | The network namespace of the running machine NAME, like docker run --network container:NAME: its interfaces, address, hosts file and resolv.conf, nothing of the machine’s own. App images only; see container:NAME. | |
veth | Images not installed by nspawn | The classic systemd-nspawn setup: a virtual ethernet pair whose host end is configured by systemd-networkd through the stock 80-container-ve.network. Booted images only. |
On the first start of a bridged machine (or with nspawn network up, which
is handy at boot) nspawn creates the bridge with the first address of the
subnet, enables IPv4 forwarding and installs the nftables table ip nspawn
with masquerading for the subnet. The defaults are the bridge nspawn0 and the
subnet 10.99.0.0/24; both, and the DNS servers, can be changed in
the configuration file. An interface that already has
that name is only taken over when it is a bridge nspawn made, or an empty one;
bridge = "docker0" is refused rather than acted on. Nothing else on the host
is touched, so it works the same with systemd-networkd, NetworkManager or no
network manager at all.
Each machine gets a fixed address from the subnet, remembered with its record.
A booted machine receives it through a .network file that nspawn
generates and bind-mounts at /run/systemd/network/10-host0.network, for the
systemd-networkd inside to apply, together with the DNS servers: the host’s
upstream resolvers by default, dns from the configuration if set, --dns
and --dns-search of the machine over both, and public resolvers as a last
resort, with a warning, when none can be determined.
An app machine has nothing inside to configure an interface, so nspawn
builds its network namespace before the program starts: ip netns, a veth pair
on the bridge, the address and the default route, handed to systemd-nspawn with
NamespacePath=, plus a generated /etc/resolv.conf. The namespace lives at
/run/netns/nspawn-NAME while the machine runs and goes away with it, and
--sysctl sets net.* keys in it.
A generated /etc/hosts, mounted into every bridged machine, resolves the names
of the other machines on the bridge and host.nspawn.internal for the host;
--add-host HOST:IP adds lines of your own, host-gateway standing for the
host’s address on the machine’s network. On hosts with systemd 258 or newer,
machined also lets the host resolve machine names by itself. The bridge
carries IPv4 only, so neither the machines nor the bridge get an IPv6
link-local address, and a machine’s name leads to its bridge address:
ping web from the host answers from 10.99.0.x.
sudo nspawn network inspect front
[
{
"created": 1790331611,
"gateway": "10.99.1.1",
"interface": "nsbr-front",
"internal": false,
"labels": {
"tier": "edge"
},
"machines": [
{
"address": "10.99.1.2",
"aliases": [
"www"
],
"name": "web",
"ports": [
"127.0.0.1:8080->80/tcp"
],
"running": true
}
],
"name": "front",
"subnet": "10.99.1.0/24"
}
]
Like docker’s user-defined networks, network create makes a bridge of its own
for a group of machines:
sudo nspawn network create back --internal
sudo nspawn network create front --label tier=edge
sudo nspawn run -d --name db --network back --network-alias postgres docker.io/library/postgres:17
sudo nspawn run -d --name web --network front --network back --network-alias front=www -p 127.0.0.1:8080:80 docker.io/library/nginx:latest
back is up: nsbr-back on 10.99.2.0/24
front is up: nsbr-front on 10.99.1.0/24
The bridge is nsbr-NAME (a hash of the name when it is too long for an
interface name), and its subnet the next /24 of network_pool (10.99.0.0/16
unless set in the configuration file) that overlaps no
other network and nothing the host routes already; --subnet chooses one.
Machines of one network reach each other by name, through the same generated
/etc/hosts, and host.nspawn.internal is that network’s gateway. Nothing
of another network reaches them, the default one included: the forward chain
of the ip nspawn table drops what crosses from one bridge to another, and a
drop there holds whatever firewalld or iptables allow. Ports a machine
publishes are the exception, reachable from every network through the host,
as from the LAN. --internal makes a network with no way out: its machines
reach each other and the host, nothing beyond, and cannot publish ports.
A machine may join several networks: --network front --network back puts it
on both, with an address on each, the first one primary. The primary network
is where its published ports lead and, unless it is internal, where its default
route goes; an internal primary leaves the route to the first network that is
not. A booted machine gets one interface per network (host0, host1, …),
each configured by the systemd-networkd inside through a .network file of
its own; an app machine gets them in the namespace nspawn prepares. Its
/etc/hosts lists, for every network it is on, the members of that network
with their addresses there, so a proxy on front and back reaches both
sides while front and back still do not reach each other. veth and
host go alone.
--network-alias NAME gives a machine another name on its primary network,
--network-alias NETWORK=NAME on that network, as docker run --network-alias does: every member of the network resolves the alias as well,
which is how db can stand for postgres-17 today and for another machine
tomorrow. An alias several machines share leads to the first of them by name.
--network-alias none forgets them.
network ls lists the networks with the machines whose records name them,
network inspect NAME shows one with its machines, their addresses, aliases
and ports, and network rm and network prune remove the ones no machine
uses, bridge, rules and firewall exceptions included. network create --label KEY=VALUE tags a network for whoever reads network inspect, as docker’s
does:
sudo nspawn network ls
NETWORK INTERFACE SUBNET INTERNAL MACHINES
bridge nspawn0 10.99.0.0/24 no bb
back nsbr-back 10.99.2.0/24 yes api db web
front nsbr-front 10.99.1.0/24 no web
The /etc/hosts of web, on front and back with www as its alias on
front, lists the members of both networks with their addresses there:
# Generated by nspawn; do not edit.
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
10.99.1.2 web www
10.99.2.3 web
10.99.1.1 host.nspawn.internal
10.99.2.4 api
10.99.2.2 db postgres
sudo nspawn start web -p 8080:80 -p 5353:53/udp -p 127.0.0.1:9090:9090 -p 8000-8010:8000-8010
Each -p [IP:]HOST:CONTAINER[/udp] becomes a DNAT entry in the ip nspawn
table. Without an address the port is reachable from other hosts, from the
host’s own addresses and from 127.0.0.1 (through route_localnet, as docker
does without its userland proxy); with one, 127.0.0.1:9090:9090 say, on that
address of the host alone, for a reverse proxy in front. A range,
8000-8010:8000-8010, is one mapping per port, which inspect shows one by
one. The entries are installed once the machine is registered and removed when
it ends, by the unit hooks, so they also go away after a crash or when the
program exits on its own. A port another running machine publishes, on every
address or on that one, or one a service of the host already listens on, is
refused before the machine starts, and a refused port is not remembered. On a
machine with several networks the port leads to its primary address.
The list is remembered for the machine: nspawn start web next time publishes
the same ports, and -p none forgets them all. Ports need the bridge network;
a machine on the host’s network listens on the host’s ports directly.
trusted zone at runtime, which
also lets published ports through. The binding does not survive
firewall-cmd --reload; the next start or nspawn network up puts it
back.FORWARD
policy to DROP, which would silence every machine on the bridge. start
then adds two rules to the DOCKER-USER chain, which docker reserves for
that, or to the top of FORWARD itself: anything out of the bridge, and into
the bridge only what was published or belongs to a connection a machine
opened. That needs the iptables command, which those tools bring with them.iptables command to edit them,
start says so and the exception has to be made by hand. Without it the
machines reach nothing beyond the bridge and published ports answer on this
host alone.--network veth keeps the classic systemd-nspawn behaviour for booted
machines: systemd-nspawn creates a virtual ethernet pair and systemd-networkd on
the host brings up the host end (ve-NAME), gives it an address, serves DHCP
to the machine and masquerades its traffic, all through the stock
80-container-ve.network. App images refuse it, since nothing inside would
configure the pair.
Because that needs systemd-networkd, start activates it when the host has no
.network files of its own in /etc/systemd/network or /run/systemd/network,
and refuses with an explanation when it has, so that nspawn never takes over
interfaces another network manager is handling. A masked systemd-networkd is an
error. With firewalld, ve-NAME is bound to the trusted zone while the
machine runs; otherwise the default zone would drop the machine’s DHCP
requests.
--network host sets VirtualEthernet=no: the machine shares the host’s
network namespace, sees the host’s interfaces and binds to the host’s ports.
Published ports do not apply, and an app that runs this way keeps its user
namespace.
--network none sets Private=yes: the machine has lo and nothing else,
like docker run --network none. Published ports do not apply, and an app that
runs this way keeps its user namespace too.
--network container:NAME puts an app machine in the network namespace of the
machine NAME, docker’s sidecar pattern (a VPN client with the programs that
must go through it, for one): the same interfaces and address, NAME’s hosts
and resolv.conf files, and nothing of the machine’s own. A port the program
serves is published with -p on NAME, since that is where the address is.
NAME can be an app or a booted machine on any bridge network, has to be
running when the machine starts, and cannot be removed while a machine names
it. When NAME stops or restarts, the machine keeps the namespace it joined,
which then leads nowhere, and has to be restarted to join the new one, as with
docker.
sudo nspawn run -d docker.io/qmcgaw/gluetun --name vpn --cap-add NET_ADMIN --device /dev/net/tun -p 8080:8080
sudo nspawn run -d docker.io/library/nginx:1.27 --name web --network container:vpn
--interface IFACE on run, create and start gives a machine a network
interface of the host, whole: a second ethernet port, or a wifi adapter for a
machine doing wireless work. --device cannot do it, since a network interface
is not a node under /dev, and --network host cannot either: the machines
run in a user namespace, which may not configure the host’s interfaces. The
interface is moved into the machine’s network namespace before its program or
init runs and is back on the host when the machine stops or is removed, with
its name kept on both sides, as systemd-nspawn’s Interface= and LXC’s phys
type do it. It arrives down and unconfigured, as the kernel moves it: the
machine brings it up and configures it, with ip in an app or its
systemd-networkd in a booted machine. The flag is repeatable and remembered
like the rest, --interface none forgets them, and inspect lists them as
interfaces.
sudo nspawn run -d kali:latest --name kali --interface wlp11s0f3u2u3
sudo nspawn exec kali -- iw dev
sudo nspawn stop kali # the adapter is back on the host
Where systemd-nspawn makes the machine’s network namespace (a booted machine,
on the bridge or with veth, and any machine with --network none) the
interface goes into the machine’s settings file as Interface= and
systemd-nspawn moves it at start and back at exit. An app machine on a bridge
network gets its namespace from nspawn itself (NamespacePath=, which allows
no Interface= beside it), so the unit’s hooks move the interface in with ip
before systemd-nspawn runs and give it back after; such an app runs without a
user namespace, so iw reg set and monitor mode work inside as on the host.
Either way the unit waits for the interface’s device unit, so that a machine
started at boot waits for a USB adapter udev has not seen yet.
A wifi adapter cannot leave its phy, so the phy moves whole, every interface it
carries with it (nspawn says which), and a driver without namespace support
(ath6kl, wilc1000) refuses the move naming the phy. An app on the bridge
takes a wireless interface on any systemd, through iw, which has to be
installed on the host; Interface= moves a phy since systemd 256, so on an
older host a wireless interface is refused for a booted machine or one with
--network none, with a message that says so. The regulatory domain (iw reg set) and rfkill are host-wide: they are set on the host, or from an app on
the bridge; --device /dev/rfkill hands the switch to such an app.
One machine at a time takes an interface: a second one naming it is refused
while the first runs (stop it, or start it with --interface none). Refused
as well: lo, the bridges and veth ends of nspawn’s own, a port of a bridge or
bond (ip link set IFACE nomaster first), a name that is not on the host (a
machine that ended a moment ago may still hold it: the kernel gives interfaces
back a little after their namespace dies), a machine with --network host, one
with --network container:NAME (give the interface to NAME) and the mstack
backend. The interface of the host’s default route is allowed with a warning,
since the host loses its route while the machine runs. When the interface
comes back, NetworkManager or systemd-networkd manage it again as before, and
a name taken on the host in the meantime makes the kernel rename it (dev0,
wlan0). stop --force kills the machine: an app on the bridge still gets
every interface back from its release hook, while a machine that took them
through Interface= may not get to give them back, and the kernel then
returns a physical interface or a wifi phy a moment later and destroys a
virtual one (a VLAN, a macvlan, a dummy) with the namespace.
Behind the scenes the service binds the name systemd-nspawn looks for
(/run/netns/nspawn-web) to the network namespace of vpn’s leader process,
the mount ip netns attach would make, so the settings are the ones of any
app on a bridge, and drops the name when web stops while the namespace
stays vpn’s. -p,
--network-alias, --dns, --dns-search, --add-host and --sysctl are
refused on such a machine, since they shape a network of its own, and so is a
booted image, whose systemd would configure the shared interfaces again. ps
and inspect show the network as container:NAME; network inspect does not
list the machine among the network’s members, as it has no address there.
sudo nspawn build [DIRECTORY] -t REFERENCE [--name NAME]
[-d DISTRIBUTION] [-r RELEASE] [--profile PROFILE]...
[--backend BACKEND] [--mode MODE] [--force] [--keep-output]
[-- MKOSI_ARGUMENTS...]
build runs mkosi on a directory that has
a mkosi.conf or a mkosi.conf.d (the current directory by default), with
--format=oci, and imports the result exactly as pull would import an image
from a registry. It needs root and mkosi in PATH. The same configuration
tree that works with mkosi on its own works here; --distribution,
--release, --profile and anything after -- are passed through.
The command line that mkosi receives is:
mkosi --directory=DIRECTORY --format=oci --compress-output=zstd
--output-directory=/var/lib/nspawn/builds/NAME-TIMESTAMP
--cache-directory=/var/lib/nspawn/cache/mkosi
--image-id=REPOSITORY [--distribution=...] [--release=...] [--profile=...]...
[MKOSI_ARGUMENTS...] --force build
The / characters of the repository are replaced by - in the image id. The
package cache is kept between builds; the output directory is deleted after the
import unless --keep-output is given.
When mkosi is done, nspawn finds the OCI layout it produced, adds a few
annotations to the manifest (org.opencontainers.image.version with the tag,
org.opencontainers.image.ref.name with the full reference, and
org.nspawn.builder), verifies every blob against its digest, copies the blobs
into the store and installs the image with the chosen backend, with build as
its origin. The image can be started right away, cloned with create or
pushed. The labels of its OCI configuration, which mkosi writes from
OciLabels=, are kept with it: nspawn inspect shows them as image_labels,
and they travel with the image when it is pushed.
A minimal configuration for a bootable Fedora machine, taken from the test suite of nspawn:
[Distribution]
Distribution=fedora
Release=44
[Output]
ImageId=e2e-built
OciLabels=org.nspawn.e2e=built
[Content]
Bootable=no
SELinuxRelabel=no
RootPassword=root
Packages=
systemd
systemd-networkd
systemd-resolved
dbus-broker
passwd
util-linux
systemd-networkd is what configures host0 on the bridge, and dbus-broker
(or dbus) is what lets shell reach the machine’s systemd. Then:
sudo nspawn build -t team/app:1 ./app
sudo nspawn start team-app-1
sudo nspawn push IMAGE [--to REFERENCE]
IMAGE is a local image name or the reference it was pulled from or built as.
By default the image is pushed under its own reference; --to pushes it under
another one, for example to retag it or to send it to a different registry:
sudo nspawn push team/app:1
sudo nspawn push team-app-1 --to registry.example/team/app:2
push authenticates first, with the credentials stored for that registry, and
tells you which nspawn login to run when the registry wants some it does not
have. The blobs that are already on the registry are skipped and the manifest
is uploaded last. A push needs a tag, not a digest, and the image’s manifest
and blobs must still be in the store (images rm removes them). See
Registries and credentials.
The images the team publishes on hub.nspawn.org are built the same way, from
the definitions in
nspawn/mkosi-definitions. Use
them as a starting point for your own images, or open a pull request there to
change what the hub ships.
nspawn needs no configuration to work: the defaults point at the hub, use
/var/lib/machines and /var/lib/nspawn, pick the best backend for the host
and run the bridge on 10.99.0.0/24. Everything below is optional.
--registry or
NSPAWN_REGISTRY, --ca-cert or NSPAWN_CA_CERT).The file is /etc/nspawn/nspawn.toml, read when it exists; --config FILE or
NSPAWN_CONFIG names another one, which then must exist. Every key is
optional, and unknown keys are an error so that a typo does not silently fall
back to a default.
| Key | Type | Default | Meaning |
|---|---|---|---|
registry | string | hub.nspawn.org | Registry for references without a host part. |
ca_cert | path | none | Extra CA certificate (PEM) to trust when talking to the hub. |
backend | auto, overlay, flat or mstack | auto | Backend for pull and build when they are not given --backend: auto is overlay, or flat without overlayfs; mstack is experimental. |
machines_dir | absolute path | /var/lib/machines | Where machines are assembled. |
state_dir | absolute path | /var/lib/nspawn | Blobs, layers, records, volumes and everything else nspawn keeps. |
bridge | string, 1 to 15 letters, digits, - or _ | nspawn0 | Name of the bridge the machines join. It must be free, or a bridge nspawn made. |
subnet | IPv4 CIDR, prefix 8 to 30 | 10.99.0.0/24 | Subnet of the bridge; its first address is the bridge’s own. |
network_pool | IPv4 CIDR | 10.99.0.0/16 | Where network create takes the /24 of a network given no --subnet: the first one that overlaps no other network and nothing the host routes. |
dns | list of IPv4 addresses | the host’s upstream servers | DNS servers handed to bridged machines. They must be reachable from the bridge: no loopback, no IPv6. |
An example:
# /etc/nspawn/nspawn.toml
registry = "registry.example:5000"
ca_cert = "/etc/pki/tls/certs/example-ca.pem"
backend = "overlay"
bridge = "br-lab"
subnet = "172.30.5.0/24"
dns = ["172.30.5.1", "9.9.9.9"]
Changing bridge or subnet affects machines started afterwards; the address
recorded for a machine is reassigned from the new subnet on its next start.
What the images of a registry must carry, one table per registry (host, or
host:port). The hub’s policy is built in (the project’s key or the build
workflow’s keyless identity, one of them required, its transparency log entry
checked): a table for hub.nspawn.org replaces it, and verify = false turns
the check off. Every other registry is verified only when it has a table.
[registries."registry.example.com"]
key = "/etc/nspawn/keys/example.pub" # a cosign public key (PEM); keys = [...] for several
identity = "https://github.com/org/repo/.github/workflows/build.yml@refs/heads/main"
issuer = "https://token.actions.githubusercontent.com" # goes with identity
required = true # default: no verifying signature fails the pull
rekor = true # default: the signature's transparency log entry must verify
trusted_root = "/etc/nspawn/trusted_root.json" # another Sigstore deployment; the public one otherwise
[registries."hub.nspawn.org"]
verify = false # nothing is checked for this registry
| Key | Type | Default | Meaning |
|---|---|---|---|
verify | boolean | true | false turns verification off for the registry, the hub’s built-in policy included. |
key, keys | absolute path, list of absolute paths | none | Cosign public keys (PEM) the images may be signed with; key and keys add up. |
identity, issuer | strings, both or neither | none | A keyless signer: the certificate’s identity (the SAN, a URI or an email) and the issuer of the token behind it. A table names at least one key or an identity, unless verify = false. |
required | boolean | true | false turns a missing signature into a note and lets the pull go on; a signature that is there but does not verify still fails it. |
rekor | boolean | true | false trusts the key or certificate alone, without the transparency log entry (a private deployment that keeps no log). A bundle still has to carry a log entry or a timestamp, which cosign always adds. |
trusted_root | absolute path | the public Sigstore root the binary embeds | The trusted_root.json of another Sigstore deployment. |
The keys are read by the service when a pull needs them, and checked when it
starts: a key or a trusted root it cannot read is reported in its journal and
fails the pulls of that registry. What the command line was given with
--registry chooses the registry, never the policy.
The file is read by the service, so a change takes effect on its next start:
sudo systemctl restart nspawn.service (or simply waiting for it to go idle)
picks it up. --config on the command line does not reach it: the command line
is a client and only carries the registry and its CA certificate. To put the
service on another file, name it when installing the service,
sudo nspawn --config /etc/nspawn/lab.toml daemon --install; the unit then
carries it, and so do the hooks the service writes for every machine.
| Variable | Same as |
|---|---|
NSPAWN_REGISTRY | --registry |
NSPAWN_CA_CERT | --ca-cert |
NSPAWN_CONFIG | --config |
They are convenient for scripts and for the end-to-end tests, which run against a private registry with a private CA:
NSPAWN_REGISTRY=hub.nspawn.test:8443 NSPAWN_CA_CERT=/etc/zot/ca.crt nspawn hub ls
nspawn login keeps registry credentials in /etc/nspawn/auth.json, and that
file is the only one consulted. See
Registries and credentials.
Some settings belong to a machine rather than to the host, and are given when it is created or started; every one of them is remembered until it is changed:
--backend and --mode on pull and build; --backend on create.--name on pull and build, to choose the local name.--network (bridge, veth, host, none or networks made with
network create, several of them), --network-alias, -p, -e, -v,
--label, --entrypoint and the arguments after -- on start, run and
create. -p none, -e none, -v none, --label none,
--network-alias none and --image-command forget what was remembered.--restart, -m/--memory, --memory-swap, --cpus and --pids-limit on start, run
and create, applied at the next start, and on update, applied at once.
--restart no and a limit of 0 remove them.--health-cmd and the other --health-* flags,
--no-healthcheck) on start, run, create and update, which changes
it on a running machine at once.--hostname, -u, -w, --cap-add, --cap-drop, --privileged,
--read-only, --tmpfs, --shm-size, --device, --dns, --dns-search,
--add-host, --ulimit, --oom-score-adj, --stop-signal,
--stop-timeout, --timezone, --log-driver, --init, --sysctl and --secret on start, run and
create, applied at the next start. A list takes none to forget it, a
value an empty string or 0, and --privileged=false and
--read-only=false take those back.--rm on run: the machine is removed when that run ends.nspawn --help and nspawn COMMAND --help print the same information. Errors
are printed as error: ... on standard error and the exit status is 1;
exec and an attached run exit with the status of the program they ran.
Every command but daemon, completions and the unit hooks is a call to the
service on the system bus, which asks polkit
whether the caller may take the action: the listings (images ls, ps,
machines ls, inspect, top, stats, events, network ls,
network inspect, volume ls, secret ls, secret inspect) ask for
org.nspawn.inspect; everything else, search and hub included, asks for
org.nspawn.manage. Both are for administrators by default, and root is never
asked.
These are accepted by every command and can also come from the environment:
| Option | Environment | Meaning |
|---|---|---|
--registry REGISTRY | NSPAWN_REGISTRY | Registry (hub) for image references without a host part. |
--ca-cert FILE | NSPAWN_CA_CERT | Extra CA certificate (PEM) to trust when talking to the registry. |
--config FILE | NSPAWN_CONFIG | Configuration file; see Configuration. |
-h, --help | Help. |
login and logout take the registry as their argument, so there --registry
has to come before the subcommand: nspawn --registry hub.example login -u me.
-V, --version belongs to nspawn itself, not to the subcommands.
Query the hub (an OCI registry).
nspawn hub ls [FILTER] [--no-tags]
Lists the repositories of the registry with their tags. FILTER keeps the
repositories whose name contains that text; --no-tags skips the tag query of
every repository, which is faster on big registries. hub list is an alias.
nspawn hub tags REPOSITORY
Prints the tags of one repository, one per line.
nspawn search TERM [--source hub|dockerhub] [-n LIMIT]
Finds images on the hub and on Docker Hub, like docker search, and prints
every hit with its source and the reference pull takes.
| Option | Meaning |
|---|---|
TERM | Text to look for in image names. |
--source hub|dockerhub | Only one source instead of both. |
-n, --limit LIMIT | Results per source. Default: 25. |
nspawn login [REGISTRY] [-u USERNAME] [--password-stdin]
Checks credentials against a registry, like docker login, and keeps them in
/etc/nspawn/auth.json for pull, push, search and hub.
| Option | Meaning |
|---|---|
REGISTRY | Registry host, for example docker.io or hub.nspawn.org. Default: the hub. |
-u, --username USERNAME | User name; asked for on the terminal when missing. |
--password-stdin | Read the password from standard input instead of the terminal. |
nspawn logout [REGISTRY]
Forgets the credentials stored for a registry (the hub by default).
nspawn pull REFERENCE [-n NAME] [--backend BACKEND] [--mode MODE] [-f] [--no-verify]
Downloads an image from the hub or another registry, checking its signature first where the registry has a policy (the hub’s is built in; see Signed images), and makes it available to systemd-machined. On a terminal a bar shows how far each blob got, with its size, speed and time left; in a pipe or a log only the lines are written.
| Option | Meaning |
|---|---|
REFERENCE | [registry/]repository[:tag|@digest], for example fedora:44 or docker.io/library/nginx. |
-n, --name NAME | Local image name. Default: derived from the reference, for example fedora-44. |
--backend auto|overlay|flat|mstack | How to assemble the image on this host. Default: auto, or the backend of the configuration file: overlay, or flat without overlayfs. mstack (systemd 261 or newer, managed user namespaces) is experimental. App images are assembled as overlay even when mstack is chosen. |
--mode auto|boot|app | Whether the image boots an init system or runs a single program. Default: auto. |
-f, --force | Replace an existing image with the same name. |
--no-verify | Skip the signature check: the hub’s images are signed and refused without a valid signature, and the configuration can ask the same of other registries; like docker’s --disable-content-trust. |
nspawn create SOURCE NAME [--backend BACKEND] [--network NETWORK]... [--network-alias [NETWORK=]NAME]...
[-p [IP:]HOST:CONTAINER[/udp]]... [--entrypoint PROGRAM] [-e VAR[=VALUE]]...
[-v SOURCE:TARGET[:ro]]... [-l KEY=VALUE]... [--restart POLICY] [-m SIZE] [--memory-swap SIZE]
[--cpus N] [--pids-limit N] [HEALTHCHECK OPTIONS] [OTHER OPTIONS] [--interface IFACE]... [--secret SECRET]...
[-f] [--no-verify] [-- ARGUMENTS...]
Makes another machine from a local image, like docker create, without
touching the registry; a reference that is not local is pulled first, under
the image’s own name, as run does. The layers are shared with the source.
| Option | Meaning |
|---|---|
SOURCE | Local image to start from: its name, or the reference it was pulled from. A reference that is not local is pulled. |
NAME | Name of the new machine. |
--backend BACKEND | How to assemble it. Default: like the source. |
--network NETWORK | Networks of the new machine, as for start. Default: the source’s kind; a network made with network create is not inherited, as ports and labels are not. |
--network-alias [NETWORK=]NAME | Other names of the machine on its networks, as for start. |
-p, --publish [IP:]HOST:CONTAINER[/udp] | Ports to publish on the host, like start -p. Not inherited from the source. |
--entrypoint PROGRAM | Replace the image’s entrypoint; an empty string runs the arguments alone. App images only. |
-e, --env VAR[=VALUE] | Environment for the program, VAR=value or VAR copied from the calling shell, like docker -e. App images only. |
-v, --volume SOURCE:TARGET[:ro] | Mount a host directory or a named volume, like docker -v. |
-l, --label KEY=VALUE | Label the machine, on top of the image’s own labels, like docker --label. Not inherited from the source. |
--restart, -m, --memory-swap, --cpus, --pids-limit | Restart policy and limits, as for start. Not inherited from the source. |
the healthcheck options, the other options, --interface, --secret | As for start. Not inherited from the source. |
-f, --force | Replace an existing machine with the same name. |
--no-verify | Skip the signature check of an image that has to be pulled (a local source is not checked); like docker’s --disable-content-trust. |
-- ARGUMENTS... | App images: replace the image’s cmd; they follow its entrypoint, as with docker. |
nspawn build [DIRECTORY] -t TAG [-n NAME] [-d DISTRIBUTION] [-r RELEASE]
[--profile PROFILE]... [--backend BACKEND] [--mode MODE] [-f]
[--keep-output] [-- MKOSI_ARGS...]
Builds an image with mkosi and makes it available locally, ready to push. Needs mkosi.
| Option | Meaning |
|---|---|
DIRECTORY | Directory with the mkosi configuration (mkosi.conf, mkosi.conf.d, …). Default: .. |
-t, --tag TAG | Reference for the result, for example myapp:1 or hub.example/team/app:2. Required. |
-n, --name NAME | Local image name. Default: derived from the tag. |
-d, --distribution DISTRIBUTION | Distribution to build (mkosi --distribution). |
-r, --release RELEASE | Release to build (mkosi --release). |
--profile PROFILE | mkosi profile to enable; repeatable. |
--backend, --mode, -f | As for pull. |
--keep-output | Keep the mkosi output directory instead of deleting it after the import. |
-- MKOSI_ARGS... | Extra arguments passed to mkosi verbatim. |
nspawn push IMAGE [--to REFERENCE]
Uploads a local image to the hub or another registry, with the credentials
stored for it. IMAGE is a local image name, or the reference it was pulled
from or built as. --to pushes it under a different reference than the one
recorded for the image. The destination needs a tag, not a digest. On a
terminal a bar shows how far each blob got, as with pull.
Manage local images.
nspawn images ls [--json]
Lists the local images known to systemd-machined: name, type, and for the ones
nspawn installed the backend, origin (pull, build or create), source
reference, size and whether the image is read-only. --json prints the list as
the service returns it. images list is an alias.
nspawn images rm NAME...
Removes local images and the layers and blobs nobody uses any more. Refuses
the image of a running machine, or of one its restart policy is bringing back,
and a name nothing is behind. Named volumes are kept. A pulled image is a machine too, so this is the same as
rm without --force.
nspawn rm [-f] NAME...
Removes machines, like docker rm: the record, the tree, the unit files, the
boot link a restart policy made, and the layers and blobs nobody else uses.
Every name is tried; one that cannot be removed, or that nothing is behind
(no machine or image named NAME), is reported at the end. Named volumes are
kept, and each one kept is mentioned.
| Option | Meaning |
|---|---|
-f, --force | Stop a running (or restarting) machine first, with SIGKILL, instead of refusing it. |
nspawn ps [-a] [--json]
nspawn machines ls [-a] [--json]
Lists the running machines, like docker ps: name, image, mode, command,
state, uptime, leader PID, network and OS. A machine whose restart policy is
bringing it back shows as restarting, even without -a. -a, --all also
lists the nspawn machines that are not running: stopped, or starting and
closing while their unit comes up or goes down. --json prints what the service returns for
each machine, its whole record included. machines list is an alias of
machines ls.
nspawn inspect NAME...
Prints everything nspawn knows about machines or images, running or not, as a
JSON array with one object per name, like docker inspect: the record (image
reference, digest, backend, mode, networks, addresses, aliases, ports, volumes,
environment, command, labels, restart policy, limits, healthcheck, secrets,
the other flags, and who signed the image and when, as the pull verified it:
signed_by, signed_at), for a running machine its state, start time, leader
PID, OS and health, and for a stopped one the exit code of its last run. The keys are
those of the
D-Bus interface.
nspawn start NAME [--network NETWORK]... [--network-alias [NETWORK=]NAME]... [-p [IP:]HOST:CONTAINER[/udp]]...
[--entrypoint PROGRAM] [-e VAR[=VALUE]]... [-v SOURCE:TARGET[:ro]]... [-l KEY=VALUE]...
[--restart POLICY] [-m SIZE] [--memory-swap SIZE] [--cpus N] [--pids-limit N]
[--health-cmd COMMAND] [--health-interval D] [--health-timeout D] [--health-retries N]
[--health-start-period D] [--health-start-interval D] [--no-healthcheck]
[--hostname NAME] [-u USER[:GROUP]] [-w DIR] [--cap-add CAP]... [--cap-drop CAP]... [--privileged]
[--read-only] [--tmpfs PATH[:OPTIONS]]... [--shm-size SIZE]
[--device HOST[:CONTAINER[:PERMISSIONS]]]... [--dns ADDRESS]... [--dns-search DOMAIN]...
[--add-host HOST:IP]... [--ulimit NAME=SOFT[:HARD]]... [--oom-score-adj N]
[--stop-signal SIGNAL] [--stop-timeout SECONDS] [--timezone MODE] [--log-driver DRIVER] [--init] [--sysctl KEY=VALUE]...
[--interface IFACE]... [--secret NAME[:TARGET[:MODE[:UID:GID]]]]... [--image-command] [--no-wait]
[-- ARGUMENTS...]
Boots an image as a machine. Every option is remembered for the next start.
| Option | Meaning |
|---|---|
--network NETWORK | Network of the machine: bridge, the default network (the default); the name of a network made with network create; veth, a veth pair configured by systemd-networkd on the host (booted images only); host, the host’s own network; none, no interface but lo; or container:NAME, the network namespace of that running machine, like docker run --network container:NAME (app images only; see container:NAME). Repeatable for several bridge networks, the first one primary: its address is where published ports lead and its gateway the default route, unless it is internal, in which case the first network that is not has the route. veth, host, none and container:NAME go alone. |
--network-alias [NETWORK=]NAME | Another name for the machine on its primary network, or on NETWORK, like docker --network-alias: every member of that network resolves it. Repeatable; none forgets them. |
-p, --publish [IP:]HOST:CONTAINER[/udp] | Publish a port on the host, like docker -p: on every address of the host, or on IP alone (127.0.0.1:8080:80). 8000-8010:8000-8010 publishes a range, one mapping per port. Repeatable; none forgets them all. |
--entrypoint PROGRAM | Replace the image’s entrypoint; an empty string runs the arguments alone. App images only. |
-e, --env VAR[=VALUE] | Environment for the program, VAR=value or VAR copied from the calling shell. Repeatable; none forgets them. App images only. |
-v, --volume SOURCE:TARGET[:ro] | Mount a host directory or a named volume, made on first use with what the image has at TARGET, as docker seeds one. Repeatable; none forgets them. |
-l, --label KEY=VALUE | Label the machine, on top of the image’s own labels. Repeatable; none forgets them. |
--restart no|on-failure|always|unless-stopped | Restart policy, like docker --restart. always and unless-stopped also start the machine at boot; nspawn stop takes an unless-stopped machine off the boot list until the next start. Applied at the next start; update changes it at once. |
-m, --memory SIZE | Memory limit of the whole machine, like docker -m: 512m, 2g (at least 4m), and as much swap again unless --memory-swap says otherwise; 0 removes it. Applied at the next start; update changes it at once. |
--memory-swap SIZE | Memory and swap together, like docker --memory-swap: equal to --memory for no swap, larger for the difference, -1 for swap without a bound; 0 goes back to as much swap again as memory. Needs --memory. Applied at the next start; update changes it at once. |
--cpus N | CPU limit of the whole machine, like docker --cpus: 0.5, 2; 0 removes it. Applied at the next start; update changes it at once. |
--pids-limit N | Most processes and threads the machine may have (at least 16 for a booted machine); 0 removes the limit. Applied at the next start; update changes it at once. |
--health-cmd COMMAND | Command that says whether the machine is healthy, run inside it through /bin/sh -c at every interval, like docker --health-cmd: exit 0 is healthy. Replaces the image’s HEALTHCHECK. update changes it at once. |
--health-interval D | Time between probes: 10s, 1m30s, 500ms. Default: 30s. |
--health-timeout D | Time a probe may take before it counts as failed. Default: 30s. |
--health-retries N | Consecutive failed probes that make the machine unhealthy. Default: 3. |
--health-start-period D | Time after the start during which failed probes do not count. Default: 0s. |
--health-start-interval D | Time between probes during the start period. Default: 5s. |
--no-healthcheck | No probes, whatever the image says. |
--hostname NAME | Hostname inside the machine. Default: its name. A booted machine gets it as its /etc/hostname. |
-u, --user USER[:GROUP] | User the program runs as, a name or a uid (listed in the image’s passwd or not), instead of the image’s, with a group after a colon as docker takes it: a name of the image’s group file or a number, which becomes the primary and only group of the program; a name the image lacks is refused. nspawn resolves both from the image’s passwd and group files through a stand-in for getent, as docker does. App images only. |
-w, --workdir DIR | Working directory of the program, instead of the image’s. App images only. |
--cap-add CAP | Capability to keep on top of the default set (docker’s and SYS_BOOT for an app on a bridge network, systemd-nspawn’s for a machine in a user namespace): NET_ADMIN, CAP_NET_ADMIN, ALL. Repeatable; none forgets them. |
--cap-drop CAP | Capability to drop from the default set. --cap-drop ALL --cap-add X keeps X, as with docker. Repeatable; none forgets them. |
--privileged | Every capability, like docker --privileged; --privileged=false takes it back. |
--read-only | Mount the machine’s root read-only; --read-only=false takes it back. |
--tmpfs PATH[:OPTIONS] | An empty tmpfs at a path inside (/tmp:size=64m,mode=1777). One that lands on /run, a tmpfs of every machine already, is left out with a note. Repeatable; none forgets them. |
--shm-size SIZE | Size of /dev/shm: 64m, 1g; 0 for the default. |
--device HOST[:CONTAINER[:PERMISSIONS]] | A device node of the host for the machine, like docker --device (/dev/ttyUSB0:/dev/ttyUSB0:rw), or a directory whose nodes are allowed one by one (/dev/dri). Repeatable; none forgets them. |
--dns ADDRESS | DNS server for the machine, instead of the host’s. Repeatable; none forgets them. |
--dns-search DOMAIN | DNS search domain. Repeatable; none forgets them. |
--add-host HOST:IP | A line for the machine’s /etc/hosts, HOST:IP or HOST=IP (the second for an IPv6 address); host-gateway is the host’s address on the machine’s network. Repeatable; none forgets them. |
--ulimit NAME=SOFT[:HARD] | A resource limit of the program, like docker --ulimit (nofile=1024:4096, core=unlimited). Repeatable; none forgets them. |
--oom-score-adj N | OOM score adjustment of the machine, -1000 to 1000. |
--stop-signal SIGNAL | Signal stop sends the program, instead of the image’s (SIGTERM). |
--stop-timeout SECONDS | Seconds stop waits after the signal before SIGKILL, unless -t says otherwise. Default: 10. |
--timezone MODE | How systemd-nspawn sets the machine’s /etc/localtime at each start, its Timezone=: auto (the default, the host’s zone), off (left alone, so a zone set inside stays), copy, bind, symlink or delete. auto forgets the setting. |
--log-driver DRIVER | Where the program’s output goes, like docker --log-driver: local (the default: journald’s nspawn namespace, a journal of nspawn’s own apart from the system’s, capped at 1 GiB, which logs and an attached run read), journal (the system’s journal) or none (dropped; logs refuses the machine, an attached run of an app still shows the output straight from the program, and a booted machine runs with -d). |
--init | Accepted for docker’s sake: nspawn’s stub init reaps orphans anyway. App images only. |
--sysctl KEY=VALUE | A net.* sysctl for an app machine’s network namespace. Repeatable; none forgets them. |
--interface IFACE | A network interface of the host, moved into the machine while it runs and given back when it stops: an ethernet one, or a wifi adapter with its whole phy (iw on the host for an app on the bridge, systemd 256 for a booted machine); the name is kept inside. Not with --network host or container:NAME; one machine at a time. See Physical interfaces. Repeatable; none forgets them. |
--secret NAME[:TARGET[:MODE[:UID:GID]]] | A secret made with secret create as a read-only file inside the machine, like docker’s --secret: NAME alone is /run/secrets/NAME with mode 0444, root’s. Repeatable; none forgets them. Not on mstack machines. |
--image-command | Forget the remembered entrypoint and arguments and run the image’s own again. |
--no-wait | Do not wait for a booted machine’s init to be up before returning. Its registration is still awaited, so that ports and firewall rules can be applied. |
-- ARGUMENTS... | App images: replace the image’s cmd; they follow its entrypoint, as with docker. |
A path inside the machine, the target of -v or --secret, a --tmpfs or a
--device path, must be plain: a . or .. component is refused.
nspawn run [-d] [--rm] [-i] [-t] [-n NAME] [--pull missing|always|never]
[--backend BACKEND] [--mode MODE] [-f] [--no-verify] [OPTIONS OF start] [--no-wait]
REFERENCE [COMMAND [ARGUMENT...]]
Makes a machine from an image and starts it, like docker run: pull (or
create from a local image with the same reference) followed by start, whose
options are remembered as after start. What pull or create report goes to
standard error, so that standard output carries the program’s output alone.
Without -d, run stays attached, as docker run does. The machine’s output
follows until the machine ends, a line at a time, stdout and stderr together:
it is read from nspawn’s journal, so nspawn logs NAME shows it later too, and
the machine goes on should run be interrupted. With --log-driver none it
comes straight from the program instead, and nothing is kept. run exits with the program’s
exit code, or 128 plus the signal it died of (130 after Ctrl-C, 137 after
kill). Ctrl-C, SIGTERM, SIGHUP and SIGQUIT are passed on to the program; a
third Ctrl-C within a second leaves the machine running and returns. A booted
image shows its console until it powers off, and Ctrl-C powers it off.
| Option | Meaning |
|---|---|
REFERENCE | [registry/]repository[:tag|@digest], for example nginx:1.27. |
COMMAND [ARGUMENT...] | App images: replace the image’s cmd and follow its entrypoint, as with docker; everything after the reference that is not an option of run, or everything after --. |
-d, --detach | Start the machine in the background and return, like docker run -d; nspawn logs -f NAME follows its output. |
--rm | Remove the machine once it ends, with or without -d; named volumes stay. An image the run had to pull is kept under the image’s local name when that name is free, as docker keeps images, and without --name the machine gets a name of its own (alpine-3-1f0c9a2e). Refused with a restart policy. A run that fails to start leaves nothing behind. |
-i, --interactive | App images: give the program this standard input, like docker run -i. |
-t, --tty | App images: give the program a terminal, like docker run -t; Ctrl-C and resizes reach it through the terminal. Closing the terminal stops the machine. With -i on a booted image: wait for it to boot, open a root shell, and power it off when the shell ends, with the shell’s exit code. Not with -d. |
-n, --name NAME | Name of the machine. Default: derived from the reference, for example nginx-1.27. A name that is taken is refused, with a pointer to start. |
--pull missing|always|never | When to ask the registry. missing, the default: a local image with the same reference is made into the machine, as create does, and the registry is asked only when there is none. always: every time, for the image the tag names now. never: a local image or nothing. |
--backend auto|overlay|flat|mstack | How to assemble the machine, as for pull. |
--mode auto|boot|app | As for pull; a mode other than auto always pulls. |
-f, --force | Make the machine anew when one of that name exists; it must be stopped. |
--no-verify | Skip the signature check of the image, as for pull. |
the options of start | --network, --network-alias, -p, --entrypoint, -e, -v, -l, --restart, -m, --cpus, --pids-limit, the healthcheck options, the other options, --interface, --secret and --no-wait (with -d only), as for start. Options may come before or after the reference, as long as they come before the command. |
nspawn stop NAME... [-f] [-t TIMEOUT] [--no-wait]
Powers off running machines, one after the other: the machine’s stop signal
(--stop-signal, else the image’s) to the program of an app, a poweroff
request to a booted machine. Stopping a machine that already ended only cleans
up after it, and a paused one is thawed first. A machine with a restart policy
stays stopped, and one that was waiting to be restarted is stopped too. An
unless-stopped machine is also taken off the boot list until the next
start; an always one stays enabled and starts again at the next boot (rm
takes that away). Every name is tried; one that could not be stopped is
reported at the end.
| Option | Meaning |
|---|---|
-f, --force | Kill every process at once instead of asking the machine to stop. |
-t, --timeout TIMEOUT | App images: seconds to wait after the stop signal before killing the machine. Default: the machine’s --stop-timeout, else 10. |
--no-wait | Return right after the stop request, without waiting for the machine to be gone and without the kill after the timeout. |
nspawn restart NAME... [-t TIMEOUT]
Stops and starts machines, like docker restart: a stop followed by
a start with everything the machine remembers, and prints each name
once it runs again. A machine that is not running is started. Every name is
tried; one that could not be restarted is reported at the end.
| Option | Meaning |
|---|---|
-t, --timeout TIMEOUT | As for stop. |
nspawn kill [-s SIGNAL] NAME...
Sends a signal to running machines, like docker kill, and prints each name
once it is sent. SIGKILL, the default, stops the machine for good, as
stop --force does. Any other signal goes to the program of an app or the
init of a booted machine, and the machine lives on unless the signal ends it,
in which case its restart policy applies; the machine’s own stop signal (an
app’s, or SIGRTMIN+3 and SIGRTMIN+4 for a booted machine’s init) ends it for
good, as docker does.
| Option | Meaning |
|---|---|
-s, --signal SIGNAL | A name (KILL, SIGHUP, RTMIN+3) or a number. Default: KILL. |
nspawn pause NAME...
nspawn unpause NAME...
pause freezes every process of running machines through the cgroup freezer
of their units, like docker pause, and unpause thaws them; each name is
printed once done. ps and inspect show a frozen machine as paused, and
stop and kill thaw it first.
nspawn update NAME... [--restart POLICY] [-m SIZE] [--memory-swap SIZE] [--cpus N] [--pids-limit N]
[--health-cmd COMMAND] [--health-interval D] [--health-timeout D] [--health-retries N]
[--health-start-period D] [--health-start-interval D] [--no-healthcheck]
Changes the restart policy, the limits and the healthcheck of machines, like
docker update, and prints each name once changed. A running machine gets the
new limits and healthcheck at once (its probes start afresh) and the policy for
its next ending; a stopped one at its next start. The options are those of
start; at least one is needed.
nspawn exec MACHINE [-u USER] [-e VAR[=VALUE]]... [-w DIR] [-T] [-t] [-i] [-d] COMMAND...
Runs a command inside a running machine of either kind, attached to the
terminal when standard input and output are one, in the machine’s namespaces,
with the image’s environment and the machine’s -e variables. The program is found on the machine’s PATH and
runs with the machine’s capabilities and resource limits, like its own
processes, except CAP_SYS_BOOT in an app that shares the host’s user namespace
(unless --privileged), since the command does not carry the seccomp filter that
keeps kexec out of the machine; right after a start it waits, a few seconds at most, until
systemd-nspawn has finished confining the machine. Exits with the command’s
status.
| Option | Meaning |
|---|---|
-u, --user USER[:GROUP] | User inside the machine, a name or a number of its passwd file, with a group of its group file after a colon (then the only group; otherwise the user’s supplementary groups come along) and the home of the passwd entry. Default: root. |
-e, --env VAR[=VALUE] | A variable for the command, VAR=value or VAR copied from the calling shell, like docker exec -e. Repeatable. |
-w, --workdir DIR | Working directory of the command, instead of the machine’s. |
-T, --no-tty | No terminal, even from one: pipes, as in a script. Without -t or -T, a terminal only when standard input and output are one, so redirected output is byte-exact. |
-t, --tty | A terminal for the command, even without one here. |
-i, --interactive | Accepted for docker’s sake: the command’s input is always this one. |
-d, --detach | Leave the command running in the background and return at once. |
COMMAND... | Command and arguments. |
nspawn shell MACHINE [-u USER]
Opens an interactive shell inside a running machine as USER (default
root): machined’s login session for booted machines, /bin/sh in the
machine’s namespaces for app machines.
nspawn top MACHINE
Lists the processes of a running machine, like docker top: PID as the
host sees it, USER as the machine sees it, TIME of CPU and COMMAND,
read from the machine’s cgroup and PID namespace.
nspawn cp MACHINE:PATH DESTINATION
nspawn cp SOURCE MACHINE:PATH
Copies files and directories between the host and a machine, like
docker cp. The machine may be running, or stopped if it is an overlay or flat
machine; a stopped mstack machine has no tree on the host until it runs.
/ must be a directory.DIR/. as the source copies the contents of DIR instead of DIR itself.cp. Modes and
modification times are kept./proc, /sys and the like).MACHINE: starts at the machine’s root. A local path
with a colon is written ./a:b.nspawn logs MACHINE... [-f] [-n N] [--since WHEN] [--until WHEN] [-t] [--all] [--inside]
Shows what machines printed, like docker logs. With several machines every
line carries its machine’s name (web | ...), the way docker compose shows
them. The lines come from nspawn’s journal namespace (--log-driver local, the
default) and from the system’s journal, which holds what a machine wrote before
1.8.0 or with --log-driver journal. A machine started with --log-driver none
keeps nothing, and logs says so (a booted one’s own journal is still there
with --inside).
| Option | Meaning |
|---|---|
-f, --follow | Keep printing new output; starts from the last 10 lines unless --lines says otherwise. |
-n, --lines N | Only the last N lines. |
--since WHEN | Only output newer than this, in journalctl --since syntax, for example "10 min ago" or -1h. |
--until WHEN | Only output older than this, in journalctl --until syntax; nothing is followed then. |
-t, --timestamps | Prefix every line with its timestamp. |
--all | Also show what systemd says about the machine’s service: start, stop, failures. |
--inside | Booted machines only: read the machine’s own journal instead of its console output. |
nspawn stats [NAME...] [--no-stream] [--json]
Shows what running machines use, like docker stats: CPU (100% is one CPU
busy), memory in use against the limit (the host’s memory without one),
network and disk traffic, and processes, read from the cgroup of each machine’s
unit, which holds the whole machine. Without names it shows every running
machine. The table is drawn again every second on a terminal.
| Option | Meaning |
|---|---|
--no-stream | Print one table, from two readings a second apart, and return. |
--json | One JSON object per machine and reading instead of the table. |
nspawn events [--since WHEN] [--until WHEN] [-f KEY=VALUE]... [--json]
Reports what happens to machines, networks, volumes and secrets, like
docker events: for machines start, die (with its exit code), stop,
restart, oom and fail, which systemd logs for every machine however it
was started, health_status (with the new status) and what nspawn does,
pull, build, create, push, kill, update and remove; for
networks, volumes and secrets create and remove. It reads the journal, so
past events can be read back.
| Option | Meaning |
|---|---|
--since WHEN | Events since this time, in journalctl --since syntax ("2026-09-24 10:00", -1h, today). Without it, only new ones. |
--until WHEN | Stop at this time instead of waiting for new events; needs --since. |
-f, --filter KEY=VALUE | Only matching events: name=NAME, type=machine|network|volume|secret, event=ACTION, label=KEY or label=KEY=VALUE. The same key given twice matches either value, different keys must all match. |
--json | One JSON object per event: time, time_usec, type, action, name, attributes, labels. |
The default network (bridge) and the ones made with network create.
nspawn network up
Brings up every network’s bridge with its NAT rules and firewall exceptions,
and brings the published ports in line with the machines that run. start
does it for its machine’s network too; this is useful at boot and for
troubleshooting.
nspawn network ls [--json]
Lists the networks, the default one first: name, bridge interface, subnet,
whether it is internal, and the machines whose records name it. --json
prints them as the service returns them. network list is an alias.
nspawn network create NAME [--subnet CIDR] [--internal] [-l KEY=VALUE]...
Makes a network of its own, like docker network create: a bridge nsbr-NAME
(hashed when the name is long) with its NAT and firewall rules, brought up at
once. Machines of one network reach each other by name, and nothing of another
network, the default one included, reaches them; ports they publish are
reachable from everywhere through the host, as from the LAN.
| Option | Meaning |
|---|---|
NAME | Letters, digits, _ and -; not bridge, host, veth, none or default. |
--subnet CIDR | The network’s IPv4 subnet. Default: the next free /24 of network_pool (see Configuration), never one the host already routes. |
--internal | No way out: its machines reach each other and the host, nothing beyond, and cannot publish ports. |
-l, --label KEY=VALUE | Label the network, like docker network create --label. Repeatable. |
nspawn network inspect NAME...
Prints networks as a JSON array, like docker network inspect: name,
interface, subnet, gateway, whether it is internal, its labels, and its
machines with their addresses, aliases, ports and whether they run. bridge
is the default network.
nspawn network rm NAME...
Removes networks no machine uses, with their bridges and rules. A network a
machine’s record names is refused, with the machines that use it; the default
network cannot be removed. network remove is an alias.
nspawn network prune [-f]
Removes every network no machine uses, after asking on standard input unless
-f is given.
Named volumes, the directories -v NAME:/path makes under
/var/lib/nspawn/volumes. Removing a machine keeps them.
nspawn volume ls [--json]
Lists the named volumes with the machines whose records mount them, when each
was made and its path. --json prints the list as the service returns it.
volume list is an alias.
nspawn volume create NAME
Makes a volume ahead of its first use (start makes it otherwise): a
directory owned by root, mode 0755. One that exists already is fine. Names have
letters, digits, _, . and -, not starting with a dot.
nspawn volume rm NAME...
Removes volumes no machine uses. A volume a machine still names is refused
until that machine is started with other volumes (or -v none) or removed.
nspawn volume prune [-f]
Removes every volume no machine uses once a yes comes on standard input, as
docker does: from a script, whose input ends without one, nothing is removed.
-f removes them without asking.
Secrets, like docker’s without a swarm: kept encrypted on the host with
systemd-creds, handed to machines as files by --secret on start, run
and create.
nspawn secret ls [--json]
Lists the secrets: name, when it was made, the size of its plaintext, its
labels and the machines that take it; never the content. --json prints the
list as the service returns it. secret list is an alias.
nspawn secret create NAME [--file FILE] [-l KEY=VALUE]...
Keeps a secret read from standard input, or from --file, encrypted for this
host: bound to its TPM2 where there is one, to its credential key otherwise,
under /var/lib/nspawn/secrets. A name that exists already is refused.
| Option | Meaning |
|---|---|
NAME | Letters, digits, _, . and -. |
--file FILE | Read the content from this file instead of standard input. |
-l, --label KEY=VALUE | Label the secret, like docker secret create --label. Repeatable. |
nspawn secret inspect NAME...
Prints what nspawn keeps about secrets as a JSON array: name, creation time, size, labels and the machines that take them; never the content.
nspawn secret rm NAME...
Removes secrets no machine takes. One a machine’s record names is refused,
with the machines that take it, until they are started with other secrets (or
--secret none) or removed. Every name is tried; one that cannot be removed is
reported at the end. secret remove is an alias.
nspawn daemon [--install] [--idle-exit SECONDS]
Serves org.nspawn on the system bus. The bus starts it on demand, so this is
not a command to type; --install is.
| Option | Meaning |
|---|---|
--install | Write the bus policy, the polkit actions, the activation file and the unit that let the bus start this binary on demand, then return. A package does the same. |
--idle-exit SECONDS | Exit after this long without a call or a running job. Default: 60; 0 keeps serving. |
nspawn completions bash|elvish|fish|powershell|zsh
Writes the completions for that shell on standard output. Commands and flags come from the same definition the command line itself is built from; besides them, TAB completes names, asked from the service as it goes:
| After | TAB offers |
|---|---|
stop, exec, kill, pause, unpause, top, shell, stats | the running machines |
start | the images that do not run |
rm, inspect, logs, restart, update, images rm | every machine and image |
create, push | the local images |
run | the references of the local images |
--network | the networks, and host, none and veth |
network rm, network inspect | the networks |
volume rm | the volumes |
The service hands those names to any user without asking polkit, so this works
under sudo and without a password; secrets are not completed. The flags come
once the word starts with - (- brings the short forms, -- the long ones),
or where there is no name to offer. The packages install the completions, so
this is for a binary you built yourself:
nspawn completions bash > ~/.local/share/bash-completion/completions/nspawn
man nspawn is the same reference as this page, generated the same way.
nspawn network prepare NAME, nspawn network publish NAME and
nspawn network release NAME are what the drop-in of
systemd-nspawn@NAME.service runs as ExecStartPre, ExecStartPost and
ExecStopPost, and nspawn health-run NAME is what
nspawn-health-NAME.service runs to probe a machine’s healthcheck. They are
not meant to be typed and are hidden from --help.
A hub for systemd-nspawn images and the tool that uses them. The team builds
images of common Linux distributions with mkosi,
publishes them on an OCI registry at hub.nspawn.org, and maintains nspawn,
the command line tool documented on this site. The
mkosi definitions of the images
are public.
Yes, version 255 or newer. nspawn drives systemd-nspawn through systemd-machined and the systemd service manager, so the host needs both, with cgroup v2. The images on the hub contain systemd as well; images without an init system run as apps.
No. The commands and flags look alike on purpose (-p, -e, -v,
--entrypoint, create, exec, logs), but the machines are systemd-nspawn
containers managed by systemd: they are systemd-nspawn@NAME.service units,
appear in machinectl list, log to journald and boot a full init when the
image has one. nspawn’s own service holds none of them open, there is no compose
file and no orchestration.
Yes. sudo nspawn search nginx finds them, and
sudo nspawn pull docker.io/library/nginx:latest --name web fetches one like
any other registry image. Since it has no init system, it is installed as an
app image: its entrypoint runs under nspawn’s stub init, on the bridge, with
-p for its ports. Docker Hub limits anonymous pulls per address;
sudo nspawn login docker.io -u USER lifts that. See
Getting started.
Not necessarily. Every command goes through the service on the system bus, and
the service asks polkit about the caller: root is never asked, an administrator
is asked for a password (a desktop session prompts; in a plain terminal, start
pkttyagent first), and a rule of your own can hand the actions to a group so
that nobody is asked at all. The work does need privileges, since the machines
live below /var/lib/machines and the bridge is the host’s, which is why the
service is the one holding them.
On a host without polkit the service answers root alone. See The service for the two actions and the rule.
Where the work happens. The command line is its client, the way machinectl
and systemctl are clients of machined and systemd: every command is a method
call, and pulls, pushes, builds, creates, removals and copies come back as job
objects that report their output. The bus starts the service when a command arrives and it exits
after a minute without work, so nothing of nspawn’s runs in the background
otherwise. A package installs it; for a binary you built yourself,
sudo nspawn daemon --install. Its journal is journalctl -u nspawn.service,
and the interface is described in docs/DBUS.md of the repository. After an
upgrade the service keeps running the binary it started with until it goes
idle or is restarted; a command of another version says so and asks for
sudo systemctl restart nspawn.service (the rpm and the deb restart it
themselves).
Yes. The workflow that builds the images of the hub, in
nspawn/mkosi-definitions, signs
each one twice with cosign after pushing
it: keyless, with the identity of that workflow on master (a short-lived
certificate from Fulcio, recorded in the Rekor transparency log), and with the
project’s key, whose public half is cosign.pub in that repository. The
signatures are stored on the hub next to the image, as OCI referrers of its
digest, so they cover every tag that points to it, and the hub verifies the
key one itself and shows the image as signed.
nspawn pull verifies them too, since 1.5.0: before a single layer is
downloaded, one of the two signatures has to verify offline against the
project’s key and the workflow’s identity, both built into nspawn, and an
image without a valid signature is refused (--no-verify skips the check for
one command; see Signed images). Other
registries get a policy in the configuration.
To verify an image yourself, with cosign:
cosign verify \
--certificate-identity https://github.com/nspawn/mkosi-definitions/.github/workflows/mkosi.yml@refs/heads/master \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
hub.nspawn.org/fedora:44
or, with the key from the repository, cosign verify --key cosign.pub hub.nspawn.org/fedora:44. Every blob is also checked against the sha256
digest in the manifest while it downloads, and registries are reached over
HTTPS only.
The first generation of nspawn was a shell wrapper around machinectl that
downloaded tar.xz and raw.xz images from hub.nspawn.org/storage, signed
with the nspawn.org master key and verified by systemd-importd. Its last
release was 0.6; version 1.0 replaced it with the OCI based client described
here: images are OCI images on
a registry, layers are shared, and machinectl is no longer involved. The
history of the old script is still in the
nspawn repository.
Yes. nspawn creates the bridge, its addresses and its nftables rules itself,
and hands each machine its address through a generated file or a prepared
network namespace. Neither systemd-networkd nor NetworkManager on the host is
involved; only --network veth depends on systemd-networkd. With firewalld the
bridge is put in the trusted zone, and on hosts where docker or ufw set the
forward policy to drop, nspawn adds the exception the bridge needs. See
Firewalls.
exec enters the machine’s namespaces, like docker exec: nothing is needed
inside, the exit code comes back and the program is found on the machine’s
PATH. shell opens an interactive shell: machined’s login session for a
booted machine, which needs D-Bus inside, or /bin/sh in the namespaces for an
app.
With volumes, as in docker: -v /srv/data:/data mounts a host directory,
-v pgdata:/var/lib/postgresql a named volume that nspawn keeps under
/var/lib/nspawn/volumes/pgdata. Named volumes survive rm and images rm. See
Volumes.
Yes: sudo nspawn create fedora-44 db makes another machine that shares the
layers of fedora-44 and has a writable layer, an address, ports and settings
of its own. See More machines from one image.
Yes. sudo nspawn start NAME --restart always (or unless-stopped) enables the
unit and also restarts the machine when it ends; systemctl enable systemd-nspawn@NAME.service (or machinectl enable NAME) works too. Either way
the drop-in nspawn installs on the unit prepares the network and publishes the
ports whoever starts the machine.
sudo nspawn ps shows it, and sudo nspawn network inspect bridge (or the
name of another network) lists the machines of a network with their addresses. Inside the machines, the other
machines are reachable by name and the host as host.nspawn.internal; on hosts
with systemd 258 or newer, machined resolves the machine names on the host as
well.
sudo nspawn rm NAME (or images rm) removes a machine or image, and
afterwards every layer and blob that no remaining image references. Overlay
machines keep their writes in a private directory under /var/lib/nspawn,
which goes away with the image; named volumes stay until sudo nspawn volume rm NAME or sudo nspawn volume prune.
For questions, #nspawn-org on Matrix
or on Libera.Chat. For bugs, the issues
of github.com/nspawn/nspawn for the
tool, mkosi-definitions
for the images, and nspawn.org for
this site.
nspawn is developed on GitHub under the nspawn organization:
| Repository | What it holds |
|---|---|
| nspawn | The tool (Rust). |
| mkosi-definitions | The mkosi configuration of the images on the hub. |
| nspawn.org | This site. |
| nspawn.github.io | The blog. |
Issues and pull requests are welcome in all of them. The code of conduct applies everywhere.
A Rust toolchain of version 1.85 or newer is enough for the unit tests and clippy, which need no root and touch nothing outside temporary directories:
cargo fmt
cargo clippy --all-targets -- -D warnings
cargo test
cargo build --release
The repository documents itself: docs/HACKING.md for building and testing,
docs/ARCHITECTURE.md for how the pieces fit, docs/DBUS.md for the interface
the service exposes, and packaging/ for the Fedora, Arch and Debian packages
and the SELinux policy. A pull request runs the checks and builds those
packages.
A binary you built has to be installed before it can serve:
sudo install -Dm755 target/release/nspawn /usr/local/bin/nspawn followed by
sudo nspawn daemon --install. Replacing the binary of a running service is
fine; the next command starts the new one.
The end-to-end test runs the built binary against a real registry and systemd-machined, as root, on a host with systemd-nspawn:
NSPAWN=./target/release/nspawn sudo -E tests/e2e.sh
It exercises the whole tool: hub ls, search on the hub and on Docker Hub,
login and logout, a pull with both the overlay and the flat backend,
booting, exec through the PTY, volumes, the bridge network with names and
published ports between two machines, create from a local image, layer
sharing and garbage collection, an app from Docker Hub with entrypoint,
environment and volumes, machinectl start through the unit hooks, programs
that exit on their own or ignore their stop signal, and (when mkosi is
installed) a build, push and pull round trip. NSPAWN_REGISTRY (and
NSPAWN_CA_CERT for a private CA) must point at a registry that serves the
image named in IMAGE, fedora:44 by default; the Docker Hub steps need
internet access.
Keep pull requests focused, add a unit test when the change has logic that can be tested without a host, and run clippy before sending.
The images are plain mkosi configuration trees. Changes to what the hub ships,
new distributions or releases go to
mkosi-definitions. A definition
can be tried locally with nspawn build -t test/NAME:1 DIRECTORY before
sending it.
The site is built with Hugo and the Docsy theme. With Docker installed, a live preview needs nothing else:
git clone https://github.com/nspawn/nspawn.org.git
cd nspawn.org
docker compose up
Then open http://localhost:1313. The container runs as uid and gid 1000; the
README of the repository explains how to pass yours if they differ. Without
Docker, install Node.js, Go and git and run npm run install:safe followed by
npm run serve. The pages are Markdown files under content/en/; every
documentation page has an “Edit this page” link that opens the file on GitHub.