8. Firecracker Volumes

Hint

Abstract

This chapter explains how a local host path is made available inside of a Firecracker VM application, which commands create and delete that setup and how the result looks in the flake configuration.

A Firecracker VM has no way to share a directory with the host directly, unlike a container it does not use the host kernel and therefore cannot bind mount a host path into the guest. flake-pilot bridges this gap with NFS: a local path is exported by the host and mounted by the guest over the private network described in Firecracker Networking. The setup is created and deleted with the flake-ctl firecracker volume commands, no manual NFS configuration is needed.

The setup works within the following requirements:

  • The application has to be connected to the host network with flake-ctl firecracker network add, see Firecracker Networking. The volume setup reads the gateway address from that network setup and the address of the VM from the address allocations of the host.

  • The local path has to be owned by a user other than root. Every access of the VM to the volume is performed as that user.

  • The guest needs a working network setup at boot and has to provide the NFS client tools, in particular the mount.nfs helper program.

Managing volumes is a two step process: a local path is exported through NFS to the VM of an application, and it is added to the flake configuration of that application, which mounts it at a path of its choice. A path can be exported to and added to more than one application or instance.

8.1. Export a Local Path

flake-ctl firecracker volume export \
    --path /some/local/path --app $HOME/bin/claude

Writes a flake-pilot managed entry for the given path to /etc/exports which allows exactly the VM of the given application as a client. The VM is identified by its address in the private firecracker network, e.g 172.16.0.2, read from the address allocations in /var/lib/firecracker/network-state.yaml, which flake-ctl firecracker network add writes. The ip= boot option in the flake configuration is not used for this, the configuration of a user belongs to that user and could name the VM of another user, the state file belongs to root. If the nfs-server systemd service is not running yet it is started, otherwise the export table is reloaded so the new export becomes effective right away.

An instance of the application which was connected to the network with its own address, see Firecracker Networking, needs an export of its own:

flake-ctl firecracker volume export \
    --path /some/local/path --app $HOME/bin/claude --instance @id1

Exporting the same path for further applications or instances adds their VMs to the same entry, exporting it again for a VM which is already allowed keeps the entry unchanged. An entry written by former versions of flake-pilot for the whole private network is replaced.

The command expects an existing directory which is owned by the calling user. Called as root, the directory has to be owned by a user other than root, a directory owned by root is never exported. The path must not lead through a symbolic link, the command names the resolved path to use instead. As it changes a system wide NFS configuration, the export is created by the helper, see Privileges and Trust, which checks the directory, its owner and the address of the VM again.

Every access of the VM to the volume is performed as the owner of the exported path. The entry maps all users of the VM, including root, to the user and group ID of the exported path (all_squash with anonuid and anongid). The application inside of the VM can therefore read and write the volume like its owner can, but it gets no further permissions on the host, in particular files it creates belong to the owner and not to root. Changing the ownership of a file on the volume fails for the same reason, see Troubleshooting and Known Issues for this and for applications which expect to run as the owner of their files.

Note

The export alone is not enough if a firewall is active on the host, it has to allow the NFS traffic of the private firecracker network as well. See Troubleshooting and Known Issues for the services to open up with firewalld.

8.2. Release a Local Path

flake-ctl firecracker volume release --path /some/local/path

Removes the flake-pilot managed NFS export entry for the given path from /etc/exports and restarts the nfs-server service so the updated export table becomes effective. Only entries previously managed by flake-pilot are touched, an export created by other means is left in place.

To withdraw the volume from one VM only, name its application, and its instance if it has an address of its own:

flake-ctl firecracker volume release \
    --path /some/local/path --app $HOME/bin/claude --instance @id1

The entry is removed once no VM is left in it.

Note

Releasing a path does not check whether it is still in use by a flake. Remove it from every flake it was added to first, see below, before releasing the export.

8.3. Add a Volume to an Application

flake-ctl firecracker volume add \
    --app $HOME/bin/claude --volume /some/local/path:/some/guest/path

Looks up the flake configuration of the application from its path and writes an nfs= boot option to it in the form:

nfs=172.16.0.1:/some/local/path:/some/guest/path

The server part of the option is not configurable, it is the gateway address of the private network the application is connected to, 172.16.0.1, read from the rd.route= boot option written by flake-ctl firecracker network add. This is why the network setup has to exist before a volume can be added.

The --volume option can be given more than once to add several volumes in one call:

flake-ctl firecracker volume add --app $HOME/bin/claude \
    --volume /some/local/path:/some/guest/path \
    --volume /other/local/path:/other/guest/path

As every instance needs its own volume setup if it should differ from the application, the command can be called with --instance as well:

flake-ctl firecracker volume add --app $HOME/bin/claude \
    --volume /some/local/path:/some/guest/path --instance @id1

The volume(s) of an instance take the place of the volumes configured for the application itself, they are not merged. Calling the application without the @id1 selector still mounts the volumes configured globally, if any.

A volume is identified by its guest path: adding a volume with a guest path that is already configured replaces the entry, e.g to point it at a different local path or, after the path was exported again, a different server.

Note

The command only changes the flake configuration of the application, no privileged operations are required. The local path itself still has to be exported to the VM of the application or instance, see above, or the mount inside of the guest fails.

8.4. Remove a Volume From an Application

flake-ctl firecracker volume remove \
    --app $HOME/bin/claude --volume /some/local/path:/some/guest/path

Reverts what volume add has performed: the matching entry is deleted from the nfs= boot option, and the option itself is deleted once no volume is left. --volume can be repeated the same way as with add, and --instance removes the volume(s) of that instance only. A volume is matched by its host and guest path, no matter which server it is currently provided from.

Removing a volume does not release its NFS export nor does it change the network setup, both may still be needed by other flakes or instances. Release the export explicitly once it is no longer used by any application.

8.5. The Result in the Flake Configuration

The flake configuration for the registered claude app from Claude AI as a Firecracker VM App can be found at:

vi ~/.config/flakes/claude.yaml

Adding a volume to the application and another one to its @id1 instance leads to the following settings, on top of the network setup from Firecracker Networking:

vm:
  runtime:
    firecracker:
      boot_args:
        - ip=172.16.0.2::172.16.0.1:255.255.255.0::eth0:off
        - rd.route=172.16.0.1/24::eth0
        - nameserver=8.8.8.8
        - nfs=172.16.0.1:/some/local/path:/some/guest/path
      instance:
        "@id1":
          boot_args:
            - ip=172.16.0.3::172.16.0.1:255.255.255.0::eth0:off
            - nfs=172.16.0.1:/other/local/path:/other/guest/path

More than one volume of the same section is folded into a single nfs= option as a comma separated list, which is the format the init process of the guest, sci, reads the volumes from. It mounts each of them with mount -t nfs before the application is started, creating the guest path if it does not exist yet. A volume which cannot be parsed, or which fails to mount, is skipped, the remaining volumes are still mounted. For details refer to man 8 sci.

Note

As with the network instance section, the @ character is reserved in YAML and the key has to be quoted. The plain name without the @ prefix, e.g id1, is accepted as a key as well.