Skip to content

Mount live virtio-fs shares inside the sandbox container - #294

Merged
Pedro Henrique Penna (ppenna) merged 3 commits into
devfrom
fix/issue-290-virtiofs-sandbox
Oct 1, 2026
Merged

Pedro Henrique Penna (ppenna) merged 3 commits into
devfrom
fix/issue-290-virtiofs-sandbox

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Closes #300. Part of #290 (requirement 1 and the NVX side of requirement 4).

What

Sandbox mode can now live-share a host directory into the workload container, removing the stage-then-copy-back step described in the issue.

  • CLI: sandbox run / sandbox provision accept --mount GUEST_TARGET,HOST_PATH[,ro|rw] (default ro) and repeatable --mount-deny HOST_PATH, forwarded to OpenVMM's existing microVM virtio-fs export, which enforces the access mode and denied paths on the host side. --mount is rejected for start/exec/stop/deprovision; --mount-deny requires --mount; virtfs_* tokens in --cmdline are reserved; and the OpenVMM-appended bootstrap tokens count toward the 1024-byte sandbox command-line budget.
  • Guest (nvx-init-agent): after overlay assembly and workload-identity checks, the agent validates the target (absolute, canonical; /, /etc, and the /proc, /sys, /dev, /.nvx-agent trees are reserved because the container runtime mounts or binds over them), creates it inside the container rootfs one component at a time while refusing symbolic links from a layer, and mounts virtiofs with <ro|rw>,nosuid,nodev before nvx-container-launch enters the private mount namespace (so it survives unshare --mount and chroot). Any failure exits with status 125 instead of starting the workload without its share. A one-shot exit, a managed stop, and any failure after mounting all unmount the share before the overlay or power-off. In sandbox mode the init agent now supervises nvx-managed-agent, which returns on stop instead of powering the VM off itself.
  • Managed lifecycle: the share is persisted in config.json and reattached on every start. Mounted configurations use config format 2 so an older NVX refuses them rather than silently dropping the share; format 1 configs (no share) still load. On stop, the init agent unmounts the share, overlay, layers, and scratch before powering off; previously a managed stop powered off with all of them mounted. This also fixes managed sandbox exec, which always failed with status 125 on dev because /run/nvx/workload-machine-id was written only for one-shot runs.
  • CI (run-nvx-microvm-tests.yml, Linux and Windows): runs /sbin/nvx-sandbox-smoke against an rw /workspace share with a denied subdirectory, verifies the guest's file and directory reach the host, then against an ro /opt/hostedtoolcache share and requires writes to fail. A managed sandbox repeats the rw check through provision/start/exec/stop and must report success with a cleanly unmounted scratch filesystem (ext4 journal-recovery flag clear).
  • Docs: doc/run.md, doc/usage.md, doc/ci.md, doc/project-structure.md.

Validation

  • python -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_microvm_tests.py scripts/test_development_release.py (390 tests), scripts/test_adversarial.py, ruff check/format, pyright (Linux + Windows), shellcheck + shfmt on guest scripts: all pass.
  • New tests cover parsing/validation, OpenVMM argument forwarding, the command-line budget, persistence/replay (including format mismatches), and a POSIX-shell execution of the real nvx-init-agent share functions (mount placement, reserved/non-canonical targets, symlink refusal, teardown order, fatal unmounting the share). A Linux test compiles nvx-managed-agent.c with the production flags and drives the control protocol over a pty; sandbox-mode STOP must return exit 0 (the previous agent exits 125).
  • Local end-to-end on Windows/WHP with a patched initramfs: rw share read/write round-trip as UID 65534, nvx-denied hidden, ro share rejected touch with Read-only file system, /bin/share (symlinked /bin in Ubuntu) failed closed with status 125, and a managed provision/start/exec/stop/deprovision cycle with a share (exec succeeds, the outcome is success, and scratch is clean; with the dev guest, exec fails with 125 and scratch is left needs_recovery). The new Windows CI steps also passed locally. KVM/MSHV are covered by CI.

Not in this PR (needs OpenVMM / nanvix/openvmm changes and a pin promotion)

Accept --mount GUEST_TARGET,HOST_PATH[,ro|rw] and repeatable --mount-deny
for sandbox run and provision, and forward them to OpenVMM's microVM
virtio-fs export. The host-side export enforces the access mode and the
denied paths.

After the guest agent assembles the container overlay and verifies the
workload identity, it creates the target inside the container root one
component at a time, refuses symbolic links, and mounts the share with
nosuid,nodev before the workload enters its private mount namespace.
Invalid targets, reserved runtime paths, and mount failures abort the
sandbox instead of starting the workload without its share. Teardown
unmounts the share before the overlay.

Managed sandboxes persist the share in a new configuration format so
older NVX releases refuse it rather than start without the share. CI now
round-trips guest writes through a read-write share, hides a denied
subdirectory, and requires a read-only share to reject writes.

Part of #290.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 1, 2026 06:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Managed shutdown bypasses the new ordered share-unmount path.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds live virtio-fs host-directory sharing to sandbox workloads.

Changes:

  • Adds CLI validation, OpenVMM forwarding, and managed-state persistence.
  • Mounts and validates shares inside guest container root filesystems.
  • Adds documentation, unit tests, and cross-platform smoke coverage.
File Description
scripts/​nvx.py Adds sandbox mount CLI options.
scripts/​nvx_tools/​sandbox.py Implements mount validation and launch arguments.
scripts/​nvx_tools/​sandbox_lifecycle.py Persists and restores managed mounts.
guest/​common/​nvx-init-agent Mounts and unmounts guest shares.
guest/​common/​nvx-sandbox-smoke Tests guest share behavior.
scripts/​test_nvx_tools.py Covers parsing, persistence, and guest logic.
.github/​workflows/​run-nvx-microvm-tests.yml Adds Linux and Windows smoke tests.
doc/​run.md Documents live-share behavior.
doc/​usage.md Documents CLI options.
doc/​ci.md Documents CI coverage.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread guest/common/nvx-init-agent Outdated
Give config.json its own format constants instead of reusing the
runtime-state format, so the no-share and live-share configuration
formats can evolve independently of runtime.json.

Add regression tests showing that a format-1-only reader rejects a
mounted configuration and that persisting a relative mount path keeps
symbolic-link components for OpenVMM to reject instead of resolving them.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 1, 2026 14:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Managed sandbox shutdown bypasses the new ordered share teardown.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)

A managed sandbox exec'd nvx-managed-agent from nvx-init-agent, and the
agent's stop handler powered the VM off directly. The live share,
overlay, layers, and scratch therefore stayed mounted on every managed
stop, and fatal errors after mounting also powered off with the share
mounted.

In sandbox mode, nvx-init-agent now supervises the managed agent, which
returns after a stop request so both lifecycles share one teardown that
unmounts the share before the overlay, layers, and scratch. fatal
unmounts the share before powering off, and lifecycle validation runs
before the share is mounted. Direct managed mode still powers off from
the agent.

Write the workload machine ID for managed sandboxes as well. The
container entry helper requires it, so every managed sandbox exec
previously failed with status 125.

Cover the teardown order with shell tests, the agent stop path with a
pty-driven test of the compiled agent, and the full managed flow in CI,
where a managed stop must leave the scratch filesystem cleanly
unmounted.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 1, 2026 15:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It changes privileged guest mounting and shutdown behavior across shell, C, Python, and three hypervisor backends.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@ppenna
Pedro Henrique Penna (ppenna) merged commit 63b31d2 into dev Oct 1, 2026
24 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the fix/issue-290-virtiofs-sandbox branch October 1, 2026 16:58
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Oct 2, 2026
dev moved from ae123d1 to 20353f8: bounded egress (#289), the shared
git output runner (#287), Ubuntu manifest errors (#291), the perf gate
history reset (#293), sandbox teardown through the init agent (#294),
virtiofs symlinks with openvmm 2728f33ea (#302), the required CI result
check (#305), and new performance baselines.

- .github/actions/validate-nvx/action.yml: run dev's
  test_egress_policy.py and the time ABI's test_guest_time.py and
  test_time_abi.py.
- scripts/nvx.py: keep the doctor parser import and dev's
  compile_policy_file import.
- scripts/test_nvx_tools.py: keep both the re and select imports.
- scripts/test_microvm_tests.py (no textual conflict): dev's L3/L4
  egress dispatch test expects the Ubuntu guest's default memory, which
  25bac37 raised from 256 to 512 MiB on this branch.

The openvmm gitlink takes dev's 2728f33ea (microvm/v2.1); lockstep
moves it to the time ABI head rebased onto it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sandbox: mount the live virtio-fs share inside the container rootfs

2 participants