Summary
We build coreutils (ls, stat, chmod, …) for wasm32-wasip1. Because that
target is not cfg(unix), every crate's #[cfg(unix)] filesystem code —
the code that reads real st_mode permission bits — is compiled out, and the
#[cfg(not(unix))] fallback (which fabricates permissions from a single
readonly boolean) is compiled in. To get native-Linux-accurate output we've
had to hand-patch each tool to bypass that fallback and fetch mode bits from the
sidecar via a host_fs.path_mode import (landed in #268 / #269).
This issue tracks the larger question: should we make the WASM build target
present as cfg(unix) (a Linux-like target) so these tools "just work" with no
per-tool patches, instead of maintaining a growing pile of #[cfg(target_os = "wasi")] shims?
Why a non-Linux target breaks filesystem tools
uucore has two implementations of display_permissions, cfg-gated:
#[cfg(unix)] // reads REAL bits
pub fn display_permissions(md, ...) -> String {
display_permissions_unix(md.mode() as mode_t, ...) // st_mode
}
#[cfg(not(unix))] // WASI lands HERE — fabricates
pub fn display_permissions(md, display_file_type) -> String {
let write = if md.permissions().readonly() { '-' } else { 'w' };
format!("{file_type}r{write}xr{write}xr{write}x") // r & x HARDCODED on
}
Upstream root cause (uutils/coreutils, uucore 0.5.0):
https://github.com/uutils/coreutils/blob/main/src/uucore/src/lib/features/fs.rs
Consequences of being cfg(not(unix)):
md.mode() isn't even callable — std::os::unix::fs::MetadataExt doesn't
exist in the WASI std.
- Rust's own
std::os::wasi::fs::MetadataExt deliberately exposes
dev/ino/nlink/size/atim/… but no mode(), because WASI preview1's
filestat struct has no mode field. So the bits are absent at the std layer
regardless of cfg.
- Net effect:
ls -l printed -rwxrwxrwx (or -r-xr-xr-x if readonly) for
every file — a uniform string derived from one boolean.
Note this is degradation, not deliberate stripping: the tools have "WASI
support," it's just a lossy fallback nobody wired to a host that has the bits.
Evidence: the per-tool workarounds this forces (all at 8145b07)
Each of these exists only because the target isn't cfg(unix):
ls — inject a host_fs.path_mode import + mode_for_path and swap
display_permissions → display_permissions_unix:
|
+#[cfg(target_os = "wasi")] |
|
+use uucore::fs::mode_t; |
|
+ |
|
+#[cfg(target_os = "wasi")] |
|
+mod wasi_host_fs { |
|
+ use std::env; |
|
+ |
|
+ use super::{Metadata, Path}; |
|
+ |
|
+ mod host_fs { |
|
+ #[link(wasm_import_module = "host_fs")] |
|
+ unsafe extern "C" { |
|
+ pub fn path_mode( |
|
+ fd: u32, |
|
+ path_ptr: *const u8, |
|
+ path_len: u32, |
|
+ follow_symlinks: u32, |
|
+ ) -> u32; |
|
+ } |
|
+ } |
|
+ |
|
+ fn fallback_mode(metadata: &Metadata) -> u32 { |
|
+ if metadata.is_dir() { 0o040755 } else { 0o100644 } |
|
+ } |
|
+ |
|
+ fn raw_mode_for_path(path: &Path, follow_symlinks: bool) -> u32 { |
|
+ let Some(path_str) = path.to_str() else { |
|
+ return 0; |
|
+ }; |
|
+ unsafe { |
|
+ host_fs::path_mode( |
|
+ 3, |
|
+ path_str.as_ptr(), |
|
+ path_str.len() as u32, |
|
+ if follow_symlinks { 1 } else { 0 }, |
|
+ ) |
|
+ } |
|
+ } |
|
+ |
|
+ pub fn mode_for_path(path: &Path, metadata: &Metadata, follow_symlinks: bool) -> u32 { |
|
+ let mode = raw_mode_for_path(path, follow_symlinks); |
|
+ if mode != 0 { |
|
+ mode |
|
+ } else if path.is_relative() { |
|
+ env::current_dir() |
|
+ .ok() |
|
+ .map(|cwd| raw_mode_for_path(&cwd.join(path), follow_symlinks)) |
|
+ .filter(|mode| *mode != 0) |
|
+ .unwrap_or_else(|| fallback_mode(metadata)) |
|
+ } else { |
|
+ fallback_mode(metadata) |
|
+ } |
|
+ } |
|
+} |
|
|
|
mod dired; |
|
use dired::{DiredOutput, is_dired_arg_present}; |
|
@@ -2982,7 +3036,15 @@ |
|
let is_acl_set = false; |
stat:
https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/crates/uu_stat/0001-wasi-metadata-compat.patch
chmod:
https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/crates/uu_chmod/0001-wasi-compat.patch
- First-party shims hit the same wall —
which and the shell builtins each
carry their own #[cfg(target_os = "wasi")] host_fs extern:
|
#[cfg(target_os = "wasi")] |
|
mod host_fs { |
|
#[link(wasm_import_module = "host_fs")] |
|
unsafe extern "C" { |
|
// Signature must match the sidecar host_fs.path_mode |
|
// (dir_fd, path_ptr, path_len, follow_symlinks). |
|
pub fn path_mode( |
|
dir_fd: u32, |
|
path_ptr: *const u8, |
|
path_len: u32, |
|
follow_symlinks: u32, |
|
) -> u32; |
|
} |
|
} |
|
|
|
fn print_usage<W: Write>(out: &mut W) -> io::Result<()> { |
|
writeln!(out, "Usage: which [-a] name [...]") |
|
} |
|
|
|
fn is_executable_path(path: &Path) -> bool { |
|
let Ok(metadata) = fs::metadata(path) else { |
|
return false; |
|
}; |
|
|
|
metadata.is_file() && executable_mode_bits(path, &metadata) |
|
} |
|
|
|
#[cfg(unix)] |
|
fn executable_mode_bits(_path: &Path, metadata: &fs::Metadata) -> bool { |
|
(metadata.mode() & 0o111) != 0 |
|
} |
|
|
|
#[cfg(target_os = "wasi")] |
|
fn executable_mode_bits(path: &Path, _metadata: &fs::Metadata) -> bool { |
|
let path_string = path.to_string_lossy(); |
|
let bytes = path_string.as_bytes(); |
|
let Ok(path_len) = u32::try_from(bytes.len()) else { |
|
return false; |
|
}; |
|
// dir_fd 3 = cwd preopen; absolute paths ignore it. |
|
let mode = unsafe { host_fs::path_mode(3, bytes.as_ptr(), path_len, 1) }; |
|
(mode & 0o111) != 0 |
|
} |
|
#[cfg(target_os = "wasi")] |
|
mod host_fs { |
|
#[link(wasm_import_module = "host_fs")] |
|
unsafe extern "C" { |
|
// Signature must match the sidecar host_fs.path_mode |
|
// (dir_fd, path_ptr, path_len, follow_symlinks). |
|
pub fn path_mode( |
|
dir_fd: u32, |
|
path_ptr: *const u8, |
|
path_len: u32, |
|
follow_symlinks: u32, |
|
) -> u32; |
|
} |
|
} |
- And the C side already has the mirror-image fix in wasi-libc (which Rust can't
reach, because Rust std bypasses libc stat() on wasip1):
https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/wasi-libc/0016-host-fs-mode-and-chmod.patch
Every new fs-touching Rust tool we add will need another such patch.
Options
A. Force cfg(unix) on the current target (cheap, does NOT work)
Passing --cfg unix via RUSTFLAGS flips the cfg on our crates, but the
#[cfg(unix)] path does use std::os::unix::fs::MetadataExt, which the
precompiled WASI std doesn't contain → unresolved import. And even if it
resolved, the WASI Metadata's underlying filestat has no st_mode to
return. Rejected.
B. Custom Linux-like target spec + -Z build-std (the real "treat it like native")
Define a target derived from wasm32-wasip1 with target-family = ["unix", …]
and recompile std/core so Rust's fs routes through the libc-backed unix fs
backend (calls libc stat(), reads st_mode). Then the wasi-libc mode patch
(0016) flows up into Rust automatically and all #[cfg(unix)] tool code works —
no per-tool patches. Blast radius, however, is large:
- Nightly + build-std on every build; per-Rust-version maintenance of a bespoke
target.
cfg(unix) flips the entire dep graph, not just perms — signals, process,
users/groups (getpwuid), termios, mmap, net — much of which calls libc
symbols WASI libc doesn't implement → link/ENOSYS breakage to chase.
- The fs backend (
target_os = "wasi") and MetadataExt (target_family = unix) are gated on different cfgs and don't cleanly compose; effectively a
std fork.
C. Move the runtime to WASI preview2 / wasi:filesystem (the clean long-term fix)
preview2 carries richer metadata than preview1's filestat. Migrating (or
upstreaming a wasi MetadataExt::mode() backed by an extended filestat) would
let these tools read real bits natively and let us delete all the patches —
without pretending to be unix.
Ask / decision needed
Decide between: (B) invest in a Linux-like custom target + build-std so fs tools
need zero patches, vs. (C) target preview2, vs. (status quo) keep adding small
per-tool host_fs patches. Leaning C long-term; B is the "make it native" ask
but has broad blast radius. Capturing so the tradeoff is explicit before the
patch pile grows.
Related PRs: #268 (filesystem native-parity: wasi-libc + sidecar host_fs),
#269 (coreutils stat/chmod/ls real permission bits).
Summary
We build coreutils (
ls,stat,chmod, …) forwasm32-wasip1. Because thattarget is not
cfg(unix), every crate's#[cfg(unix)]filesystem code —the code that reads real
st_modepermission bits — is compiled out, and the#[cfg(not(unix))]fallback (which fabricates permissions from a singlereadonlyboolean) is compiled in. To get native-Linux-accurate output we'vehad to hand-patch each tool to bypass that fallback and fetch mode bits from the
sidecar via a
host_fs.path_modeimport (landed in #268 / #269).This issue tracks the larger question: should we make the WASM build target
present as
cfg(unix)(a Linux-like target) so these tools "just work" with noper-tool patches, instead of maintaining a growing pile of
#[cfg(target_os = "wasi")]shims?Why a non-Linux target breaks filesystem tools
uucorehas two implementations ofdisplay_permissions, cfg-gated:Upstream root cause (uutils/coreutils, uucore 0.5.0):
https://github.com/uutils/coreutils/blob/main/src/uucore/src/lib/features/fs.rs
Consequences of being
cfg(not(unix)):md.mode()isn't even callable —std::os::unix::fs::MetadataExtdoesn'texist in the WASI
std.std::os::wasi::fs::MetadataExtdeliberately exposesdev/ino/nlink/size/atim/…but nomode(), because WASI preview1'sfilestatstruct has no mode field. So the bits are absent at thestdlayerregardless of cfg.
ls -lprinted-rwxrwxrwx(or-r-xr-xr-xif readonly) forevery file — a uniform string derived from one boolean.
Note this is degradation, not deliberate stripping: the tools have "WASI
support," it's just a lossy fallback nobody wired to a host that has the bits.
Evidence: the per-tool workarounds this forces (all at
8145b07)Each of these exists only because the target isn't
cfg(unix):ls— inject ahost_fs.path_modeimport +mode_for_pathand swapdisplay_permissions→display_permissions_unix:secure-exec/registry/native/patches/crates/uu_ls/0001-wasi-host-fs-mode-display.patch
Lines 16 to 74 in 8145b07
stat:https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/crates/uu_stat/0001-wasi-metadata-compat.patch
chmod:https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/crates/uu_chmod/0001-wasi-compat.patch
whichand the shell builtins eachcarry their own
#[cfg(target_os = "wasi")]host_fsextern:secure-exec/registry/native/crates/libs/shims/src/which.rs
Lines 16 to 58 in 8145b07
secure-exec/registry/native/crates/libs/builtins/src/lib.rs
Lines 14 to 27 in 8145b07
reach, because Rust std bypasses libc
stat()on wasip1):https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/wasi-libc/0016-host-fs-mode-and-chmod.patch
Every new fs-touching Rust tool we add will need another such patch.
Options
A. Force
cfg(unix)on the current target (cheap, does NOT work)Passing
--cfg unixvia RUSTFLAGS flips the cfg on our crates, but the#[cfg(unix)]path doesuse std::os::unix::fs::MetadataExt, which theprecompiled WASI
stddoesn't contain → unresolved import. And even if itresolved, the WASI
Metadata's underlyingfilestathas nost_modetoreturn. Rejected.
B. Custom Linux-like target spec +
-Z build-std(the real "treat it like native")Define a target derived from
wasm32-wasip1withtarget-family = ["unix", …]and recompile
std/coreso Rust's fs routes through the libc-backed unix fsbackend (calls libc
stat(), readsst_mode). Then the wasi-libc mode patch(0016) flows up into Rust automatically and all
#[cfg(unix)]tool code works —no per-tool patches. Blast radius, however, is large:
target.
cfg(unix)flips the entire dep graph, not just perms — signals, process,users/groups (
getpwuid), termios, mmap, net — much of which calls libcsymbols WASI libc doesn't implement → link/
ENOSYSbreakage to chase.target_os = "wasi") andMetadataExt(target_family = unix) are gated on different cfgs and don't cleanly compose; effectively astd fork.
C. Move the runtime to WASI preview2 /
wasi:filesystem(the clean long-term fix)preview2 carries richer metadata than preview1's
filestat. Migrating (orupstreaming a wasi
MetadataExt::mode()backed by an extended filestat) wouldlet these tools read real bits natively and let us delete all the patches —
without pretending to be unix.
Ask / decision needed
Decide between: (B) invest in a Linux-like custom target + build-std so fs tools
need zero patches, vs. (C) target preview2, vs. (status quo) keep adding small
per-tool
host_fspatches. Leaning C long-term; B is the "make it native" askbut has broad blast radius. Capturing so the tradeoff is explicit before the
patch pile grows.
Related PRs: #268 (filesystem native-parity: wasi-libc + sidecar host_fs),
#269 (coreutils
stat/chmod/lsreal permission bits).