fix(desktop): enable Windows mesh-llm builds and address Compute Share startup/MeshLLM debug logging/non-image models trying to parse image input - #3223
Conversation
|
@stevepresley thanks - that is a big change and not all windows related is it? |
It started as mostly Windows related, but these three items are cross-platform.
I'll update the title |
|
@michaelneale opened #3300 to track the 3 other issues fixed in this PR. Let me know if you want me to split it out into a second PR |
Signed-off-by: stevepresley <github@stevepresley.net>
cb8e386 to
006c158
Compare
…m-2836-upstream Signed-off-by: stevepresley <github@stevepresley.net> # Conflicts: # desktop/src-tauri/src/commands/mesh_llm.rs # desktop/src/features/mesh-compute/ui/MeshComputeSettingsCard.tsx
Signed-off-by: stevepresley <github@stevepresley.net>
dcd0ed9 to
c35bbda
Compare
Signed-off-by: stevepresley <github@stevepresley.net>
073672b to
b478dad
Compare
…#4524) ## Summary Official Linux desktop packages (`.deb` / AppImage) are built without `--features mesh-llm`, so they ship the `mesh_llm_stubs` backend and Settings → Compute always fails with `mesh-llm feature not enabled`. This PR adds the feature flag to the two Linux build commands: - `release.yml` → `release-linux` job - `linux-canary.yml` → canary build That's the whole diff — 2 lines. Fixes #3788 (Linux); see also #3841 (dup with UI-gating PR #3914) and the Windows twin #2836/#3223. ## Why no native prebuild step (unlike the macOS job) The macOS job carries Metal llama prebuild/cache steps from #798. Linux doesn't need an equivalent: - `mesh-llm-host-runtime` is compiled with `dynamic-native-runtime` and installs the recommended runtime on first use (verified by sha256 checksum over HTTPS; upstream's signature verification path is not yet implemented — default policy is `RequireChecksum`, per `mesh-llm-runtime-install/src/lib.rs`) (`desktop/src-tauri/src/mesh_llm/mod.rs` — `initialize_mesh_native_runtime`), so release builds work on clean machines without bundling llama.cpp. - Upstream publishes Linux x86_64/aarch64 runtime bundles for the pinned `v0.74.0` line, and `scripts/ensure-mesh-native-runtime.sh` already maps `meshllm-native-runtime-linux-x86_64-cpu` / `linux-aarch64-cpu` for local/e2e use. - The unmerged branch `micn/mesh-node-download` (`96f29417a`) treats even the macOS prebuild steps as removable dead weight for the same reason. ## Background The omission is historical drift, not a decision: Linux packaging predates the mesh feature flag (#693), mesh became opt-in for build-cost/reliability reasons (#823, #1183), and #1221 re-enabled it for releases by editing only the macOS build line. `release-linux` and the later `linux-canary` copy were never revisited. The mesh shutdown hard-exit/relaunch path is gated `all(mesh-llm, target_os = "macos")` because ggml/Metal destructors abort on macOS; ordinary mesh shutdown (`shutdown_mesh_runtime`) is cross-platform, so Linux falls through to the generic path. ## Validation - [x] `./bin/cargo check --manifest-path desktop/src-tauri/Cargo.toml --features mesh-llm` green at base `2c0ac2467` (feature graph compiles at the pinned v0.74.0 line) - [ ] Linux canary run with this change: AppImage/.deb build succeeds and binary contains real `mesh_llm` symbols (not `mesh_llm_stubs`) - [ ] Installed package: cold-start → Settings → Compute → runtime download → serve → clean shutdown The last two need a Linux run/host. **Note (from review):** `linux-canary.yml` is `workflow_dispatch`-only and its `Require main` step rejects non-main refs, so the canary cannot run on this branch pre-merge — and `.github/workflows/**` matches no ci.yml paths-filter, so this PR's own CI does not exercise the changed lines. Validation sequencing is therefore merge → dispatch linux-canary on main → live-package pass, with a trivial 2-line revert as the escape hatch. Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Signed-off-by: stevepresley <github@stevepresley.net>
|
Built and tested this PR on Windows 11 / RTX 5060 Ti. The feature side works. Two things worth The build is correct — verified against a controlBuilt
Binary 147.9 MB vs 86.5 MB. Settings → Compute renders a real panel, detects the GPU, and populates 1. CI blocker:
|
Signed-off-by: stevepresley <github@stevepresley.net>
Signed-off-by: stevepresley <github@stevepresley.net>
…wn bootstrap Our 08-09 fork run installed the MinGW runtime DLLs before the bundler. PR block#3223 now bootstraps them itself (pacman -> choco -> assert). Leaving our step in would let that bootstrap borrow strength from ours and pass without being exercised, so it is removed. Replaced with a read-only probe that installs nothing and records which of the four DLLs the stock windows-latest runner actually has, so a bundler PASS can be attributed to the bootstrap rather than to a runner that already had them. Signed-off-by: jds1-ai <jds1-ai@users.noreply.github.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks @jds1-ai - made the following updates:
Validation performed:
Give it another go and let me know how it works for you. I'm trying to get another box with an nVidia card setup to test locally as well. |
|
Tested 1. The bundler bootstrap works on a stock runner — confirmedI re-ran the canary in a fork with our own MinGW-install workaround removed, so the bootstrap in To make the result attributable I added a read-only probe before the bundler that installs nothing So the runner really was missing PASS on the MSYS2 branch; the choco fallback was not exercised (and so remains untested by this To be precise about what this rests on: the verdict comes from the bundler step's own log, not One incidental effect worth knowing: because (Minor: 2.
|
| DLL | pristine (archive) | after run (bundled) |
|---|---|---|
libstdc++-6.dll |
3529d11c… |
887c21db… |
libgcc_s_seh-1.dll |
278f1101… |
b37c1770… |
libgomp-1.dll |
ae3939c3… |
acf25eee… |
libwinpthread-1.dll |
1bbd639c… |
d54ed5ba… |
The MeshLLM diagnostic log added by this PR confirms it directly — this is the app's own output on
every sharing attempt, for all four DLLs and for both installed runtimes:
windows mesh runtime dependency already present; not replacing
file=…\native-runtimes\0.74.0\meshllm-native-runtime-windows-x86_64-vulkan\lib\libstdc++-6.dll
windows mesh runtime dependency already present; not replacing
file=…\native-runtimes\0.74.0\meshllm-native-runtime-windows-x86_64-cuda12\lib\libstdc++-6.dll
prepared windows mesh runtime dependency dirs=…\Buzz\resources\mesh-llm\windows-x86_64;
…\meshllm-native-runtime-windows-x86_64-cuda12\lib;…-vulkan\lib
Note it skips for cuda12 as well, whose archive ships none of these four — because an earlier
build of this PR, with the unconditional copy, already put them there. That's the sharper form of
the issue: the guard is sticky. It isn't only "the MinGW archive already occupies those
filenames"; it's that once any version has populated them by any means, no later version can
refresh them. A machine that has run today's build will keep today's DLLs indefinitely, even if a
future Buzz bundles corrected ones — the upgrade silently no-ops. That's the part I'd weigh, more
than the current-release behaviour.
To be clear about impact: this breaks nothing today. Every libstdc++-6.dll I have — the
archive's, the bundled one, and Git for Windows' — is missing the same five codecvt exports, so
the substitution was never fixing the 127. And skip-if-present does cure a real hazard I'd flagged
earlier in this thread: the old behaviour would silently clobber a correct archive DLL if a future
release shipped one.
The two halves of 2b16d630 pull against each other, though. Per §1, the bootstrap changes
where the bundled DLLs come from — after pacman fires, libstdc++-6.dll is MSYS2 GCC 16.2.0's
rather than Git for Windows'. I checked whether that newer build helps: parsing its export table, it
is also missing all five symbols (7058 exports), so it would not have fixed the 127 even if it
had been applied. That's why I'm confident the skip costs nothing today.
The structural point is what remains: one half of the commit changes what goes into the bundle, and
the other half guarantees it can never reach a runtime directory that has already been populated. If
a future MeshLLM release ships a corrected libstdc++-6.dll the skip is right; if the intended cure
for the 127 was ever "bundle a good one", the skip closes that door — and closes it hardest on the
machines that already ran an affected build.
The question is just whether "archive is authoritative, the bundle only fills gaps" is the intended
semantics. If it is, this is right and worth a comment saying so. If the bundle was meant to be able
to repair a bad archive DLL, this removes that ability on exactly the two runtimes that are
currently broken — and the debug line reads "already present; not replacing", which is accurate but
easy to skim past when diagnosing a load failure.
3. Nothing in this PR's CI exercises the script
The checks on 2b16d630 are zizmor, Semgrep OSS, and DCO Check. windows-canary.yml is
workflow_dispatch:-only and release.yml runs on release, so bundle-windows-mesh-runtime-deps.ps1
— including the bootstrap just added to it — runs on no check in this PR. A green PR here
carries no signal about the thing that was fixed. Not necessarily worth restructuring CI over, but
worth knowing that the fix is currently only validated by someone running the canary by hand, which
is what this comment is.
4. 7e2644d4 (dedicated VRAM) — confirmed fixed in an installed build; the fallback is not
Confirmed end-to-end. I installed a canary built from 7e2644d4 (verified by hash that it was
actually that build — see §5) on the 16 GB RTX 5060 Ti / 32 GB RAM machine. Settings → Compute now
reads:
Recommended for this machine (NVIDIA GeForce RTX 5060 Ti, 16 GB AI memory).
That's the ~32 GB over-report I raised, fixed, and the recommendation is sized sensibly against it
(a 2.7 GB model, "Fits well"). Marker scan confirms it's a real mesh build rather than the stub:
mesh_llm_stubs 0, mesh-llm-host-runtime 813, iroh 445, 147.9 MB vs stock 0.5.8's 86.5 MB.
The premise in your code comment ("a 16 GB card with 32 GB system RAM reads as ~32 GB") is this
machine exactly, which is why I reported it. Running the two probes the patch uses:
> nvidia-smi --query-gpu=name,memory.total --format=csv,noheader,nounits
NVIDIA GeForce RTX 5060 Ti, 16311
> Get-CimInstance Win32_VideoController | Select-Object Name,AdapterRAM | ConvertTo-Json -Compress
{"Name":"NVIDIA GeForce RTX 5060 Ti","AdapterRAM":4293918720}
The nvidia-smi path gives 16311 MiB and format_rated_capacity renders "16 GB" — correct, and it
fixes the ~32 GB over-report I reported.
The Win32_VideoController fallback reports 4 GB for the same 16 GB card. AdapterRAM is a
uint32, so it saturates at 4293918720 bytes (4095 MiB) and cannot represent any card above 4 GB.
This is a long-standing WMI limitation, not something specific to this machine. So whenever
nvidia-smi isn't usable — not on PATH, non-NVIDIA hardware, or windows_command_output
returning None because the exit status isn't success — the catalog silently swaps a ~2x
over-report for a 4x under-report, and the picker starts recommending models far smaller than the
card can hold. It fails quietly and plausibly, which is the awkward part: 4 GB is a believable
number for a GPU, so nothing looks wrong.
Worth noting that the accompanying unit test can't catch this. Its fixture is:
r#"[{"Name":"GPU A","AdapterRAM":8589934592},{"Name":"GPU B","AdapterRAM":"4294967296"}]"#Both values are unreachable in practice — 4294967296 is exactly one byte past u32::MAX, so
AdapterRAM can never return either. The parser is being tested against input the data source
cannot produce, so it passes regardless of the saturation behaviour. If the fallback is kept, a
fixture of 4293918720 for a large card would be the one that bites.
If DXGI is available to you, IDXGIAdapter::GetDesc / DedicatedVideoMemory is the usual
replacement — it's 64-bit and doesn't need a child process. Failing that, it may be better to return
no VRAM figure at all than a saturated one, since "unknown" degrades more honestly than "4 GB".
Second, smaller point on both paths: they sum VRAM across adapters. parse_nvidia_smi_catalog_output
accumulates total over every row, and the controller path does .map(|(_, bytes)| *bytes).sum().
For a catalog that ranks what will fit, that reproduces the class of error this commit fixes — two
16 GB cards would present as 32 GB and attract a model that fits in neither one alone, absent layer
splitting. summarize_gpu_names already special-cases multi-GPU for display ("N GPUs"), so the
multi-adapter case is clearly in scope. Max-per-adapter is likely the safer figure for
recommendations, with the sum kept for any pooled-placement view. Flagging this one as reasoning
from the code rather than a measurement — this machine is single-GPU, so unlike the points above I
have not actually observed the two-card behaviour.
The console-window half is fixed for the case you described, but not for enabling sharing.
Opening Settings → Compute is now clean — no flashes, and a process watcher confirms the catalog's
probe (nvidia-smi --query-gpu=name,memory.total …, i.e. your windows_nvidia_smi_gpus) spawns
with no conhost attached. That also settles which path produced the 16 GB: the primary one, not
the Win32_VideoController fallback.
Toggling Share this machine still flickers, repeatedly. Watching child processes, the sharing
path spawns all of these with a conhost.exe attached — which is what the visible window is:
nvidia-smi -L + conhost
nvidia-smi --query-gpu=index,compute_cap --format=csv,noheader + conhost
nvidia-smi + conhost
vulkaninfo + conhost
powershell -NoProfile -Command "Get-CimInstance Win32_VideoController | Select-Object -ExpandProperty Name" + conhost
That last one is not your call site — yours is Select-Object Name,AdapterRAM | ConvertTo-Json -Compress. This is the upstream mesh_llm_system::hardware::survey(), which your commit bypasses
for the catalog but which the runtime-resolution path still calls. The set repeats while sharing
retries, which is why it reads as several flickers rather than one — the diagnostic log shows the
frontend polling mesh_node_status every ~4 s and the dependency-preparation block re-running
alongside it, so the survey is not a one-shot at toggle time.
So the fix is real and correctly scoped to what your comment claims; the remaining flashes come from
the upstream survey rather than from catalog.rs. Flagging it mainly so the symptom isn't reported
back at you as "still broken" after this merges.
5. Separate pre-existing bug: windows-canary.yml can upload a stale installer
This one isn't yours — it's in the canary workflow on main, and it matters here because the canary
is how people are going to validate this PR by hand.
Three consecutive green runs on my fork uploaded the same binary:
| run | head | makensis produced |
uploaded | sha256 |
|---|---|---|---|---|
| 4 | 957b5fab |
Buzz_0.5.9-test.4 |
test.4 |
4ccf4da0… |
| 5 | 34859e0c |
Buzz_0.5.9-test.5 |
test.4 |
4ccf4da0… |
| 6 | eaf9aa9f |
Buzz_0.5.9-test.6 |
test.4 |
4ccf4da0… |
Byte-identical, 68,095,414 bytes each. Runs 5 and 6 compiled their own installers and then shipped
run 4's. Two things combine:
- The Cargo cache
path:exclusionremoves the directory entry but not its contents —!desktop/src-tauri/target/**/release/bundle@actions/globneeds a trailing/**— so
bundle/nsis/*.exefrom a previous run is restored into the workspace. Locate NSIS installerthen doesand takes whichever entry comes first, with no check that it's the one this run built.EXE=$(find "$BUNDLE_DIR/nsis" -name '*.exe' -type f | head -1)
So the canary can hand you a green artifact that does not contain the commit you dispatched it for,
with nothing in the run to indicate it. I nearly drew a conclusion about §4 from one of these before
hashing it — the artifact's ID and timestamp are legitimately new each run even when the bytes are
recycled, so only the hash distinguishes them.
What worked in my fork, if it's useful: delete bundle/nsis after the cache restore so exactly one
installer can exist, and have the locate step assert the filename carries the version the run
derived, erroring on more than one match rather than picking. Fixing the glob to
!desktop/src-tauri/target/**/release/bundle/** would address the cache half, but the assertion is
worth having regardless — it's the part that turns a silent wrong answer into a failed step. Happy
to open this as its own issue or PR against main rather than tangling it with yours.
Still outside this PR
With this build installed, enabling sharing on the 5060 Ti fails as expected:
mesh native runtime failed to install or load: load native runtime
meshllm-native-runtime-windows-x86_64-vulkan from
C:\Users\<user>\AppData\Local\mesh-llm\native-runtimes\0.74.0\meshllm-native-runtime-windows-x86_64-vulkan:
LoadLibraryExW failed
That confirms the resolver rejects windows-x86_64-cuda12 on sm_120 and falls through to
vulkan, rather than attempting cuda12 and failing there. I also re-checked whether bundling a
newer libstdc++ could rescue the Vulkan path: after the bootstrap runs, the DLL Buzz bundles is
MSYS2 GCC 16.2.0's, and parsing its export table it is also missing all five symbols (7058
exports). So no available libstdc++-6.dll on this machine satisfies ggml.dll — which, usefully
for §2, means the skip-if-present guard costs nothing today.
Unchanged from my earlier comment, and not an argument against merging: sm_120 is published only
for linux-x86_64-cuda13, so on RTX 50-series the resolver correctly rejects windows-x86_64-cuda12
(gpu_arches = 75,80,86,87,89,90) and falls through to the MinGW runtimes, which don't load. A
reporter on the upstream issue has also now measured that the coordinator excludes v0.74.0 peers
from stage-splits on stage_protocol_generation, so pinning back isn't a workaround either. Expect
#2836 reports to continue from Blackwell users after this merges.
Signed-off-by: stevepresley <github@stevepresley.net>
|
Our comments crossed — mine went up about twenty minutes after yours and doesn't reference it, sorry One thing worth separating out, because your description and the code differ slightly:
That's the right intent, and it settles the question I raised — but So the guard reads as "never replace the archive's files" but behaves as "never replace anything, Not urgent for this PR — as noted above, no On validation:
Happy to keep covering that in the meantime — this machine is a 16 GB Blackwell (RTX 5060 Ti, |
|
Two things — one on
|
| archive | ggml.dll imports |
toolchain | load |
|---|---|---|---|
v0.74.0 vulkan |
libgcc_s_seh-1, libstdc++-6 |
MinGW | 127 |
v0.75.0 vulkan |
libgcc_s_seh-1, libstdc++-6 |
MinGW | 127 |
v0.75.0 cpu |
msvcp140, vcruntime140, vcruntime140_1 |
MSVC | ok |
The switch is cpu-only; I re-downloaded v0.75.0 vulkan to confirm. Details and a matching
retraction upstream: Mesh-LLM/mesh-llm#1215.
What it does and doesn't change here:
- GPU story unchanged.
vulkanis still MinGW and still fails,sm_120is still published only
forlinux-x86_64-cuda13, so everything I reported about Windows + RTX 50-series stands. - A CPU participant looks reachable, but not from here.
Cargo.tomlpins all six mesh crates to
v0.74.0, andskippy_abidiffers across versions (0.1.32 / 0.1.35) and is matched exactly, so
SDK and runtime have to move as a pair. Deliberate work, not a bump — and whether CPU-only sharing
is worth offering is a product call. - One interaction: the v0.75.0
cpuarchive ships no MinGW runtime DLLs at all (it carries
vcomp140.dlland drops thelibprefix), so for that runtime
bundle-windows-mesh-runtime-deps.ps1would have nothing to do. The bundler stays necessary for
vulkan.
Not asking you to widen this PR — it does what it says and I'd merge it. Flagging it because "no
Windows path exists" was my framing and it turns out to be true only of the GPU path.
Signed-off-by: stevepresley <github@stevepresley.net>
fix: enable Windows mesh-llm desktop builds
Closes #2836
Closes #3300
Summary
mesh-llmfeature for Windows release and canary desktop builds.Validation
Validated locally on Windows 11:
cargo fmt --manifest-path desktop/src-tauri/Cargo.toml --allcargo check --manifest-path desktop/src-tauri/Cargo.toml --features mesh-llmpnpm typecheckcargo check -p buzz-agentcargo build -p buzz-agent --release --target x86_64-pc-windows-msvcbash scripts/bundle-sidecars.sh x86_64-pc-windows-msvcpnpm tauri build --target x86_64-pc-windows-msvc --bundles nsis --features mesh-llm --config "{\"bundle\":{\"createUpdaterArtifacts\":false}}"Manual validation:
buzz-agentstarts from the installed app.422caused by replayed tool-result images.Related/out of scope findings
Windows relay-mesh validation also reproduced existing ACP/agent delivery behavior where a model may produce Activity/final text without publishing a DM/channel message, and failed ACP turns can be retried later. That is out of scope for this Windows mesh-llm packaging/runtime PR.
Refs #2698
Refs #2421
Refs #2681