IMX500 sensor state machine fails ("failed to load loader firmware") during sustained streaming, despite correct power sequencing

[TBC3d]

Environment

• Board: Raspberry Pi 4 Model B Rev 1.5, 8GB RAM
• Camera: Raspberry Pi AI Camera (IMX500)
• RP2040 bridge firmware: ver. 15 (dmesg: rp2040_gbdg_probe() found dev ID ..., fw ver. 15)
• OS: Raspberry Pi OS
• Kernel: reproduced on both 6.18.39+rpt-rpi-v8 and 6.18.34+rpt-rpi-v8
• rpicam-apps with imx500 post-processing (imx500-all package)

Command used

rpicam-hello -t 0s --post-process-file /usr/share/rpi-camera-assets/imx500_mobilenet_ssd.json --viewfinder-width 1920 --viewfinder-height 1080 --framerate 30

Symptom

After a variable amount of running time (observed anywhere from ~2m30s to ~46 minutes across repeated runs, and ~7 minutes at reduced load of 1280x720@15fps), the stream fails with:

ERROR V4L2 v4l2_videodevice.cpp:1787 /dev/video0[10:cap]: Failed to queue buffer X: Resource temporarily unavailable
ERROR RPISTREAM rpi_stream.cpp:267 Failed to queue buffer for Unicam Image
ERROR: Device timeout detected, attempting a restart!!!

This then loops indefinitely, cycling through buffers 0-3 with the same error.

dmesg at time of failure

unicam fe801000.csi: stream on failed in subdev
imx500 10-001a: DD_REF_STS register did not update
imx500 10-001a: imx500_transition_to_standby_wo_network: failed to load loader firmware
imx500 10-001a: imx500_start_streaming failed to transition from program empty state

Followed by a kernel WARNING/backtrace:

WARNING: CPU: 2 PID: xxxx at drivers/media/common/videobuf2/videobuf2-core.c:2222 __vb2_queue_cancel+0x244/0x2e0 [videobuf2_common]
...
videobuf2_common: driver bug: stop_streaming operation is leaving buffer 0 in active state
videobuf2_common: driver bug: stop_streaming operation is leaving buffer 1 in active state
videobuf2_common: driver bug: stop_streaming operation is leaving buffer 2 in active state

What has been ruled out

• Cabling (reseated, confirmed good)
• Power supply — vcgencmd get_throttled = 0x0 at time of failure (checked live, in a parallel terminal, at the exact moment the error occurred)
• Thermal — ~56°C at failure, no throttling
• SD card health — no mmc/fs errors in dmesg, plenty of free space
• CMA memory — dtoverlay=vc4-kms-v3d,cma-512, plenty free
• Reinstalling imx500-firmware/imx500-all packages — no change
• Kernel version — downgraded from 6.18.39 to 6.18.34, same failure (~13 min in)
• Load/resolution — reduced to 1280x720@15fps, still failed (~7 min in)
• A full physical power-cycle (unplug 30s+) temporarily clears the issue — a basic rpicam-hello -t 10000 (no post-processing) then runs cleanly — but the failure recurs after resuming the object-detection post-processing pipeline.

This points to the sensor's internal state machine (as flagged by a Raspberry Pi engineer on the forum) failing during a firmware reload cycle mid-stream, not at initial power-up — the RP2040-mediated loader firmware transfer appears to fail specifically during these later reload attempts rather than the very first one.

Raspberry Pi forum thread with the same symptom family: https://forums.raspberrypi.com/viewtopic.php?t=399493

Happy to provide additional logs (full dmesg, rpicam-hello --verbose output, camera-bug-report output) if useful.

[TBC3d]

Update: the wedge appears to worsen with repeated use, and outlasts short power cycles

Following up on my original report — I spent several more hours today building and testing a custom YOLO11n model exported via imx500-converter/imx500-package to a .rpk, using it through a Picamera2 script with the IMX500 device class.

New observations:

1. The bug is not specific to a particular model or to rpicam-apps vs Picamera2. I reproduced the exact same symptom (Failed to queue buffer: Resource temporarily unavailable, ERROR: Device timeout detected, attempting a restart!!!) across:

   • The original imx500_mobilenet_ssd.json post-process pipeline via rpicam-hello
   • A custom YOLO11n model exported to .rpk via imx500-converter[pt]==3.18.2 + imx500-package, run through Picamera2's IMX500 class in a Python script
   • Multiple official pre-packaged models from /usr/share/imx500-models/ (e.g. imx500_network_efficientdet_lite0_pp.rpk)

   All three fail with the identical error pattern, ruling out anything model-specific or pipeline-specific (CLI vs Python API).

2. A short physical power-cycle (unplug 30s+), which reliably cleared the wedge earlier in the day, stopped being effective after repeated testing cycles. Over the course of ~6 hours of iterative testing (many dozens of start/stop/restart cycles while debugging export tooling, buffer configs, etc.), the time-to-failure got progressively shorter:

   • Early in the session: first failure after ~2m30s–46 minutes of streaming
   • Later in the session: failures within 1–3 seconds of starting the stream, immediately after a 30-second power-cycle, with the RP2040 firmware upload progress showing 0.00 bytes before the buffer errors even start

   This suggests the wedge state may be cumulative/persistent across power cycles once a certain number of firmware reload attempts have occurred, rather than being cleanly reset by a brief power interruption every time.

Environment (same as original report):

• Raspberry Pi 4 Model B Rev 1.5, 8GB RAM
• RP2040 bridge firmware: ver. 15
• Kernel: 6.18.39+rpt-rpi-v8 (also reproduced on 6.18.34+rpt-rpi-v8 earlier)

I haven't yet tried a longer power-off period (10+ minutes) or physically disconnecting the CSI cable during the outage — planning to test that next and will report back if it makes a difference. Wanted to flag the apparent cumulative/worsening behavior in case it's a useful data point for reproducing or narrowing down the root cause.