CPU_RUNTIME_CACHE_CAPACITY is unsupported in OVMS plugin_config but works in raw OpenVINO Runtime

[Isochronism]

The config key CPU_RUNTIME_CACHE_CAPACITY is valid and functional when used directly with OpenVINO Runtime:

config = {
    "CPU_RUNTIME_CACHE_CAPACITY": "0",
    "PERFORMANCE_HINT": "THROUGHPUT"
}

However, if I put this key inside the plugin_config block of OVMS model config JSON, the server fails to start and throws an error log in terminal:
CPU_RUNTIME_CACHE_CAPACITY not found in supported config keys for device: CPU.

OVMS config snippet I used:

"config": {
  "name": "vocos",
  "base_path": "./models/vocos",
  "target_device": "CPU",
  "plugin_config": {
    "CPU_RUNTIME_CACHE_CAPACITY": "0",
    "PERFORMANCE_HINT": "THROUGHPUT"
  }
}

Is there an equivalent supported configuration in OVMS to disable the CPU runtime cache (set CPU_RUNTIME_CACHE_CAPACITY to 0)?

[dtrawins]

@Isochronism Parameter CPU_RUNTIME_CACHE_CAPACITY is used only in internal API. It is not exposed in public API.
It is applied to reduce memory usage with dynamic shapes?
@mg-intel @yuxu42 can you comment what are the alternatives in public API?

[azhai219]

@dtrawins

• No public API is alternatives.
• It controls the low level primitive items. Default value is 5000, which should be enough. So far, I haven't identified any typical use cases to expose it as a public API. If there is such typical case, we consider to publish it.
• I see the PR Allow CPU_RUNTIME_CACHE_CAPACITY in plugin_config #4326 is to support such API, which should work as an experimental try.
• In short, it’s a trade-off rather than a technical issue.