Skip to content

feat(cache): generate a shell-format runtime config alongside runtime.json - #6191

Open
btxu-db wants to merge 3 commits into
fluid-cloudnative:masterfrom
btxu-db:feat/cacheruntime-runtime-sh
Open

feat(cache): generate a shell-format runtime config alongside runtime.json#6191
btxu-db wants to merge 3 commits into
fluid-cloudnative:masterfrom
btxu-db:feat/cacheruntime-runtime-sh

Conversation

@btxu-db

@btxu-db btxu-db commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Ⅰ. Describe what this PR does

Phase 1 of #6176: generate runtime.sh into the runtime ConfigMap alongside runtime.json, so that a cache system without a FUSE client (Mooncake is the motivating case) can source its runtime configuration instead of parsing JSON.

generateRuntimeConfigData now returns both keys, built from the same CacheRuntimeConfig, so the two files cannot drift apart. The script flattens the config into shell exports following the naming convention in #6176:

export RUNTIME_TARGETPATH='/runtime-mnt/cache/default/demo'
export MOUNTS_COUNT='1'
export MOUNT_0_NAME='mc'
export MASTER_ENABLED='true'
export MASTER_NAME='demo-master'
export MASTER_OPTIONS='{"k":"v"}'
export WORKER_TIEREDSTORE_LEVEL_0_QUOTAS_COUNT='1'
export WORKER_TIEREDSTORE_LEVEL_0_QUOTA_0='4Gi'
export CLIENT_ENABLED='false'
...

A few choices worth calling out, since some differ from the example in the issue:

  • Single quotes, not double. The example in the issue uses export KEY="VALUE". Values here are single-quoted with '\'' escaping, so the shell takes them literally: a $, a backtick or a quote in a dataset option cannot expand or execute when the script is sourced.
  • String slices are indexed, not joined. Access modes, tier mount paths and tier quotas are exported as a count plus one variable per element, the same way mounts are. Joining on a comma would be ambiguous, since a host path may legally contain one, and paths and quotas are paired by position, so a stray separator would shift the pairing silently.
  • An absent component still exports every variable, from a zero value. When the topology leaves a component out (the client, in the client-less case), its variables are present and empty, so a consumer running under set -u can read any of them. <COMPONENT>_ENABLED alone tells an absent component from a present one.
  • Maps stay JSON, as the issue specifies. json.Marshal sorts map keys, so the script is byte-stable across reconciles and does not rewrite the ConfigMap on every sync.

The volume, setup and client code needed no change for the client-disabled path. The second commit adds tests that pin the existing behaviour there rather than altering it.

Ⅱ. Does this pull request fix one issue?

Part of #6176 (phase 1). I have not marked it as fixing the issue, because phase 2, the webhook plugin that injects runtime.sh into application Pods, is still to come.

Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.

cm_test.go:

  • TestGenerateRuntimeConfigDataIncludesRuntimeSh: both keys are generated, and the script agrees with the JSON field by field.
  • TestRuntimeShCoversEveryConfigField: walks every field reachable from CacheRuntimeConfig and fails if one is not exported to runtime.sh, or if the export table names a field that no longer exists. Adding a field to the config without exporting it fails this test.
  • TestRuntimeShExportsEveryComponentAlike: the master and the worker, populated identically, export the same variables.
  • TestRuntimeShExportsAbsentComponentsUnderNounset: an absent component exports the same variables as a present one, and the script can be sourced and read under set -u.
  • TestGenerateRuntimeShQuotesHostileValues: quotes, $, backticks, spaces and newlines round-trip through a real sh.
  • TestGenerateRuntimeShIsDeterministic, TestGenerateRuntimeShWithoutAnyEnabledComponent, TestGenerateRuntimeShExportsTieredStoreLevels.

volume_test.go: TestCreateVolumeWithClientDisabled, the PV and PVC are still created when the client is disabled.

sync_test.go: three specs covering how the creation and sync paths treat a disabled client.

The tests that source the script shell out to sh, and skip if it is not available.

Ⅳ. Describe how to verify it

go test ./pkg/ddc/cache/engine/ -run 'RuntimeSh|RuntimeConfigData|ClientDisabled' -count=1

On a cluster, create a CacheRuntime with spec.client.disabled: true and read the script back:

kubectl get cm fluid-runtime-config-<name> -o jsonpath='{.data.runtime\.sh}'

Ⅴ. Special notes for reviews

The in-Pod path of runtime.sh is not defined here. It is left to phase 2, together with its first consumer, the injector.

….json

A cache system without a FUSE client has no way to consume runtime.json
unless its image ships a JSON parser. Generate runtime.sh into the same
ConfigMap, flattening the config into shell exports, so that such a client
can source it directly.

Values are single-quoted so the shell takes them literally whatever the
dataset spec contains, and maps are exported as JSON with sorted keys so the
script stays byte-stable across reconciles. String slices are exported as a
count plus one variable per element rather than joined on a separator: a
mount path or a quota may legally contain one, and paths and quotas are
paired by position, so a stray separator would shift the pairing silently.
A disabled component exports only its ENABLED flag, so consumers can test it
without unset checks.

Part of fluid-cloudnative#6176.

Signed-off-by: btxu-db <btxu-db@outlook.com>
The runtime.sh work targets cache systems that run without a FUSE client.
Pin the behaviour the creation and sync paths already have when the client
component is disabled, so that the no-client topology is covered before
anything else changes around it.

Part of fluid-cloudnative#6176.

Signed-off-by: btxu-db <btxu-db@outlook.com>
@fluid-e2e-bot

fluid-e2e-bot Bot commented Sep 11, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign zwwhdls for approval by writing /assign @zwwhdls in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fluid-e2e-bot

fluid-e2e-bot Bot commented Sep 11, 2026

Copy link
Copy Markdown

Hi @btxu-db. Thanks for your PR.

I'm waiting for a fluid-cloudnative member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.40%. Comparing base (54a41cd) to head (9c06a62).

Files with missing lines Patch % Lines
pkg/ddc/cache/engine/cm.go 79.54% 12 Missing and 6 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6191      +/-   ##
==========================================
+ Coverage   65.24%   65.40%   +0.16%     
==========================================
  Files         486      486              
  Lines       34194    34284      +90     
==========================================
+ Hits        22309    22423     +114     
+ Misses      10135    10097      -38     
- Partials     1750     1764      +14     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

getRuntimeShPath had no caller: the injector that needs the script's
in-Pod path belongs to phase 2, so the helper can come back with it.
Removing it clears the unused (golangci-lint) and U1000 (staticcheck)
failures.

Spell reflect.Ptr as reflect.Pointer in cm_test.go, as the govet inline
check now requires.

Part of fluid-cloudnative#6176.

Signed-off-by: btxu-db <btxu-db@outlook.com>
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant