Skip to content

Record installation defaults in the status instead of the spec - #5203

Open
caseydavenport wants to merge 11 commits into
tigera:masterfrom
caseydavenport:casey-installation-defaults
Open

Record installation defaults in the status instead of the spec#5203
caseydavenport wants to merge 11 commits into
tigera:masterfrom
caseydavenport:casey-installation-defaults

Conversation

@caseydavenport

@caseydavenport caseydavenport commented Aug 17, 2026

Copy link
Copy Markdown
Member

Description

The operator writes its computed defaults back into the Installation spec, which makes the operator the field manager for anything it defaulted. Helm and Argo then conflict on those fields, and a server-side apply from the chart gets rejected. See #5102.

This changes the defaulting to leave the spec alone:

  • computed defaults are recorded on the status instead of the spec, as a sparse object holding only the fields the user left unset
  • defaulting seeds from the recorded values first, so a default that changes between operator versions doesn't change an existing cluster
  • a field drops out of the recorded set as soon as the user sets it, and the effective config is still published in the status as before
  • every other controller reads that effective config and waits until the core controller has published it, so there is one place that merges the spec, the defaults and the overlay
  • the write that persists finalizers and migrated Calico config stays, so an upgrade from a manifest install is unaffected

Also fixes a bug this uncovered: an empty ipPools list marshalled the same as an unset one, so asking for no IP pools got the default pool instead. The documented behavior now holds.

IP pool semantics, spelled out in the CRD docs as part of this:

  • ipPools: [] means no pools. The operator deletes the pools it owns and creates none.
  • Omitting ipPools leaves the choice to the operator. On a cluster where it has already recorded pool defaults, that keeps the pools in use, where before it deleted them.

First-install pool defaulting takes two status round-trips: the IP pool controller records the pool defaults, then the core controller republishes the effective config with them included. A few FV timeouts went up to cover that window.

Existing clusters where the operator already owns spec fields need one helm upgrade --force-conflicts to hand ownership back to the chart.

Constants that could be plain CRD schema defaults are left alone here - that's a follow-up.

Related: CORE-13376

Fixes server-side apply conflicts on Installation fields between the operator and the tool that manages the resource, such as Helm or Argo CD. The operator records the values it defaults in the Installation status rather than writing them into the spec.
Fixes an issue where an empty list of IP pools in the Installation was treated as unset, causing a default IP pool to be created.

@marvin-tigera marvin-tigera added this to the v1.45.0 milestone Aug 17, 2026
@caseydavenport
caseydavenport marked this pull request as ready for review August 18, 2026 12:34
@caseydavenport
caseydavenport requested review from a team and marvin-tigera as code owners August 18, 2026 12:34
Comment thread pkg/controller/installation/core_controller.go
Comment thread pkg/controller/installation/core_controller.go Outdated
Comment thread pkg/controller/ippool/pool_controller.go Outdated
Comment thread pkg/controller/ippool/pool_controller.go Outdated
Comment thread pkg/controller/ippool/pool_controller.go Outdated
Comment thread pkg/controller/utils/utils.go Outdated
Comment thread pkg/controller/utils/utils.go Outdated
Comment thread test/gatewayapi_test.go Outdated
Defaulting seeds from the recorded values, so a user-set field takes over
and drops out of the recorded set.
Downstream controllers read the computed spec and wait until the core controller publishes it.
The pool list merges whole, so a declared pool that omits the name dropped the
recorded default for it. Per-pool fields now come from the recorded default with
the same CIDR, and the IP pool controller diffs its defaults against the raw spec.
Each controller declares the spec paths it defaults, and downstream controllers read the computed spec published on the status.
Keeps the object we write back exactly as the user declared it, so the
patch and the status writes don't need to restore the spec afterwards.
Records IP pool defaults per declared pool instead of all-or-nothing, so a user's own pools stay theirs.
Stops the API server injecting field defaults into the status, which caused a rewrite every reconcile.
@caseydavenport
caseydavenport force-pushed the casey-installation-defaults branch from 41e8c6d to 0e3a5d8 Compare August 20, 2026 13:12

@tmjd tmjd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the code for having the IPPools managed by the ippool controller is quite complicated. I'm not a big fan of that.

// We rely on the core controller for defaulting, so wait until it has done so before continuing
if reflect.DeepEqual(instanceStatus, operatorv1.InstallationStatus{}) {
err := fmt.Errorf("InstallationStatus is empty")
r.status.SetDegraded(operatorv1.ResourceNotReady, "InstallationStatus is empty", err, reqLogger)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use to degrade in this case, should we do the same now when status.Computed is not set?

return nil
}

// Write a copy, so the response can't replace the effective spec we're working from.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's interesting that this is needed. I wouldn't have thought that Status().Update would not have updated the main spec in what was passed to it.

}

// Write default IP pool configuration back to the Installation object using patch.
preDefaultPatchFrom := client.MergeFrom(installation.DeepCopy())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any idea why we patched before? I'm wondering if we should still be patching for updating the defaults

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants