Skip to content

Add webhook-injector propagation alerts#26

Merged
videlov merged 5 commits into
mainfrom
webhook-injector-propagation-alerts
Jun 26, 2026
Merged

Add webhook-injector propagation alerts#26
videlov merged 5 commits into
mainfrom
webhook-injector-propagation-alerts

Conversation

@videlov

@videlov videlov commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Adds five new alerts to controlplane-remote.yaml covering the propagation-path metrics that webhook-injector exposes.

The two existing alerts added in #23 (WebhookCertificateNearExpiry, WebhookCertificateAboutToExpire) cover the cert expiry side. These five cover the apply / stamp / GRM-bridge side that is just as important for caBundle freshness reaching the shoot — if the cert rotates but the caBundle doesn't propagate, every TLS call from the shoot apiserver to the webhook server still breaks.

What's added

Alert Severity For What it catches
WebhookTargetStampErrors critical 10m target_stamps_total{result="error"} rate > 0 — MWC/VWC apply on the target cluster is failing. The freshly rotated caBundle is not reaching the admission webhooks; CR writes may be rejected with TLS errors.
WebhookManagedResourceStampErrors critical 10m managed_resource_stamps_total{result="error"} rate > 0 — MR Secret stamping on the hosting cluster is failing. Embedded CRDs will not refresh in the shoot via GRM. Conversion webhooks for those CRDs will fail TLS once the old cert is pruned.
WebhookManagedResourceUnhealthy warning 15m managed_resource_unhealthy_total rate > 0 — a labeled MR has ResourcesApplied or ResourcesHealthy condition with Status != True. GRM can't apply or keep the resources healthy in the shoot.
WebhookManagedResourceKeepObjectsViolation warning 15m keep_objects_violations_total rate > 0 — a labeled MR ships CRDs without spec.keepObjects=true. Destructive shape: deleting that MR would cascade-delete every shoot CRD it carries and every CR of those kinds.
WebhookManagedResourceForceReconcilePatchErrors warning 15m force_reconcile_total{result="error"} rate > 0 — patching gardener.cloud/operation=reconcile on MRs is failing. GRM still picks up the change on its own watch/sync cadence, but with extra latency.

The metric names and the "this counter is the alertable signal" framing come directly from the webhook-injector docs — each metric here is one the docs explicitly flag as alertable.

Style

Follows the exact convention of the existing alerts in this file (Argora + webhook cert):

  • {{ if not .Values.prometheusRules.disabled.<AlertName> }} toggle per alert
  • {{ dig "<AlertName>" "for"|"severity"|"service"|"support_group" "<default>" .Values.prometheusRules }} for all four labels
  • {{ include "controlplane-operations.additionalRuleLabels" . }} for global labels
  • Playbook URL placeholder with #TODO: add playbook comment
  • Block-scalar expr: | matching the existing webhook-cert alerts

Verification

$ helm lint charts/controlplane-operations
==> Linting charts/controlplane-operations
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, 0 chart(s) failed

$ helm template charts/controlplane-operations | yq '... | select(.kind=="PrometheusRule" and .metadata.name=="release-name-alerts-controlplane-remote") | .spec' | promtool check rules /dev/stdin
SUCCESS: 11 rules found

All 11 alerts (6 existing + 5 new) parse correctly. PromQL syntax + label keys + rule structure validated.

Disabling

Each alert can be individually disabled via the existing pattern:

prometheusRules:
  disabled:
    WebhookTargetStampErrors: true
    WebhookManagedResourceStampErrors: true
    # etc.

Adds five new alerts covering the propagation-path metrics that
webhook-injector exposes. The two existing alerts (WebhookCertificate
NearExpiry, WebhookCertificateAboutToExpire) cover the cert expiry
side; these five cover the apply / stamp / GRM-bridge side that is
just as important for caBundle freshness reaching the shoot.

Alerts:

- WebhookTargetStampErrors (critical, 10m for): rate of target_stamps
  _total{result="error"} > 0. MWC/VWC apply on the target cluster
  is failing — the freshly rotated caBundle is NOT reaching the
  admission webhooks and subsequent CR writes may be rejected with
  TLS errors.

- WebhookManagedResourceStampErrors (critical, 10m for): rate of
  managed_resource_stamps_total{result="error"} > 0. MR Secret
  stamping on the hosting cluster is failing — embedded CRDs will not
  refresh in the shoot via GRM, and conversion webhooks for those
  CRDs will fail TLS once the old cert is pruned.

- WebhookManagedResourceUnhealthy (warning, 15m for): rate of
  managed_resource_unhealthy_total > 0. A labeled MR has
  ResourcesApplied or ResourcesHealthy != True. GRM is unable to
  apply or keep resources healthy in the shoot — caBundle propagation
  is degraded.

- WebhookManagedResourceKeepObjectsViolation (warning, 15m for): rate
  of keep_objects_violations_total > 0. A labeled MR ships CRDs
  without spec.keepObjects=true — destructive shape: deleting that
  MR (intentionally or accidentally) would cascade-delete shoot-side
  CRDs and every custom resource of those kinds.

- WebhookManagedResourceForceReconcilePatchErrors (warning, 15m for):
  rate of force_reconcile_total{result="error"} > 0. Patching
  gardener.cloud/operation=reconcile on MRs is failing. GRM will
  still pick up the change on its own watch/sync cadence, but with
  extra latency.

Style follows the existing webhook cert alerts and the rest of this
file: Helm dig templating for for/severity/service/support_group,
disabled toggle, additionalRuleLabels include, playbook link. Each
alert can be individually disabled via .Values.prometheusRules.disabled.

Verified:
- helm lint charts/controlplane-operations: clean
- promtool check rules: SUCCESS, 11 rules found (6 existing + 5 new)
@videlov videlov requested a review from a team as a code owner June 25, 2026 15:09
Copilot AI review requested due to automatic review settings June 25, 2026 15:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds propagation-path Prometheus alerts for webhook-injector metrics in the controlplane-remote PrometheusRule, complementing the existing certificate-expiry alerts to detect failures in caBundle propagation to target clusters / ManagedResources.

Changes:

  • Add 5 new Prometheus alert rules covering target stamping errors, ManagedResource stamping errors, ManagedResource unhealthy state, keepObjects violations, and force-reconcile patch errors.
  • Introduce per-alert disable toggles and standard labels (severity/service/support_group/additionalRuleLabels) consistent with existing rules in this file.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread charts/controlplane-operations/alerts/controlplane-remote.yaml Outdated
Comment thread charts/controlplane-operations/alerts/controlplane-remote.yaml Outdated
Comment thread charts/controlplane-operations/alerts/controlplane-remote.yaml Outdated
Comment thread charts/controlplane-operations/alerts/controlplane-remote.yaml Outdated
Comment thread charts/controlplane-operations/alerts/controlplane-remote.yaml Outdated
videlov added 2 commits June 25, 2026 17:12
Patch bump for the additive webhook-injector propagation alerts.
Follows the convention from #23 (1.1.9 → 1.1.10 for the previous
additive alerts PR).
Addresses Copilot review feedback on #26:

Using rate(metric[N]) > 0 together with for: N is a known
Prometheus gotcha — a single error increment can fall out of the
trailing N-minute rate window right as the for clause would
otherwise satisfy, so a sustained but low-frequency error stream
fails to fire the alert.

Fix: keep the for: durations (which encode the persistence
threshold we actually want — 10m for critical apply failures,
15m for warning-severity MR conditions) but shrink the rate
window to a fixed [5m]. That ensures the rate observation
continues to evaluate > 0 across the full for period for any
error stream that's persistent at all.

Same change applied to all five new alerts:
- WebhookTargetStampErrors
- WebhookManagedResourceStampErrors
- WebhookManagedResourceUnhealthy
- WebhookManagedResourceKeepObjectsViolation
- WebhookManagedResourceForceReconcilePatchErrors

Patch-bump chart + plugin to 1.1.12.

Verified:
- helm lint charts/controlplane-operations: clean
- promtool check rules: SUCCESS, 11 rules found.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comment thread charts/controlplane-operations/plugindefinition.yaml Outdated
videlov and others added 2 commits June 26, 2026 11:39
Annotations on the five propagation alerts now identify the affected
shoot and the per-metric labels in summary/description, following the
pattern from controlplane-gardener.yaml ({{` $labels.<name> `}}).

Switch sum() → sum by (...) so the labels survive aggregation:

- WebhookTargetStampErrors: sum by (namespace, kind)
  → summary names {{ $labels.kind }} (Mutating/Validating) and
    {{ $labels.namespace }} (the shoot's control-plane namespace).

- WebhookManagedResourceStampErrors: sum by (namespace)
  → identifies the shoot whose MR Secret stamp failed.

- WebhookManagedResourceUnhealthy: sum by (namespace, condition, status)
  → summary spells out which condition (ResourcesApplied/Healthy) is
    at which status (False/Progressing/Unknown).

- WebhookManagedResourceKeepObjectsViolation: sum by (namespace)
  → identifies the shoot whose MR ships CRDs without keepObjects=true.

- WebhookManagedResourceForceReconcilePatchErrors: sum by (namespace)
  → identifies the shoot.

All descriptions now suggest checking webhook-injector logs in the
affected namespace, since the exact MR name is not a metric label
(by design — cardinality-bounded) but is in the log line.

Cardinality bound is small: target_stamps has 2×3 series max,
managed_resource_stamps and force_reconcile have 3, unhealthy has
2×3, keep_objects_violations has 1 — all per-Pod. {result="error"}
narrows further.

Patch-bump chart + plugin to 1.1.13.

Verified:
- helm lint: clean
- helm template + promtool check rules: SUCCESS, 11 rules found
- rendered output contains literal {{ $labels.namespace }} etc. as
  Prometheus expects, not Helm template artifacts.
Co-authored-by: Dmitri Fedotov <13087245+defo89@users.noreply.github.com>
Signed-off-by: Vladimir Videlov <vladimir.videlov@sap.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@videlov videlov merged commit e59f5d1 into main Jun 26, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants