MON-4619: Declare KubeStateMetricsTimezonePanic risk for 4.20.z - #10074
Conversation
|
@shahsahil264: This pull request references Jira Issue OCPBUGS-87957, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
@wking Can you PTAL! Thanks |
|
/hold I don't want us to block all updates to 4.20 until such a time that we fix this. Please look closer at 4.20.28 and 4.20.29, /usr/share/zoneinfo exists in the former but not the latter. Why? |
sdodson
left a comment
There was a problem hiding this comment.
https://redhat.atlassian.net/browse/OCPBUGS-87957?focusedCommentId=17692434 has the appropriate affected version ranges.
…d versions Adds conditional update risk for versions where /usr/share/zoneinfo is missing from the kube-state-metrics container image: 4.20.29-30, 4.21.24-25, and 4.22.0-6. Clusters upgrading from unaffected versions with CronJobs using .spec.timeZone will experience kube-state-metrics crash loops, breaking monitoring and potentially stalling upgrades. First affected versions verified by Scott Dodson via rpm -Va tzdata: 4.20.29, 4.21.24, 4.22.0. Earlier versions (4.20.28, 4.21.23) have zoneinfo present and are not affected. Bug: https://redhat.atlassian.net/browse/OCPBUGS-87957 Impact: https://redhat.atlassian.net/browse/MON-4619 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Sahil Shah <sahshah@redhat.com>
fe6078e to
890e4f3
Compare
|
Updated, narrowed to 4.20.29-30, 4.21.24-25, 4.22.0-6 per your findings |
|
/retitle MON-4619: Declare KubeStateMetricsTimezonePanic risk for 4.20.z |
|
@shahsahil264: This pull request references MON-4619 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the spike to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
…4.21.26, 4.22.7 Declare fixedIn 4.20.32, 4.21.27, and 4.22.8 respectively. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@shahsahil264: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: shahsahil264, wking The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/hold cancel Version ranges have been cleared up and represent actual regressed versions. Kube-state-metrics actually fixed their issue in 4.22.7 but we don't know what other coponents are broken and no one has noticed yet. As a result I am happy to wait for the more complete fix of restoring /usr/share/zoneinfo in https://issues.redhat.com/browse/OCPBUGS-99767 and cloned bugs be the one that clears this risk |
Summary
Adds conditional update risk for all 4.20.z GA releases (4.20.0 through 4.20.30) to warn clusters upgrading from 4.19.z that kube-state-metrics may crash loop if CronJobs with
.spec.timeZoneare present.Root cause:
/usr/share/zoneinfo/is missing from the kube-state-metrics container image. The statically-compiled Go binary panics whentime.LoadLocation()fails for timezone-aware CronJobs.fixedIn: fix PRs openshift/kube-state-metrics#147 and #150 still opentype: Always: no PromQL metric for CronJob timezone usageBug
Test plan
./hack/validate-blocked-edges.pypasses