| #OCPBUGS-30267 | issue | 4 weeks ago | [IBMCloud] MonitorTests liveness/readiness probe error events repeat MODIFIED |
Mar 12 18:52:24.937 - 58s E namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/firing severity/critical ALERTS
{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="1h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="5m"}
| |||
| #OCPBUGS-15430 | issue | 13 days ago | KubeAPIDown alert rename and/or degraded status ASSIGNED |
We have many guards making sure that there are always at least two instances of the kube-apiserver. If we ever reach a single kube-apiserver and it causes disruption for the clients, other alerts such as KubeAPIErrorBudgetBurn will fire. KubeAPIDown is here to make sure that Prometheus and really any client can reach the kube-apiserver, which they can even when there is only one instance of kube-apiserver running. If they can't or that availability is disrupted, `KubeAPIErrorBudgetBurn` will fire. Comment 23058588 by Marcel Härri at 2023-09-19T06:57:07.949+0000 | |||
| periodic-ci-openshift-multiarch-master-nightly-4.14-ocp-e2e-aws-ovn-arm64 (all) - 33 runs, 27% failed, 144% of failures match = 39% impact | |||
| #1784542557464694784 | junit | 9 hours ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 2m56s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 2m56s, firing for 0s:
Apr 28 11:52:44.047 - 176s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1784179852299997184 | junit | 33 hours ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 5m24s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 5m24s, firing for 0s:
Apr 27 11:53:46.287 - 324s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1783970994096967680 | junit | 47 hours ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 3m22s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 3m22s, firing for 0s:
Apr 26 22:09:17.874 - 202s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1783831373329469440 | junit | 2 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 18m54s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 17m56s, firing for 58s:
Apr 26 12:54:35.414 - 68s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 26 12:54:35.414 - 68s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Apr 26 12:56:41.414 - 110s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 26 12:56:41.414 - 830s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Apr 26 12:55:43.414 - 58s E alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="6h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="30m"}
| |||
| #1782822463848058880 | junit | 5 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 5m44s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 5m44s, firing for 0s:
Apr 23 18:00:55.702 - 22s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 23 18:00:55.702 - 322s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1782702607987380224 | junit | 5 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 3m26s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 3m26s, firing for 0s:
Apr 23 10:05:57.781 - 90s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Apr 23 10:07:59.781 - 28s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Apr 23 10:09:29.781 - 88s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1782498109918547968 | junit | 6 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 3m10s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 3m10s, firing for 0s:
Apr 22 20:32:43.129 - 190s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1782366179705229312 | junit | 6 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1m22s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1m22s, firing for 0s:
Apr 22 11:41:21.272 - 82s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1783682135727542272 | junit | 2 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 4m44s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 4m44s, firing for 0s:
Apr 26 02:54:29.234 - 284s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1781715020514594816 | junit | 8 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 21m16s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 20m16s, firing for 1m0s:
Apr 20 16:51:01.719 - 218s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 20 16:51:01.719 - 998s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Apr 20 16:50:01.719 - 60s E alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="6h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="30m"}
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above info
KubeAPIErrorBudgetBurn was at or above info for at least 1m0s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 20m16s, firing for 1m0s:
| |||
| #1781670643247353856 | junit | 8 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1m34s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1m34s, firing for 0s:
Apr 20 13:39:36.307 - 94s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1781033989155328000 | junit | 10 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 2m54s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 2m54s, firing for 0s:
Apr 18 19:32:35.234 - 174s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1780467083029516288 | junit | 11 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 5m10s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 5m10s, firing for 0s:
Apr 17 06:08:13.409 - 310s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
Found in 39.39% of runs (144.44% of failures) across 33 total runs and 1 jobs (27.27% failed) in 149ms - clear search | chart view - source code located on github