| #OCPBUGS-15430 | issue | 13 days ago | KubeAPIDown alert rename and/or degraded status ASSIGNED |
We have many guards making sure that there are always at least two instances of the kube-apiserver. If we ever reach a single kube-apiserver and it causes disruption for the clients, other alerts such as KubeAPIErrorBudgetBurn will fire. KubeAPIDown is here to make sure that Prometheus and really any client can reach the kube-apiserver, which they can even when there is only one instance of kube-apiserver running. If they can't or that availability is disrupted, `KubeAPIErrorBudgetBurn` will fire. Comment 23058588 by Marcel Härri at 2023-09-19T06:57:07.949+0000 | |||
| #OCPBUGS-30267 | issue | 4 weeks ago | [IBMCloud] MonitorTests liveness/readiness probe error events repeat MODIFIED |
Mar 12 18:52:24.937 - 58s E namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/firing severity/critical ALERTS
{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="1h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="5m"}
| |||
| periodic-ci-openshift-multiarch-master-nightly-4.15-upgrade-from-stable-4.14-ocp-e2e-upgrade-gcp-ovn-heterogeneous (all) - 165 runs, 15% failed, 50% of failures match = 7% impact | |||
| #1784507448137093120 | junit | 8 hours ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 21m12s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 21m12s, firing for 0s:
Apr 28 09:46:53.155 - 1214s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Apr 28 10:08:39.155 - 58s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1784439289589796864 | junit | 12 hours ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 42m48s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 42m48s, firing for 0s:
Apr 28 05:16:16.940 - 144s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 28 05:16:16.940 - 2424s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1784303087557545984 | junit | 21 hours ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 33m30s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 33m30s, firing for 0s:
Apr 27 20:17:41.590 - 2010s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1784212749866242048 | junit | 27 hours ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 34m6s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 34m6s, firing for 0s:
Apr 27 14:22:22.305 - 2046s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1783819520394137600 | junit | 2 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 17m40s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 17m40s, firing for 0s:
Apr 26 12:09:18.822 - 1060s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1782333194016657408 | junit | 6 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 36m34s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 36m34s, firing for 0s:
Apr 22 09:53:18.219 - 2194s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1781934497428672512 | junit | 7 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 43m4s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 43m4s, firing for 0s:
Apr 21 07:20:36.449 - 18s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 21 07:20:36.449 - 2178s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Apr 21 07:58:26.449 - 388s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1781813912811868160 | junit | 7 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1h30m54s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1h30m54s, firing for 0s:
Apr 20 23:26:23.172 - 1182s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 20 23:26:23.172 - 4272s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1781775919879294976 | junit | 7 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 34m38s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 34m38s, firing for 0s:
Apr 20 21:01:59.594 - 64s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 20 21:01:59.594 - 2014s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1781240148386648064 | junit | 9 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1h25m48s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1h25m48s, firing for 0s:
Apr 19 09:31:07.988 - 1194s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 19 09:31:07.988 - 3954s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1781164339869257728 | junit | 9 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 17m48s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 17m48s, firing for 0s:
Apr 19 04:24:49.339 - 380s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Apr 19 04:32:41.339 - 688s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
| |||
| #1779837333235830784 | junit | 13 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1h19m16s on platformidentification.JobType{Release:"4.15", FromRelease:"4.14", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1h19m16s, firing for 0s:
Apr 15 12:34:18.991 - 584s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
Apr 15 12:34:18.991 - 4054s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
Apr 15 12:45:34.991 - 118s I namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/pending severity/warning ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
| |||
Found in 7.27% of runs (50.00% of failures) across 165 total runs and 1 jobs (14.55% failed) in 2.932s - clear search | chart view - source code located on github