Job:
#OCPBUGS-15430issue4 weeks agoKubeAPIDown alert rename and/or degraded status ASSIGNED
We have many guards making sure that there are always at least two instances of the kube-apiserver. If we ever reach a single kube-apiserver and it causes disruption for the clients, other alerts such as KubeAPIErrorBudgetBurn will fire.
KubeAPIDown is here to make sure that Prometheus and really any client can reach the kube-apiserver, which they can even when there is only one instance of kube-apiserver running. If they can't or that availability is disrupted, `KubeAPIErrorBudgetBurn` will fire.
Comment 23058588 by Marcel Härri at 2023-09-19T06:57:07.949+0000
#OCPBUGS-30267issue6 weeks ago[IBMCloud] MonitorTests liveness/readiness probe error events repeat MODIFIED
Mar 12 18:52:24.937 - 58s E namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/firing severity/critical ALERTS
{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="1h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="5m"}
periodic-ci-openshift-multiarch-master-nightly-4.14-ocp-e2e-aws-sdn-arm64 (all) - 28 runs, 14% failed, 200% of failures match = 29% impact
#1790238535098306560junit15 hours ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 3m20s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"sdn", Topology:"ha"} (maxAllowed=0s): pending for 3m20s, firing for 0s:
May 14 05:16:19.269 - 200s  I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1789569035977887744junit2 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 7m12s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"sdn", Topology:"ha"} (maxAllowed=0s): pending for 7m12s, firing for 0s:
May 12 08:47:23.032 - 96s   I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"}
May 12 08:47:23.032 - 336s  I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1789176333369085952junit3 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 30s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"sdn", Topology:"ha"} (maxAllowed=0s): pending for 30s, firing for 0s:
May 11 06:49:22.495 - 30s   I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1788953236535775232junit4 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 3m0s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"sdn", Topology:"ha"} (maxAllowed=0s): pending for 3m0s, firing for 0s:
May 10 16:07:03.892 - 180s  I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1788424759873114112junit5 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 3m44s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"sdn", Topology:"ha"} (maxAllowed=0s): pending for 3m44s, firing for 0s:
May 09 04:54:33.903 - 224s  I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1787596817874030592junit7 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 2m46s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"sdn", Topology:"ha"} (maxAllowed=0s): pending for 2m46s, firing for 0s:
May 06 22:14:34.918 - 166s  I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1786867967925948416junit9 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 2m28s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"sdn", Topology:"ha"} (maxAllowed=0s): pending for 2m28s, firing for 0s:
May 04 21:53:23.086 - 148s  I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}
#1786810472947781632junit10 days ago
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending
KubeAPIErrorBudgetBurn was at or above pending for at least 1m14s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"aws", Architecture:"arm64", Network:"sdn", Topology:"ha"} (maxAllowed=0s): pending for 1m14s, firing for 0s:
May 04 18:06:19.798 - 74s   I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"}

Found in 28.57% of runs (200.00% of failures) across 28 total runs and 1 jobs (14.29% failed) in 185ms - clear search | chart view - source code located on github