#OCPBUGS-15430 | issue | 4 weeks ago | KubeAPIDown alert rename and/or degraded status ASSIGNED |
We have many guards making sure that there are always at least two instances of the kube-apiserver. If we ever reach a single kube-apiserver and it causes disruption for the clients, other alerts such as KubeAPIErrorBudgetBurn will fire. KubeAPIDown is here to make sure that Prometheus and really any client can reach the kube-apiserver, which they can even when there is only one instance of kube-apiserver running. If they can't or that availability is disrupted, `KubeAPIErrorBudgetBurn` will fire. Comment 23058588 by Marcel Härri at 2023-09-19T06:57:07.949+0000 | |||
#OCPBUGS-30267 | issue | 6 weeks ago | [IBMCloud] MonitorTests liveness/readiness probe error events repeat MODIFIED |
Mar 12 18:52:24.937 - 58s E namespace/openshift-kube-apiserver alert/KubeAPIErrorBudgetBurn alertstate/firing severity/critical ALERTS {alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="1h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="5m"} | |||
periodic-ci-openshift-multiarch-master-nightly-4.14-ocp-e2e-ovn-remote-libvirt-s390x (all) - 20 runs, 90% failed, 56% of failures match = 50% impact | |||
#1788766186381512704 | junit | 4 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 1m28s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1m28s, firing for 0s: May 10 03:44:44.139 - 88s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} | |||
#1789853232227422208 | junit | 25 hours ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 2h11m26s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 1h40m0s, firing for 31m26s: May 13 04:17:49.043 - 1074s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} May 13 04:21:19.043 - 864s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"} May 13 04:55:11.043 - 1568s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} May 13 05:33:17.043 - 2494s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} May 13 05:21:19.043 - 718s W alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="1d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="2h"} May 13 04:35:43.043 - 1168s E alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="firing", long="6h", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="critical", short="30m"} | |||
#1788041267737792512 | junit | 6 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 5m28s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 5m28s, firing for 0s: May 08 03:37:13.819 - 328s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} | |||
#1788222677148766208 | junit | 5 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 4m58s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 4m58s, firing for 0s: May 08 15:51:59.535 - 298s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} | |||
#1790034403188019200 | junit | 14 hours ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 28s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 28s, firing for 0s: May 13 15:45:23.916 - 28s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} | |||
#1786229459003641856 | junit | 11 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 2h0m34s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 2h0m34s, firing for 0s: May 03 03:33:29.835 - 7234s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} | |||
#1785685921845219328 | junit | 12 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 6m58s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 6m58s, firing for 0s: May 01 16:04:48.035 - 418s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} | |||
#1785504760518938624 | junit | 13 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 15m26s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 15m26s, firing for 0s: May 01 03:56:52.769 - 538s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} May 01 05:13:52.769 - 388s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} | |||
#1785323536986935296 | junit | 13 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 2m58s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 2m58s, firing for 0s: Apr 30 15:40:18.037 - 178s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} | |||
#1786410657982189568 | junit | 10 days ago | |
# [bz-kube-apiserver][invariant] alert/KubeAPIErrorBudgetBurn should not be at or above pending KubeAPIErrorBudgetBurn was at or above pending for at least 11m54s on platformidentification.JobType{Release:"4.14", FromRelease:"", Platform:"libvirt", Architecture:"s390x", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 11m54s, firing for 0s: May 03 15:34:27.785 - 478s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} May 03 15:43:57.785 - 58s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} May 03 17:19:57.785 - 178s I alert/KubeAPIErrorBudgetBurn namespace/openshift-kube-apiserver ALERTS{alertname="KubeAPIErrorBudgetBurn", alertstate="pending", long="3d", namespace="openshift-kube-apiserver", prometheus="openshift-monitoring/k8s", severity="warning", short="6h"} |
Found in 50.00% of runs (55.56% of failures) across 20 total runs and 1 jobs (90.00% failed) in 95ms - clear search | chart view - source code located on github