Job:
periodic-ci-openshift-release-master-nightly-4.12-upgrade-from-stable-4.11-e2e-aws-sdn-upgrade (all) - 15 runs, 40% failed, 250% of failures match = 100% impact
#1791119352230055936junit12 hours ago
May 16 16:20:28.313 E ns/openshift-console pod/console-6b4959bdd8-n5gxl node/ip-10-0-176-252.ec2.internal uid/e652b36c-1d5b-48ad-9ba7-249ca7f160a4 container/console reason/ContainerExit code/2 cause/Error W0516 16:00:21.429884       1 main.go:228] Flag inactivity-timeout is set to less then 300 seconds and will be ignored!\nI0516 16:00:21.429909       1 main.go:348] cookies are secure!\nI0516 16:00:21.473295       1 main.go:809] Binding to [::]:8443...\nI0516 16:00:21.473324       1 main.go:811] using TLS\n
May 16 16:23:54.933 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-16-144840: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-d723f4766ebe07725e929a5e7b832287 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-2c75e2fc10e8e1caa93d8b52e7e093fc, retrying]
May 16 16:23:54.933 - 115s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-16-144840: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-d723f4766ebe07725e929a5e7b832287 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-2c75e2fc10e8e1caa93d8b52e7e093fc, retrying]

... 1 lines not shown

#1790845800167772160junit30 hours ago
May 15 22:16:16.372 E ns/e2e-k8s-sig-apps-job-upgrade-1211 pod/foo-bcbvw node/ip-10-0-185-202.us-west-1.compute.internal uid/0ecb5509-670e-46bf-84f0-99edc0adf596 container/c reason/ContainerExit code/137 cause/Error
May 15 22:17:03.461 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-15-204102: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-0578eaa34accfb4fd5456342aa54a1e1 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-b63737dfecc6d87203c02d99347d44c2, retrying]
May 15 22:17:03.461 - 547s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-15-204102: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-0578eaa34accfb4fd5456342aa54a1e1 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-b63737dfecc6d87203c02d99347d44c2, retrying]

... 1 lines not shown

#1790497075381669888junit2 days ago
May 14 23:15:47.680 - 999ms E disruption/oauth-api connection/reused reason/DisruptionBegan disruption/oauth-api connection/reused stopped responding to GET requests over reused connections: error running request: 500 Internal Server Error: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"rpc error: code = Unknown desc = malformed header: missing HTTP content-type","code":500}\n
May 14 23:17:03.105 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-14-213601: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-7e2cae5f5b05e31dd9886103bdc30169 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-f14bbfb5d2295eff0560f368d8c30784, retrying]
May 14 23:17:03.105 - 120s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-14-213601: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-7e2cae5f5b05e31dd9886103bdc30169 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-f14bbfb5d2295eff0560f368d8c30784, retrying]

... 1 lines not shown

#1790637181929263104junit44 hours ago
May 15 08:24:39.640 E ns/openshift-multus pod/multus-admission-controller-56d95c45-ltwqq node/ip-10-0-177-28.us-east-2.compute.internal uid/3bb340a3-0f07-45eb-b3fd-f6334032a813 container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 15 08:28:04.293 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-15-065158: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-69bb43d0f661d12272f315843d1b6c86 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-8565c02de757718afcdf222f3ccd263c, retrying]
May 15 08:28:04.293 - 115s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-15-065158: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-69bb43d0f661d12272f315843d1b6c86 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-8565c02de757718afcdf222f3ccd263c, retrying]

... 1 lines not shown

#1789211294486761472junit5 days ago
May 11 10:08:10.161 - 93ms  E clusteroperator/storage condition/Degraded status/True reason/AWSEBSCSIDriverOperatorDeploymentDegraded: Get "https://172.30.0.1:443/apis/apps/v1/namespaces/openshift-cluster-csi-drivers/deployments/aws-ebs-csi-driver-operator": context canceled
May 11 10:08:48.157 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-11-082650: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-fc9d9613217c83f710b3194e202100b4 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-5775a9b8d8846f3f6381dd644ff57b1c, retrying]
May 11 10:08:48.157 - 136s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-11-082650: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-fc9d9613217c83f710b3194e202100b4 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-5775a9b8d8846f3f6381dd644ff57b1c, retrying]

... 1 lines not shown

#1789121081806688256junit6 days ago
May 11 03:55:42.798 E ns/e2e-k8s-sig-apps-job-upgrade-8628 pod/foo-rcbh6 node/ip-10-0-185-203.us-east-2.compute.internal uid/412ab379-8ffe-4709-8b8f-9a8fdc463391 container/c reason/ContainerExit code/137 cause/Error
May 11 03:57:51.976 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-11-022650: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-d2c7c8629ba2cbae36529f093c5c0a24 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-0888d0ff69b5ed53e6b8525794553ba3, retrying]
May 11 03:57:51.976 - 422s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-11-022650: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-d2c7c8629ba2cbae36529f093c5c0a24 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-0888d0ff69b5ed53e6b8525794553ba3, retrying]

... 1 lines not shown

#1790373474938654720junit2 days ago
May 14 15:03:27.851 E ns/openshift-multus pod/multus-admission-controller-79c6c86dcc-95c4j node/ip-10-0-233-169.ec2.internal uid/e97f63b6-4ed5-4de7-9972-279d0f573b5c container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 14 15:06:48.795 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-14-132144: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-dc3666fc6f03ca5d39b68eb20dc0aa01 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-0dccdd9aeae7e1f9a2a7698e91d6c3d7, retrying]
May 14 15:06:48.795 - 119s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-14-132144: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-dc3666fc6f03ca5d39b68eb20dc0aa01 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-0dccdd9aeae7e1f9a2a7698e91d6c3d7, retrying]

... 1 lines not shown

#1789920315686522880junit3 days ago
May 13 08:54:31.922 - 999ms E disruption/oauth-api connection/reused reason/DisruptionBegan disruption/oauth-api connection/reused stopped responding to GET requests over reused connections: error running request: 500 Internal Server Error: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"rpc error: code = Unknown desc = malformed header: missing HTTP content-type","code":500}\n
May 13 08:55:39.362 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-11-022650: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-1c2a71f8488dec476cf2fe208f800689 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-81da82a09d731c7869c2af8cf44a0b44, retrying]
May 13 08:55:39.362 - 484s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-11-022650: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-1c2a71f8488dec476cf2fe208f800689 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-81da82a09d731c7869c2af8cf44a0b44, retrying]

... 1 lines not shown

#1789301761626869760junit5 days ago
May 11 16:01:54.000 - 1s    E ns/openshift-image-registry route/test-disruption-new disruption/image-registry connection/new reason/DisruptionBegan ns/openshift-image-registry route/test-disruption-new disruption/image-registry connection/new stopped responding to GET requests over new connections: Get "https://test-disruption-new-openshift-image-registry.apps.ci-op-fh34rstg-857a8.aws-2.ci.openshift.org/healthz": read tcp 10.131.208.144:37962->54.71.153.12:443: read: connection reset by peer
May 11 16:01:57.085 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-11-142650: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-5fc0d48c55540e9d9dbace40bd80fbbd expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-8ee4e725e81ac95102ac636d193a5df6, retrying]
May 11 16:01:57.085 - 499s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-11-142650: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-5fc0d48c55540e9d9dbace40bd80fbbd expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-8ee4e725e81ac95102ac636d193a5df6, retrying]

... 1 lines not shown

#1788912591758692352junit6 days ago
May 10 14:17:41.000 - 1s    E disruption/service-load-balancer-with-pdb connection/new reason/DisruptionBegan disruption/service-load-balancer-with-pdb connection/new stopped responding to GET requests over new connections: Get "http://a103bac38f55c431fb1706ba2ed65822-646851622.us-west-2.elb.amazonaws.com:80/echo?msg=Hello": read tcp 10.128.150.193:34268->44.236.103.39:80: read: connection reset by peer
May 10 14:18:07.201 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-10-124007: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-262a37c63dc65b84fe9a34cc314a9e63 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-a5c3fa48123ffd9c4f4ae7d960c1aa04, retrying]
May 10 14:18:07.201 - 117s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-10-124007: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-262a37c63dc65b84fe9a34cc314a9e63 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-a5c3fa48123ffd9c4f4ae7d960c1aa04, retrying]

... 1 lines not shown

#1788803977492566016junit6 days ago
May 10 07:02:32.160 E ns/openshift-multus pod/multus-admission-controller-79c6c86dcc-q6v6s node/ip-10-0-250-245.ec2.internal uid/67bcf73c-f386-4100-bc39-0ff47938d6bc container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 10 07:05:58.102 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-10-052846: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-b943ecd8de26a552d6bfd98401a08650 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-99c1cae7681aa990964d9ca6dd19f043, retrying]
May 10 07:05:58.102 - 102s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-10-052846: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-b943ecd8de26a552d6bfd98401a08650 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-99c1cae7681aa990964d9ca6dd19f043, retrying]

... 1 lines not shown

#1788713467251789824junit7 days ago
May 10 01:03:24.754 - 998ms E disruption/kube-api connection/new reason/DisruptionBegan disruption/kube-api connection/new stopped responding to GET requests over new connections: Get "https://api.ci-op-vhij6rdn-857a8.aws-2.ci.openshift.org:6443/api/v1/namespaces/default": net/http: timeout awaiting response headers
May 10 01:04:45.149 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-09-232846: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-6dcd26c114249d515941edb2f00838df expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-e8d18bc6fbd5e2e455b85203ce89a7b8, retrying]
May 10 01:04:45.149 - 504s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-09-232846: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-6dcd26c114249d515941edb2f00838df expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-e8d18bc6fbd5e2e455b85203ce89a7b8, retrying]

... 1 lines not shown

#1788541665602965504junit7 days ago
May 09 13:45:49.359 E ns/openshift-monitoring pod/alertmanager-main-1 node/ip-10-0-153-228.ec2.internal uid/d4509335-60fb-47d0-b0f7-e42c8164a544 container/alertmanager reason/ContainerExit code/1 cause/Error ts=2024-05-09T13:45:46.411Z caller=main.go:231 level=info msg="Starting Alertmanager" version="(version=0.24.0, branch=rhaos-4.12-rhel-8, revision=2742bd3a55bd39fee12a11daa9025fcae839c7dc)"\nts=2024-05-09T13:45:46.411Z caller=main.go:232 level=info build_context="(go=go1.19.13 X:strictfipsruntime, user=root@912fdea84186, date=20240425-02:16:52)"\nts=2024-05-09T13:45:46.479Z caller=cluster.go:680 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s\nts=2024-05-09T13:45:46.521Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/config_out/alertmanager.env.yaml\nts=2024-05-09T13:45:46.521Z caller=coordinator.go:118 level=error component=configuration msg="Loading configuration file failed" file=/etc/alertmanager/config_out/alertmanager.env.yaml err="open /etc/alertmanager/config_out/alertmanager.env.yaml: no such file or directory"\nts=2024-05-09T13:45:46.521Z caller=cluster.go:689 level=info component=cluster msg="gossip not settled but continuing anyway" polls=0 elapsed=41.739082ms\n
May 09 13:47:42.252 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-09-120424: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-d318216ec01ecb89db3ddb0fca0d5a39 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-ef45842029d2669e4aa8cdd1ef3504ea, retrying]
May 09 13:47:42.252 - 518s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-09-120424: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-d318216ec01ecb89db3ddb0fca0d5a39 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-ef45842029d2669e4aa8cdd1ef3504ea, retrying]

... 1 lines not shown

#1788189394256007168junit8 days ago
May 08 14:21:02.490 E ns/openshift-multus pod/multus-admission-controller-5b94657776-k75bn node/ip-10-0-194-127.ec2.internal uid/4765301e-633c-4b92-a7e8-60bd44e36a79 container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 08 14:24:50.982 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-08-124615: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-5a0551d1d639acef2a7b09a34599b071 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-9566343c4017a77d34319a67b0a5d5df, retrying]
May 08 14:24:50.982 - 104s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-08-124615: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-5a0551d1d639acef2a7b09a34599b071 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-9566343c4017a77d34319a67b0a5d5df, retrying]

... 1 lines not shown

#1788043272329891840junit8 days ago
May 08 04:54:07.463 E ns/openshift-multus pod/multus-admission-controller-5b94657776-8fjn6 node/ip-10-0-236-190.us-west-1.compute.internal uid/fb7743a6-b971-4012-89af-b25e28bef0a4 container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 08 04:57:13.760 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.nightly-2024-05-08-030344: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-7904b851fb602d742e9e8075ee631180 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-0d6601962024f8f9181917df2039be4d, retrying]
May 08 04:57:13.760 - 140s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.12.0-0.nightly-2024-05-08-030344: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-7904b851fb602d742e9e8075ee631180 expected 33a010772d604aff2cb625d04a9469a47f53c96e has 15d0b0288a4330b90ac89f14c781dfa7349af52c: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-0d6601962024f8f9181917df2039be4d, retrying]

... 1 lines not shown

Found in 100.00% of runs (250.00% of failures) across 15 total runs and 1 jobs (40.00% failed) in 431ms - clear search | chart view - source code located on github