May 16 21:32:12.261 E ns/openshift-monitoring pod/alertmanager-main-0 node/libvirt-ppc64le-2-0-e-lcsmx-worker-0-b9fhn uid/14765d8e-79b4-4126-abc3-8e8e81ebfca5 container/alertmanager reason/ContainerExit code/1 cause/Error ts=2024-05-16T21:32:09.639Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=rhaos-4.13-rhel-8, revision=cca15d1d9be3ebe61f12f5179e54f0c4c8b23e8b)"\nts=2024-05-16T21:32:09.639Z caller=main.go:241 level=info build_context="(go=go1.19.13 X:strictfipsruntime, user=root@36412ec39e7b, date=20240515-05:16:57)"\nts=2024-05-16T21:32:09.683Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s\nts=2024-05-16T21:32:09.722Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/config_out/alertmanager.env.yaml\nts=2024-05-16T21:32:09.727Z caller=coordinator.go:118 level=error component=configuration msg="Loading configuration file failed" file=/etc/alertmanager/config_out/alertmanager.env.yaml err="open /etc/alertmanager/config_out/alertmanager.env.yaml: no such file or directory"\nts=2024-05-16T21:32:09.727Z caller=cluster.go:690 level=info component=cluster msg="gossip not settled but continuing anyway" polls=0 elapsed=43.806067ms\n
May 16 21:33:34.192 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-16-190042: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-629596ceb31ae84d6b7f59da297763bd expected 2d7a09cc9acd5ffd96ef777a1bd2c348cd49a0c9 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-8fb60af801759a837f82a83ef615bbb2, retrying]
May 16 21:33:34.192 - 519s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-16-190042: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-629596ceb31ae84d6b7f59da297763bd expected 2d7a09cc9acd5ffd96ef777a1bd2c348cd49a0c9 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-8fb60af801759a837f82a83ef615bbb2, retrying]

... 1 lines not shown

May 14 21:40:12.863 E clusterversion/version changed Failing to True: ClusterOperatorNotAvailable: Cluster operator machine-config is not available
May 14 21:42:49.700 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-14-130502: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-3ccdad228c7f183a59477adefc311208 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-45b3c227f3902df56383e55a082a0257, retrying]
May 14 21:42:49.700 - 203s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-14-130502: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-3ccdad228c7f183a59477adefc311208 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-45b3c227f3902df56383e55a082a0257, retrying]

... 1 lines not shown

May 14 09:46:47.314 E ns/openshift-multus pod/cni-sysctl-allowlist-ds-pgrdk node/libvirt-ppc64le-1-0-e-jwrsb-worker-0-nvssh uid/e1be13d0-df8b-4a87-89c2-438863673ca5 container/kube-multus-additional-cni-plugins reason/ContainerExit code/137 cause/Error
May 14 09:48:13.862 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-10-124240: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-9176875605e25d634c4fb6421ec18058 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-948d82cc80e9dba724114935104a441c, retrying]
May 14 09:48:13.862 - 253s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-10-124240: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-9176875605e25d634c4fb6421ec18058 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-948d82cc80e9dba724114935104a441c, retrying]

... 1 lines not shown

May 13 21:55:50.542 E ns/openshift-multus pod/multus-admission-controller-6985955c7-hlhmx node/libvirt-ppc64le-2-0-e-n447p-master-2 uid/954d7603-defc-4bc9-8ebb-bd8e84f69e18 container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 13 21:58:11.614 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-10-124240: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-6067da7738946c23e1ef1efa0a0f315f expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-12c66ba5ca31b097a89aac3ea0655ba0, retrying]
May 13 21:58:11.614 - 208s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-10-124240: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-6067da7738946c23e1ef1efa0a0f315f expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-12c66ba5ca31b097a89aac3ea0655ba0, retrying]

... 1 lines not shown

May 10 21:54:11.021 E ns/openshift-monitoring pod/alertmanager-main-0 node/libvirt-ppc64le-0-2-e-vtvr7-worker-0-rm5qr uid/bdc2b075-b143-4943-b291-a918c14fb66c container/alertmanager reason/ContainerExit code/1 cause/Error ts=2024-05-10T21:54:08.279Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=rhaos-4.13-rhel-8, revision=38194c1bde7b693bb84526d1084400a1e76da31b)"\nts=2024-05-10T21:54:08.279Z caller=main.go:241 level=info build_context="(go=go1.19.13 X:strictfipsruntime, user=root@629bb2fa98ce, date=20240510-00:07:35)"\nts=2024-05-10T21:54:08.314Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s\nts=2024-05-10T21:54:08.360Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/config_out/alertmanager.env.yaml\nts=2024-05-10T21:54:08.360Z caller=coordinator.go:118 level=error component=configuration msg="Loading configuration file failed" file=/etc/alertmanager/config_out/alertmanager.env.yaml err="open /etc/alertmanager/config_out/alertmanager.env.yaml: no such file or directory"\nts=2024-05-10T21:54:08.360Z caller=cluster.go:690 level=info component=cluster msg="gossip not settled but continuing anyway" polls=0 elapsed=46.613727ms\n
May 10 21:54:43.222 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-10-124240: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-b70d439fa371a21b6537842487613666 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-2699719725dbc202d069b7bf5c8d65a7, retrying]
May 10 21:54:43.222 - 553s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-10-124240: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-b70d439fa371a21b6537842487613666 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-2699719725dbc202d069b7bf5c8d65a7, retrying]

... 1 lines not shown

May 10 09:34:54.228 E ns/openshift-multus pod/multus-admission-controller-6985955c7-4f258 node/libvirt-ppc64le-0-0-e-tk9w2-master-0 uid/e8b35183-1dd7-4936-88cc-85ffeff2f22b container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 10 09:36:48.833 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-10-045743: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-a59cd8271f011222ea40252d0a7a999c expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-e421970cef127642aeb8d8357825b89e, retrying]
May 10 09:36:48.833 - 581s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-10-045743: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-a59cd8271f011222ea40252d0a7a999c expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-e421970cef127642aeb8d8357825b89e, retrying]

... 1 lines not shown

May 15 09:41:58.453 E ns/openshift-multus pod/multus-admission-controller-6985955c7-2vbx4 node/libvirt-ppc64le-0-2-e-gpsz7-master-2 uid/ad6c70cd-6dfe-4ac7-bbe7-d628012270df container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 15 09:44:26.922 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-14-130502: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-b427f69ccce4f766e9716430e0dfd3d4 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-4f1dffb27ac7fb0f7561fbcf84de283e, retrying]
May 15 09:44:26.922 - 208s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-14-130502: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-b427f69ccce4f766e9716430e0dfd3d4 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-4f1dffb27ac7fb0f7561fbcf84de283e, retrying]

... 1 lines not shown

May 08 21:40:46.444 E ns/openshift-multus pod/cni-sysctl-allowlist-ds-v9lcr node/libvirt-ppc64le-2-0-e-kc4ml-worker-0-zrvsn uid/6657b4a3-ce16-440d-9445-dcf04a166496 container/kube-multus-additional-cni-plugins reason/ContainerExit code/137 cause/Error
May 08 21:43:42.806 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-08-174145: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-df762c5512e2410634a80876d2e6ff5b expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-3bc43312d2fc649e6fef4d9624b66273, retrying]
May 08 21:43:42.806 - 78s   E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-08-174145: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-df762c5512e2410634a80876d2e6ff5b expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-3bc43312d2fc649e6fef4d9624b66273, retrying]

... 1 lines not shown

May 09 09:36:37.673 E ns/openshift-multus pod/multus-admission-controller-767f99f9f6-vjt5c node/libvirt-ppc64le-2-1-e-wq5g6-master-2 uid/65f8ce3a-0f31-4d65-a898-8b44f58b3272 container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 09 09:39:18.544 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-08-174145: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-c458203a8c1c0686ac3dcdf8a746a955 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-de186351b4c8a967c22e9306fc929f02, retrying]
May 09 09:39:18.544 - 158s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-08-174145: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-c458203a8c1c0686ac3dcdf8a746a955 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-de186351b4c8a967c22e9306fc929f02, retrying]

... 1 lines not shown

May 08 09:38:36.457 E ns/openshift-console pod/console-7dd7c8bbcf-2s2p5 node/libvirt-ppc64le-0-1-e-vk2zz-master-0 uid/1a202384-9485-452a-b888-4b162f200e0d container/console reason/ContainerExit code/2 cause/Error W0508 09:02:59.796028       1 main.go:229] Flag inactivity-timeout is set to less then 300 seconds and will be ignored!\nI0508 09:02:59.796077       1 main.go:359] cookies are secure!\nI0508 09:02:59.856898       1 main.go:836] Binding to [::]:8443...\nI0508 09:02:59.856938       1 main.go:838] using TLS\nI0508 09:03:02.861065       1 metrics.go:138] serverconfig.Metrics: Update ConsolePlugin metrics...\nI0508 09:03:02.935949       1 metrics.go:148] serverconfig.Metrics: Update ConsolePlugin metrics: &map[] (took 74.859965ms)\nI0508 09:03:04.858104       1 metrics.go:88] usage.Metrics: Count console users...\nI0508 09:03:05.263522       1 metrics.go:174] usage.Metrics: Update console users metrics: 0 kubeadmin, 0 cluster-admins, 0 developers, 0 unknown/errors (took 405.391722ms)\n
May 08 09:42:00.669 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-08-025152: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-00aeeefe49d0da91de569e3555ba91f9 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-989ae5f205d25512fbec57733d30ad74, retrying]
May 08 09:42:00.669 - 72s   E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-08-025152: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-00aeeefe49d0da91de569e3555ba91f9 expected bb00d4f6f57ab3a3fc11644a958a6e1781389a23 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-989ae5f205d25512fbec57733d30ad74, retrying]

... 1 lines not shown

May 07 09:48:24.311 E ns/openshift-monitoring pod/alertmanager-main-0 node/libvirt-ppc64le-0-0-e-246lv-worker-0-ngjwg uid/1236219e-531b-4433-8317-fb57bd99fc29 container/alertmanager reason/ContainerExit code/1 cause/Error ts=2024-05-07T09:48:21.881Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=rhaos-4.13-rhel-8, revision=c5f095c646ea4c1f9524833e8e3579b827a501aa)"\nts=2024-05-07T09:48:21.881Z caller=main.go:241 level=info build_context="(go=go1.19.13 X:strictfipsruntime, user=root@0a12bfb51770, date=20240425-19:11:18)"\nts=2024-05-07T09:48:21.914Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s\nts=2024-05-07T09:48:21.967Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/config_out/alertmanager.env.yaml\nts=2024-05-07T09:48:21.967Z caller=coordinator.go:118 level=error component=configuration msg="Loading configuration file failed" file=/etc/alertmanager/config_out/alertmanager.env.yaml err="open /etc/alertmanager/config_out/alertmanager.env.yaml: no such file or directory"\nts=2024-05-07T09:48:21.967Z caller=cluster.go:690 level=info component=cluster msg="gossip not settled but continuing anyway" polls=0 elapsed=52.037483ms\n
May 07 09:49:11.660 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-6bedd177a9aa39602df8c52d79d76933 expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-1fcb5baceb2268c11763be4f175b7a5c, retrying]
May 07 09:49:11.660 - 449s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-6bedd177a9aa39602df8c52d79d76933 expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-1fcb5baceb2268c11763be4f175b7a5c, retrying]

... 1 lines not shown

May 06 21:34:14.877 E ns/openshift-monitoring pod/alertmanager-main-0 node/libvirt-ppc64le-2-1-e-vdb89-worker-0-nb4r4 uid/c277f2df-6e8a-462f-9dd6-a25c990bb14e container/alertmanager reason/ContainerExit code/1 cause/Error ts=2024-05-06T21:34:12.040Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=rhaos-4.13-rhel-8, revision=c5f095c646ea4c1f9524833e8e3579b827a501aa)"\nts=2024-05-06T21:34:12.040Z caller=main.go:241 level=info build_context="(go=go1.19.13 X:strictfipsruntime, user=root@0a12bfb51770, date=20240425-19:11:18)"\nts=2024-05-06T21:34:12.081Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s\nts=2024-05-06T21:34:12.119Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/config_out/alertmanager.env.yaml\nts=2024-05-06T21:34:12.119Z caller=coordinator.go:118 level=error component=configuration msg="Loading configuration file failed" file=/etc/alertmanager/config_out/alertmanager.env.yaml err="open /etc/alertmanager/config_out/alertmanager.env.yaml: no such file or directory"\nts=2024-05-06T21:34:12.119Z caller=cluster.go:690 level=info component=cluster msg="gossip not settled but continuing anyway" polls=0 elapsed=37.936356ms\n
May 06 21:35:15.758 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-aca5bec3181661d16cef46414a12c8e8 expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-73c54cadbe90b2bfcbd818ba2551125e, retrying]
May 06 21:35:15.758 - 518s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-aca5bec3181661d16cef46414a12c8e8 expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-73c54cadbe90b2bfcbd818ba2551125e, retrying]

... 1 lines not shown

May 06 09:52:01.000 - 27s   E ns/openshift-image-registry route/test-disruption-reused disruption/image-registry connection/reused reason/DisruptionBegan ns/openshift-image-registry route/test-disruption-reused disruption/image-registry connection/reused stopped responding to GET requests over reused connections: Get "https://test-disruption-reused-openshift-image-registry.apps.libvirt-ppc64le-1-0-e881e.libvirt-ppc64le-1-0.ci/healthz": net/http: TLS handshake timeout
May 06 09:55:31.777 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-99f74972b25f0e83671e6b55b5714e4c expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-ee73ea99678d460b1744f5348153e0a8, retrying]
May 06 09:55:31.777 - 75s   E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-99f74972b25f0e83671e6b55b5714e4c expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-ee73ea99678d460b1744f5348153e0a8, retrying]

... 1 lines not shown

May 03 21:36:37.516 E ns/e2e-k8s-sig-apps-daemonset-upgrade-8806 pod/ds1-6htmp node/libvirt-ppc64le-0-2-e-4z2q2-worker-0-9t8cz uid/49225a8e-7c92-4440-86fc-5b0a4a6e975c container/app reason/TerminationStateCleared lastState.terminated was cleared on a pod (bug https://bugzilla.redhat.com/show_bug.cgi?id=1933760 or similar)
May 03 21:37:56.322 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-0a95f69d8ed3c2c2b1d23dca837ffd8c expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-77b65523f5e7c162af4b0b9712302554, retrying]
May 03 21:37:56.322 - 536s  E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-0a95f69d8ed3c2c2b1d23dca837ffd8c expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 1 (ready 1) out of 3 nodes are updating to latest configuration rendered-master-77b65523f5e7c162af4b0b9712302554, retrying]

... 1 lines not shown

May 03 09:59:49.717 E ns/openshift-multus pod/multus-admission-controller-d449c598b-xzzt7 node/libvirt-ppc64le-0-2-e-thqjh-master-1 uid/4043c4f1-4105-4fe5-8905-97b7b34f743e container/multus-admission-controller reason/ContainerExit code/137 cause/Error
May 03 10:03:15.053 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-3dd8d6deb8106fb9193ebd8e20d5e599 expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-40ae2730fffebfb4ca1d5068498084f4, retrying]
May 03 10:03:15.053 - 80s   E clusteroperator/machine-config condition/Degraded status/True reason/Unable to apply 4.13.0-0.nightly-ppc64le-2024-05-02-145025: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: controller version mismatch for rendered-master-3dd8d6deb8106fb9193ebd8e20d5e599 expected 88140227ee7d79c77092efd92b5a4363971333e2 has 33a010772d604aff2cb625d04a9469a47f53c96e: 2 (ready 2) out of 3 nodes are updating to latest configuration rendered-master-40ae2730fffebfb4ca1d5068498084f4, retrying]

... 1 lines not shown