Job:
periodic-ci-openshift-release-master-nightly-4.7-e2e-gcp-rt (all) - 5 runs, 20% failed, 200% of failures match = 40% impact
#1465326004124782592junit8 hours ago
        Code: 403,
        Message: "Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'.",
        Body: "{\n  \"error\": {\n    \"code\": 403,\n    \"message\": \"Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'.\",\n    \"errors\": [\n      {\n        \"message\": \"Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'.\",\n        \"domain\": \"usageLimits\",\n        \"reason\": \"rateLimitExceeded\"\n      }\n    ],\n    \"status\": \"PERMISSION_DENIED\"\n  }\n}\n",
        Header: nil,
#1465326004124782592junit8 hours ago
                Reason: "rateLimitExceeded",
                Message: "Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'.",
            },
#1465326004124782592junit8 hours ago
    }
    googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
#1463149750948728832junit6 days ago
    <*errors.errorString | 0xc0022505e0>: {
        s: "PV Create API error: persistentvolumes \"gcepd-\" is forbidden: error querying GCE PD volume e2e-e6d6a956-c4cb-45c5-97b9-4b16ca53cd3d: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded",
    }
    PV Create API error: persistentvolumes "gcepd-" is forbidden: error querying GCE PD volume e2e-e6d6a956-c4cb-45c5-97b9-4b16ca53cd3d: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
#1463149750948728832junit6 days ago
    <*errors.errorString | 0xc001ef3560>: {
        s: "PV Create API error: persistentvolumes \"gce-\" is forbidden: error querying GCE PD volume e2e-c6305568-5d7e-4cd3-8e91-d0da94e0b88e: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded",
    }
    PV Create API error: persistentvolumes "gce-" is forbidden: error querying GCE PD volume e2e-c6305568-5d7e-4cd3-8e91-d0da94e0b88e: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
#1463149750948728832junit6 days ago
    <*errors.errorString | 0xc0028ff760>: {
        s: "PV Create API error: persistentvolumes \"gcepd-\" is forbidden: error querying GCE PD volume e2e-ba9d7d7d-82fc-4850-95a5-8a22c9aa654b: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded",
    }
    PV Create API error: persistentvolumes "gcepd-" is forbidden: error querying GCE PD volume e2e-ba9d7d7d-82fc-4850-95a5-8a22c9aa654b: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
periodic-ci-openshift-release-master-ci-4.7-e2e-gcp (all) - 45 runs, 27% failed, 17% of failures match = 4% impact
#1463919486678601728junit4 days ago
level=error
level=error msg=Error: Error when reading or editing ComputeRouterNat "************************/us-central1/ci-op-m7xnm15h-b3a34-mm6cp-router/ci-op-m7xnm15h-b3a34-mm6cp-nat-master": googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
level=error
#1463919486678601728junit4 days ago
level=error
level=error msg=Error: Error when reading or editing Instance ci-op-m7xnm15h-b3a34-mm6cp-master-1: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
level=error
#1463919486678601728junit4 days ago
level=error
level=error msg=Error: Error when reading or editing Instance ci-op-m7xnm15h-b3a34-mm6cp-master-0: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
level=error
#1463758429817409536junit4 days ago
    <*errors.errorString | 0xc00262a3f0>: {
        s: "PV Create API error: persistentvolumes \"multizone-pv\" is forbidden: error querying GCE PD volume e2e-b8c3e98a-1d8c-4734-a3bc-8ae23873948c: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded",
    }
    PV Create API error: persistentvolumes "multizone-pv" is forbidden: error querying GCE PD volume e2e-b8c3e98a-1d8c-4734-a3bc-8ae23873948c: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
#1463758429817409536junit4 days ago
    <*errors.errorString | 0xc00251c420>: {
        s: "PV Create API error: persistentvolumes \"gcepd-\" is forbidden: error querying GCE PD volume e2e-90f05e54-29f7-4155-8126-e0d08d5c6f70: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded",
    }
    PV Create API error: persistentvolumes "gcepd-" is forbidden: error querying GCE PD volume e2e-90f05e54-29f7-4155-8126-e0d08d5c6f70: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
periodic-ci-openshift-release-master-ci-4.7-e2e-gcp-upgrade (all) - 44 runs, 9% failed, 25% of failures match = 2% impact
#1463919486640852992junit4 days ago
=error
level=error msg=Error: Error when reading or editing Target Pool "ci-op-5rpb4k9h-bbf29-r664z-api": googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
level=error
#1463919486640852992junit4 days ago
level=error
level=error msg=Error: Error when reading or editing Instance Group "ci-op-5rpb4k9h-bbf29-r664z-master-us-central1-b": googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
level=error
#1463919486640852992junit4 days ago
level=error
level=error msg=Error: Error when reading or editing Instance Group "ci-op-5rpb4k9h-bbf29-r664z-master-us-central1-a": googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
level=error
#1463919486640852992junit4 days ago
level=error
level=error msg=Error: Error when reading or editing Instance Group "ci-op-5rpb4k9h-bbf29-r664z-bootstrap": googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
level=error
periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-gcp-ovn-upgrade (all) - 4 runs, 100% failed, 25% of failures match = 25% impact
#1463166364599455744junit6 days ago
level=error
level=error msg=Error: Error when reading or editing Instance ci-op-07tlmrt1-8929c-47k8n-master-0: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'.
level=error msg=Details:
periodic-ci-openshift-release-master-ci-4.7-e2e-gcp-ovn (all) - 5 runs, 40% failed, 50% of failures match = 20% impact
#1463149746267885568junit6 days ago
    <*errors.errorString | 0xc001699df0>: {
        s: "PV Create API error: persistentvolumes \"multizone-pv\" is forbidden: error querying GCE PD volume e2e-714609c6-ea2e-4a5b-849d-fbf766f235c0: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded",
    }
    PV Create API error: persistentvolumes "multizone-pv" is forbidden: error querying GCE PD volume e2e-714609c6-ea2e-4a5b-849d-fbf766f235c0: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
#1463149746267885568junit6 days ago
    <*errors.errorString | 0xc0012b85d0>: {
        s: "PV Create API error: persistentvolumes \"gcepd-\" is forbidden: error querying GCE PD volume e2e-3f8b4522-d8c3-4223-95ae-445414d150f3: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded",
    }
    PV Create API error: persistentvolumes "gcepd-" is forbidden: error querying GCE PD volume e2e-3f8b4522-d8c3-4223-95ae-445414d150f3: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
#1463149746267885568junit6 days ago
    <*errors.errorString | 0xc001a62630>: {
        s: "PV Create API error: persistentvolumes \"multizone-pv\" is forbidden: error querying GCE PD volume e2e-1c950758-f0f9-467d-b610-485958c9c62b: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded",
    }
    PV Create API error: persistentvolumes "multizone-pv" is forbidden: error querying GCE PD volume e2e-1c950758-f0f9-467d-b610-485958c9c62b: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
periodic-ci-openshift-release-master-nightly-4.7-e2e-gcp (all) - 5 runs, 40% failed, 50% of failures match = 20% impact
#1463149749266812928junit6 days ago
    <*errors.errorString | 0xc00234c0a0>: {
        s: "PV Create API error: persistentvolumes \"gce-\" is forbidden: error querying GCE PD volume e2e-e0308e77-4c2a-4bd4-b9c2-dc5efffca48e: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded",
    }
    PV Create API error: persistentvolumes "gce-" is forbidden: error querying GCE PD volume e2e-e0308e77-4c2a-4bd4-b9c2-dc5efffca48e: googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
#1463149749266812928junit6 days ago
        Code: 403,
        Message: "Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'.",
        Body: "{\n  \"error\": {\n    \"code\": 403,\n    \"message\": \"Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'.\",\n    \"errors\": [\n      {\n        \"message\": \"Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'.\",\n        \"domain\": \"usageLimits\",\n        \"reason\": \"rateLimitExceeded\"\n      }\n    ],\n    \"status\": \"PERMISSION_DENIED\"\n  }\n}\n",
        Header: nil,
#1463149749266812928junit6 days ago
                Reason: "rateLimitExceeded",
                Message: "Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'.",
            },
#1463149749266812928junit6 days ago
    }
    googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
occurred
periodic-ci-openshift-release-master-ci-4.8-upgrade-from-stable-4.7-e2e-gcp-upgrade (all) - 54 runs, 31% failed, 6% of failures match = 2% impact
#1463144419526971392junit6 days ago
level=error
level=error msg=Error: Error when reading or editing ComputeFirewall "projects/************************/global/firewalls/ci-op-59c6t0qn-3b3f8-l9fj9-internal-network": googleapi: Error 403: Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:711936183532'., rateLimitExceeded
level=error

Found in 0.11% of runs (0.80% of failures) across 8208 total runs and 322 jobs (13.65% failed) in 102ms - clear search | chart view - source code located on github