| name | investigate-e2e-failure |
| description | This gives instructions on how to investigate end-to-end test failures. |
When to Use
Whenever the user asks "investigate the e2e failure 123", execute the instructions below.
Prerequisites
Check if the gh binary is installed.
Instructions
Download the overview
In order to prevent TLS issues, just execute gh with full permissions.
- Use the following command to get an overview about the failure:
$ gh run view --job 123
X main Test · 19282487500
Triggered via schedule about 2 months ago
X cloud-tests / e2e-tests / e2e (v1.34.0-kind, encryption-at-rest) in 10m48s (ID 123)
✓ Set up job
✓ Run actions/checkout@v5
✓ Install devbox
✓ Generate kustomized all-in-one install configs
✓ Extract k8s version/platform
X Run CI E2E tests
✓ Upload logs on failure
✓ Post Install devbox
✓ Post Run actions/checkout@v5
✓ Complete job
ANNOTATIONS
! No files were found with the provided path: output/**. No artifacts will be uploaded.
cloud-tests / e2e-tests / e2e (v1.34.0-kind, encryption-at-rest): .github#20
X Process completed with exit code 2.
cloud-tests / e2e-tests / e2e (v1.34.0-kind, encryption-at-rest): .github#2510
To see the logs for the failed steps, try: gh run view --log-failed --job=123
View this run on GitHub: https://github.com/mongodb/mongodb-atlas-kubernetes/actions/runs/19282487500
Find the corresponding e2e test
In this example we see that a test in test/e2e with the label "encryption-at-rest" failed.
In this case it is the test defined in test/e2e/encryption_at_rest/encryption_at_rest_test.go.
Analyze the failure
Use the following command to download the logs for the failed job:
$ gh run view --job 123 --log
Investigate
- Correlate the logs with the test code to identify the root cause of the failure.
- Look for error messages, stack traces, or any other indicators of what went wrong.
- Check for any recent changes in the codebase that might have introduced the failure.
- If necessary, reproduce the failure locally to gain a deeper understanding.
- Document your findings and suggest potential fixes or improvements to prevent similar failures in the future.
