// Check if a failing test is a known upstream flake in Chromium's LUCI Analysis database. Provides flakiness statistics, verdict, and recommendation for filter file decisions. Triggers on: check upstream flake, is this test flaky upstream, check luci flakiness, upstream flake check.
Add a new best practice to the appropriate doc. Checks for duplicates, assigns stable IDs, creates new category docs if needed. Triggers on: add best practice, new best practice, add bp, new bp.
"Create an uplift PR that cherry-picks merged PRs from a contributor into a target branch (beta or release). Defaults to broad eligibility; the scope can be narrowed via a free-form description (e.g. 'only automated test and crash fixes'). Triggers on: /uplift, create uplift, uplift PRs."
Review PRs in the configured PR repository for best practices violations. Supports single PR (#12345), state filter (open/closed/all), and auto mode for cron. Triggers on: review prs, review recent prs, /review-prs, check prs for best practices.
Run all preflight checks (format, gn_check, presubmit, build, tests) to make sure the current work is ready for review.
Delete local branches whose PRs have been merged upstream. Checks GitHub PR status for each branch. Triggers on: clean branches, delete merged branches, prune branches, branch cleanup.
Audit and fix best practices docs for stale references, duplicates, obsolete content, and formatting issues. Triggers on: fix bp docs, fix best practice docs, audit bp docs.
| name | check-upstream-flake |
| description | Check if a failing test is a known upstream flake in Chromium's LUCI Analysis database. Provides flakiness statistics, verdict, and recommendation for filter file decisions. Triggers on: check upstream flake, is this test flaky upstream, check luci flakiness, upstream flake check. |
| argument-hint | <TestSuite.TestMethod> |
| disable-model-invocation | true |
Check if a failing test is a known upstream flake in the Chromium LUCI Analysis
database. This queries the REST API at analysis.api.luci.app to retrieve
historical pass/fail/flake data for a test in the Chromium CI infrastructure.
When invoked with a test name:
# Basic: check a specific test (default 30-day lookback)
python3 ./script/check-upstream-flake.py "WebUIURLLoaderFactoryTest.RangeRequest"
# Longer lookback window
python3 ./script/check-upstream-flake.py "WebUIURLLoaderFactoryTest.RangeRequest" --days 60
# JSON output (for programmatic use)
python3 ./script/check-upstream-flake.py "WebUIURLLoaderFactoryTest.RangeRequest" --json
# Search by test class name (finds all methods)
python3 ./script/check-upstream-flake.py "WebUIURLLoaderFactoryTest"
Arguments:
test_name (required): Test name or substring to search for--days N: Lookback window in days (default: 30, max: 90)--json: Output JSON instead of markdownExit codes:
0: Success (results found and reported)1: Error (network, API, etc.)2: No matching test IDs foundThe script produces one of five verdicts:
| Verdict | Flake Rate | Action |
|---|---|---|
| Known upstream flake | >= 5% | Safe to add to filter file. Document upstream flakiness in the filter comment. |
| Occasional upstream failures | 1-5% | Consider filtering. Document findings. May still warrant investigation. |
| Stable upstream | < 1% | Investigate Brave-specific causes. The test is stable in Chromium, so Brave code changes are likely causing the failure. |
| Insufficient data | N/A (<10 verdicts) | Cannot determine from upstream data. Manual investigation needed. |
| Not found | N/A | Test not in Chromium database. May be Brave-specific or use a different ID format. |
Flake rate is calculated as (failed + flaky) / (passed + failed + flaky).
Skipped and precluded verdicts are excluded from the rate.
brave/chromium_src/ overrides in related directoriesThe script uses the LUCI Analysis REST API (pRPC protocol):
POST https://analysis.api.luci.app/prpc/luci.analysis.v1.TestHistory/QueryTestsPOST https://analysis.api.luci.app/prpc/luci.analysis.v1.TestHistory/QueryStatsPOST https://analysis.api.luci.app/prpc/luci.analysis.v1.TestHistory/QueryNo authentication is required for public Chromium data.
Test IDs in LUCI follow the format:
ninja://{gn_path}:{target}/{TestSuite}.{TestMethod}