Configure and generate rich Allure test reports with test categorization, historical trends, environment details, and CI/CD integration for comprehensive test visibility
Configure and generate rich Allure test reports with test categorization, historical trends, environment details, and CI/CD integration for comprehensive test visibility
Allure is an open-source test reporting framework that produces rich, interactive HTML reports from test execution results. Unlike basic test reporters that show pass/fail summaries, Allure provides detailed test categorization, step-by-step execution breakdowns, attachment support for screenshots, logs, and videos, historical trend tracking across builds, and environment metadata. This skill guides AI coding agents through configuring Allure reporters for popular testing frameworks, annotating tests with meaningful metadata, integrating with CI/CD pipelines, and establishing report hosting strategies that give teams comprehensive test visibility.
Core Principles
Reports Serve Multiple Audiences: A good test report provides quick pass/fail summaries for managers, detailed failure analysis for developers, trend data for QA leads, and categorized views for test strategists. Allure's multi-view design supports all these personas from a single report.
Annotations Are Documentation: Test step annotations, severity labels, and feature/story categorization serve as living documentation of test intent. Well-annotated tests in Allure reports communicate what is being tested and why without requiring code access.
Attachments Accelerate Debugging: Screenshots, DOM snapshots, network logs, and video recordings attached to test steps eliminate the need to reproduce failures locally. Every failure should carry sufficient attachments for diagnosis from the report alone.
History Reveals Patterns: A single test run is a snapshot. Historical trend data across builds reveals flaky tests that oscillate between pass and fail, degrading tests with gradually increasing failures, and regression patterns that correlate with specific changes.
Categories Group Failures by Root Cause: Allure categories classify failures by type (product defect, test defect, infrastructure issue) rather than by test name. This grouping accelerates triage by surfacing the most common failure modes across the entire suite.
Environment Context Is Non-Negotiable: Test results without environment information (browser version, OS, API version, deployment target) are incomplete. The same test can produce different results across environments, and the report must capture this context.
Reports Must Be Accessible: Test reports that exist only on a developer's local machine provide no team value. Reports must be published to a shared location where all stakeholders can access them without technical setup.
// config/allure-categories.json[{"name":"Product Defects","description":"Failures caused by actual application bugs","matchedStatuses":["failed"],"messageRegex":".*Expected .* (to be|to have|to contain|to equal).*"},{"name":"Test Infrastructure Issues","description":"Failures caused by test environment problems","matchedStatuses":["broken"],"messageRegex":".*(timeout|ECONNREFUSED|ENOTFOUND|ETIMEDOUT|net::ERR|Navigation).*"},{"name":"Element Not Found","description":"Failures where expected UI elements are missing","matchedStatuses":["failed"],"messageRegex":".*(locator|selector|element).*(not found|not visible|not attached).*"},{"name":"API Errors","description":"Failures in API response validation","matchedStatuses":["failed"],"messageRegex":".*(status code|response|4\\d{2}|5\\d{2}).*"},{"name":"Data Setup Failures","description":"Failures in test fixture or data preparation","matchedStatuses":["broken"],"traceRegex":".*(beforeAll|beforeEach|fixture|setup).*"},{"name":"Outdated Tests","description":"Tests that need updating due to application changes","matchedStatuses":["failed"],"messageRegex":".*(deprecated|removed|changed|no longer).*"}]
Copying Categories to Results
#!/bin/bash# scripts/generate-report.sh# Copy categories to allure-results (must be present before generation)cp config/allure-categories.json allure-results/categories.json
# Generate environment properties
npx ts-node scripts/generate-environment.ts
# Copy history from previous report (for trends)if [ -d "allure-report/history" ]; thencp -r allure-report/history allure-results/history
fi# Generate the report
npx allure generate allure-results --clean -o allure-report
echo"Report generated at allure-report/index.html"
Historical Trend Tracking
Preserving History Across Builds
#!/bin/bash# scripts/setup-history.sh# Run before each test execution to preserve historical data
HISTORY_DIR="allure-results/history"
REPORT_HISTORY="allure-report/history"# If a previous report exists, copy its history to the new results directoryif [ -d "$REPORT_HISTORY" ]; thenmkdir -p "$HISTORY_DIR"cp -r "$REPORT_HISTORY/"* "$HISTORY_DIR/"echo"History preserved from previous report"elseecho"No previous history found (first run)"fi
CI History Preservation with Artifacts
# In a GitHub Actions workflow, history is preserved via artifacts-name:Downloadpreviousreporthistoryuses:actions/download-artifact@v4with:name:allure-report-historypath:allure-results/historycontinue-on-error:true# First run will not have history# After report generation-name:Savereporthistoryuses:actions/upload-artifact@v4with:name:allure-report-historypath:allure-report/historyretention-days:30
CI Integration
GitHub Actions Complete Workflow
# .github/workflows/test-and-report.ymlname:TestandReporton:push:branches: [main, develop]
pull_request:branches: [main]
permissions:contents:readpages:writeid-token:writejobs:test:runs-on:ubuntu-lateststeps:-uses:actions/checkout@v4-uses:actions/setup-node@v4with:node-version:20-run:npmci-run:npxplaywrightinstall--with-deps# Download previous Allure history for trends-name:RestoreAllurehistoryuses:actions/download-artifact@v4with:name:allure-historypath:allure-results/historycontinue-on-error:true# Run tests-name:RunPlaywrighttestsrun:npxplaywrighttestcontinue-on-error:trueenv:BASE_URL:${{secrets.STAGING_URL}}TEST_ENV:ci# Generate environment properties-name:Generateenvironmentinforun:|
cat > allure-results/environment.properties << EOF
Browser=Chromium
OS=Ubuntu (CI)
Node.Version=$(node --version)
Test.Environment=CI
Git.Commit=${{ github.sha }}
Git.Branch=${{ github.ref_name }}
Build.Number=${{ github.run_number }}
PR.Number=${{ github.event.pull_request.number || 'N/A' }}
EOF
# Copy categories-name:SetupAllurecategoriesrun:cpconfig/allure-categories.jsonallure-results/categories.json# Generate report-name:GenerateAllureReportrun:|
npm install -g allure-commandline
allure generate allure-results --clean -o allure-report
# Save history for next run-name:SaveAllurehistoryuses:actions/upload-artifact@v4if:always()with:name:allure-historypath:allure-report/historyretention-days:60# Upload full report-name:UploadAllureReportuses:actions/upload-artifact@v4if:always()with:name:allure-reportpath:allure-report/retention-days:30# Deploy report to GitHub Pagesdeploy-report:needs:testif:github.ref=='refs/heads/main'runs-on:ubuntu-latestenvironment:name:github-pagesurl:${{steps.deployment.outputs.page_url}}steps:-uses:actions/download-artifact@v4with:name:allure-reportpath:allure-report-uses:actions/configure-pages@v4-uses:actions/upload-pages-artifact@v3with:path:allure-report-id:deploymentuses:actions/deploy-pages@v4
Jenkins Pipeline Integration
// Jenkinsfile
pipeline {
agent any
stages {
stage('Test') {
steps {
sh 'npm ci'
sh 'npx playwright install --with-deps'
sh 'npx playwright test || true'
}
post {
always {
// Copy environment properties
sh '''
echo "Browser=Chromium" > allure-results/environment.properties
echo "Build.Number=${BUILD_NUMBER}" >> allure-results/environment.properties
echo "Git.Commit=${GIT_COMMIT}" >> allure-results/environment.properties
echo "Node.Version=$(node --version)" >> allure-results/environment.properties
'''
// Jenkins Allure plugin generates report and preserves history
allure([
includeProperties: true,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'allure-results']]
])
}
}
}
}
}
Allure TestOps Overview
Allure TestOps is the commercial companion to Allure Report, providing centralized test management, real-time dashboards, and analytics across multiple projects.
# docker-compose.yml for self-hosted Allure report serverversion:'3.8'services:allure-server:image:frankescobar/allure-docker-service:latestports:-"5050:5050"environment:CHECK_RESULTS_EVERY_SECONDS:5KEEP_HISTORY:25volumes:-./allure-results:/app/allure-results-./allure-reports:/app/default-reports
Configuration
Package.json Scripts
{"scripts":{"test":"playwright test","test:report":"playwright test && npm run allure:generate","allure:generate":"bash scripts/generate-report.sh","allure:open":"allure open allure-report","allure:serve":"allure serve allure-results","allure:clean":"rm -rf allure-results allure-report","allure:history":"bash scripts/setup-history.sh","allure:publish":"ts-node scripts/publish-report.ts"}}
Allure Commandline Installation
# Via npm (recommended for JavaScript projects)
npm install -g allure-commandline
# Via Homebrew (macOS)
brew install allure
# Via scoop (Windows)
scoop install allure
# Verify installation
allure --version
Best Practices
Annotate every test with severity. Use Allure severity levels (blocker, critical, normal, minor, trivial) consistently. This enables filtering the report by severity during triage sessions.
Organize tests with epic/feature/story hierarchy. Map tests to the product feature hierarchy so the Allure Behaviors view reflects the actual product structure. This helps stakeholders navigate reports by business functionality.
Attach screenshots and videos on failure only. Recording screenshots and videos for all tests inflates report size without benefit. Configure screenshot: 'only-on-failure' and video: 'retain-on-failure'.
Use meaningful step descriptions. Steps should describe intent ("Add product to cart"), not implementation ("Click button with data-testid add-to-cart"). Report readers may not have code access.
Configure categories to match your failure taxonomy. Customize categories.json to classify failures into actionable groups. Default categories are too generic for effective triage.
Preserve history across CI builds. Without history, trend charts are empty and test retries lack context. Use CI artifacts or external storage to maintain history for at least 20-30 builds.
Generate environment properties dynamically. Hardcoded environment files become stale. Generate them from CI environment variables and git metadata at report generation time.
Attach API request/response pairs for API tests. When API tests fail, the request payload and response body attached to the report step eliminate the need to reproduce the failure locally.
Include links to test case management and issue tracking. Use allure.link() and allure.issue() to connect report entries to external systems. This creates bidirectional traceability between reports and project management tools.
Host reports on a persistent URL. Ephemeral reports in CI artifacts are hard to share. Deploy to GitHub Pages, S3, or a dedicated report server where stakeholders can always find the latest report.
Set up Slack or email notifications for report availability. After CI generates a report, notify the team with a direct link. Reports that nobody opens provide no value.
Review report trends weekly. Schedule a brief weekly review of the Allure trend view to catch increasing failure rates, growing test suite duration, or emerging flaky test patterns before they become systemic problems.
Anti-Patterns to Avoid
Generating reports without categories configuration. Without categories, all failures appear in a single undifferentiated list. Categories enable meaningful failure classification that accelerates triage.
Over-attaching large files to every test. Attaching multi-megabyte videos or full HAR files to every test (including passing tests) creates enormous reports that are slow to generate, upload, and browse.
Using generic step names. Steps labeled "Step 1", "Step 2", or "Verification" provide no value. Each step name should clearly describe what is happening and why.
Ignoring the Allure report trend view. Running Allure without preserving history across builds discards the most valuable feature: the ability to see trends and identify degradation over time.
Skipping environment configuration. A report without environment details cannot be interpreted correctly. The same test result means different things on Chrome vs Firefox, or staging vs production.
Treating report generation as an afterthought. Report configuration should be established early in the project, not bolted on when stakeholders demand visibility. Retrofit is always harder than upfront setup.
Not cleaning allure-results between runs. If old results are not cleared before new test runs, the report will contain stale data from previous executions, creating confusion about current test status.
Debugging Tips
Check allure-results directory for raw data. If the report looks wrong, inspect the JSON files in allure-results/. Each test produces a -result.json file containing all metadata, steps, and attachment references.
Verify categories.json is in allure-results before generation. The categories file must be present in the results directory when the report is generated. Placing it only in the config directory without copying produces a report with no category classifications.
Validate environment.properties format. The file must use key=value format with no quotes around values. Malformed properties are silently ignored, resulting in missing environment information in the report.
Check attachment file references. If attachments show as broken links in the report, verify that the attachment files exist in allure-results/ and that the file names in the result JSON match the actual files on disk.
Use allure serve for quick local preview. The allure serve allure-results command generates a temporary report and opens it in a browser without creating the persistent allure-report directory. This is the fastest way to preview results during development.
Verify allure-commandline version compatibility. Allure Report and the framework reporters must use compatible versions. Check the compatibility matrix in the Allure documentation if reports appear empty or malformatted.
Inspect the history directory structure. Trend charts require a specific directory structure in allure-results/history/. If trends are not appearing, verify that history.json and history-trend.json from the previous report were correctly copied to the results directory.
Check for conflicting reporters. Running multiple reporters that write to the same output directory can cause data corruption. Ensure each reporter writes to a distinct directory or uses the framework's built-in multi-reporter support.
Review CI artifact retention policies. If history suddenly stops working in CI, check whether artifact retention policies expired. History artifacts need to persist across builds, so set retention days appropriately (30-60 days minimum).
Test report generation locally before CI. Always verify that allure generate allure-results -o allure-report works locally with representative test data before debugging CI-specific report issues.
