ci-cd

Guides understanding and working with Apache Beam's CI/CD system using GitHub Actions. Use when debugging CI failures, understanding test workflows, or modifying CI configuration.

Jump to install

Repository: apache/beam
Last source activity: February 9, 2026 at 14:41
Detected SKILL.md language: English
Stars: 8,639
Forks: 4,622

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/apache/beam --skill ci-cd

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

SKILL.md

Source instructions · Read-only preview

Back to search Back to installation

developing-new-io-connectors

End-to-end guide on developing new Apache Beam I/O connectors correctly, including core IO transforms, SchemaTransforms, URN proto definitions, Managed API integration, cross-language expansion service, and testing.

2026-06-11

io-connectors

Guides development and usage of I/O connectors in Apache Beam. Use when working with I/O connectors, creating new connectors, or debugging data source/sink issues.

2026-06-11

yaml-development

Guides YAML SDK development in Apache Beam, including environment setup, testing, and key concepts. Use when working with Beam YAML code in sdks/python/apache_beam/yaml/.

2026-05-19

View all from this repository

name	ci-cd
description	Guides understanding and working with Apache Beam's CI/CD system using GitHub Actions. Use when debugging CI failures, understanding test workflows, or modifying CI configuration.

CI/CD in Apache Beam

Overview

Apache Beam uses GitHub Actions for CI/CD. Workflows are located in .github/workflows/.

Workflow Types

PreCommit Workflows

Run on PRs and merges
Validate code changes before merge
Naming: beam_PreCommit_*.yml

PostCommit Workflows

Run after merge and on schedule
More comprehensive testing
Naming: beam_PostCommit_*.yml

Scheduled Workflows

Run nightly on master
Check for external dependency impacts
Tag master with nightly-master

Key Workflows

PreCommit

Workflow	Description
`beam_PreCommit_Java.yml`	Java build and tests
`beam_PreCommit_Python.yml`	Python tests
`beam_PreCommit_Go.yml`	Go tests
`beam_PreCommit_RAT.yml`	License header checks
`beam_PreCommit_Spotless.yml`	Code formatting

PostCommit - Java

Workflow	Description
`beam_PostCommit_Java.yml`

Workflow	Description
`beam_PostCommit_Python.yml`	Full Python test suite
`beam_PostCommit_Python_ValidatesRunner_*.yml`	Runner validation
`beam_PostCommit_Python_Examples_*.yml`	Examples

Workflow	Description
`beam_LoadTests_*.yml`	Load testing
`beam_PerformanceTests_*.yml`	I/O performance

Workflow	Purpose
`cut_release_branch.yml`	Create release branch
`build_release_candidate.yml`	Build RC
`finalize_release.yml`	Finalize release
`publish_github_release_notes.yml`	Publish notes

ci-cd

Source facts

Install options

Review the source files

More from this repository

More from this repository

CI/CD in Apache Beam

Overview

Workflow Types

PreCommit Workflows

PostCommit Workflows

Scheduled Workflows

Key Workflows

PreCommit

PostCommit - Java

CI/CD in Apache Beam

Overview

Workflow Types

PreCommit Workflows

PostCommit Workflows

Scheduled Workflows

Key Workflows

PreCommit

PostCommit - Java

PostCommit - Python

Load & Performance Tests

Triggering Tests

Automatic

Triggering Specific Workflows

Workflow Dispatch

Understanding Test Results

Finding Logs

Common Failure Patterns

Flaky Tests

Timeout

Resource Exhaustion

GCP Credentials

Self-hosted vs GitHub-hosted Runners

Self-hosted (majority of workflows)

GitHub-hosted

Workflow Structure

Local Debugging

Run Same Commands as CI

Common Issues

Snapshot Builds

Locations

Release Workflows