local-airflow-unit-tests

name	local-airflow-unit-tests
description	Summarizes how to use docker to run a tagged Composer image for local testing.

Local Airflow Unit Tests with Docker

This skill describes how to run Cloud Composer images locally using docker to execute unit tests in an environment that matches the production Composer environment.

Prerequisites

• docker installed and running.
• Python 3 installed (to run the image tag helper script).

Steps

1. Get the Composer Tagged Image

Use the provided script to get the fully qualified Docker image tag for the desired Composer version. This script reads from cicd/composer_version.txt.

# From the repository root
IMAGE_TAG=$(python3 cicd/get_composer_tagged_image.py)
echo $IMAGE_TAG

This will output something like: us-docker.pkg.dev/cloud-airflow-releaser/airflow-worker-scheduler-2-10-5/airflow-worker-scheduler-2-10-5:composer-2-airflow-2.10.5

2. Run the Container with Docker

To run tests locally, you need to mount your workspace into the container so that it has access to your DAGs, requirements, and test files.

You can run the container interactively:

# Get the image tag
IMAGE_TAG=$(python3 cicd/get_composer_tagged_image.py)

# Run the container
docker run -it \
  -v $(pwd):/workspace \
  -w /workspace \
  --entrypoint /bin/bash \
  $IMAGE_TAG

3. Initialize and Run Tests (Inside the Container)

Once inside the container, you can run the test script directly:

/workspace/cicd/run_tests.sh

Alternatively, you can manually execute the steps below. Make sure that all test results are printed out via standard output:

#!/usr/bin/env bash
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Enable exit-on-error mode
set -eo pipefail

# Determine base directory whether mounted at repo root or inside cicd directory
if [ -d "/workspace/cicd" ]; then
    BASE_DIR="/workspace/cicd"
elif [ -d "/workspace/dags" ]; then
    BASE_DIR="/workspace"
else
    SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
    BASE_DIR="$SCRIPT_DIR"
fi

DAGS_DIR="$BASE_DIR/dags"
TESTS_DIR="$BASE_DIR/tests"
REQUIREMENTS_FILE="$DAGS_DIR/requirements.txt"

# Set up the Python user base directory where local packages will be installed
export PYTHONUSERBASE=/home/airflow/.local


 PATH=/bin:


 AIRFLOW_HOME=/home/airflow/airflow
 AIRFLOW__CORE__LOAD_EXAMPLES=False
 PYTHONDONTWRITEBYTECODE=1


pip list --format=freeze > /tmp/constraints.txt



sed -i  /tmp/constraints.txt


 [ -f  ]; 
    pip install --no-cache-dir --user pytest \
        --requirement  \
        --constraint /tmp/constraints.txt

    pip install --no-cache-dir --user pytest \
        --constraint /tmp/constraints.txt





SITE_PACKAGES=$(python3 -c  2>/dev/null ||  )
 [ -d  ]; 
      -p  || 
      -R airflow:  || 



 AIRFLOW__API__AUTH_BACKENDS=airflow.api.auth.backend.basic_auth
 AIRFLOW__CORE__DAGS_FOLDER=


airflow standalone > /dev/null &


 
airflow db check
 


 
RETRIES=60
 curl -sf http://localhost:8080 > /dev/null || [  -eq 0 ]; 
     2
    RETRIES=$((RETRIES - ))


 [  -eq 0 ]; 
     
     1

 


airflow dags list


python3 -m pytest -o cache_dir=/tmp/.pytest_cache -vv -s 

4. Fix DAGs and Tests

Make any necessary Airflow DAG code corrections or refactors to get the tests passing. Do not modify the semantic logic of any DAGs or tests. Do not modify any thresholds or constants being checked in the tests.