galaxy-testing

name	galaxy-testing
description	Galaxy testing with pytest and run_tests.sh - run/write unit, integration, API, selenium tests. Use for: test execution, test failures, pytest errors, ApiTestCase patterns, test fixtures, writing new tests, debugging test failures, test/integration, lib/galaxy_test/api tests. CRITICAL: Always use ./run_tests.sh, never pytest directly.
argument-hint	[run|write|unit|api|integration]

Persona: You are a senior Galaxy QA engineer specializing in pytest and Galaxy's test infrastructure.

Arguments:

• $ARGUMENTS - Optional: "run", "write", "unit", "api", "integration" Examples: "", "run", "write unit", "api"

Parse $ARGUMENTS to determine which guidance to provide.

---

Galaxy Testing Guide

Galaxy uses pytest with a custom test runner script that sets up the proper environment.

CRITICAL: Always use ./run_tests.sh, never run pytest directly.

---

If $ARGUMENTS is empty or "run": Test Running Reference

Basic Test Running

Run integration tests (most common):

./run_tests.sh -integration test/integration/test_credentials.py

Run specific test method:

./run_tests.sh -integration test/integration/test_credentials.py::TestCredentialsApi::test_list_credentials

Run all tests in a directory:

./run_tests.sh -integration test/integration/

Test Type Flags

• -unit - Fast unit tests (no server, mocked dependencies)

  ./run_tests.sh -unit test/unit/managers/test_workflows.py
  

• -api - API endpoint tests (starts Galaxy server)

  ./run_tests.sh -api lib/galaxy_test/api/test_workflows.py
  

• -integration - Integration tests (full Galaxy setup)

  ./run_tests.sh -integration test/integration/test_vault.py
  

• -selenium - Browser-based E2E tests

  ./run_tests.sh -selenium test/integration_selenium/test_workflow_editor.py
  

• -framework - Test infrastructure tests

  ./run_tests.sh -framework test/framework/
  

Useful Flags

Show detailed output:

./run_tests.sh -integration test/integration/test_credentials.py --verbose_errors

Generate coverage report:

./run_tests.sh --coverage -integration test/integration/test_credentials.py

Debug mode (drop into pdb on failure):

./run_tests.sh --debug -integration test/integration/test_credentials.py

Run tests matching pattern:

./run_tests.sh -integration test/integration/test_credentials.py -k "test_create"

Show print statements:

./run_tests.sh -integration test/integration/test_credentials.py -s

Run with specific number of workers (parallel):

./run_tests.sh -integration test/integration/ -n 4

When Using pytest Directly (Advanced)

If you must use pytest directly (e.g., for IDE integration), use markers:

pytest -m "not slow" test/unit/
pytest -m "unit" test/unit/managers/test_workflows.py
pytest -m "integration" test/integration/test_credentials.py

But prefer ./run_tests.sh for normal usage.

---

If $ARGUMENTS is "write": Guide to Writing Tests

Ask the user what type of test they want to write:

1. Unit tests - Fast, isolated tests with mocked dependencies
2. API tests - Test API endpoints with Galaxy server
3. Integration tests - Full system tests with real database

Then provide guidance based on their choice (see sections below).

---

If $ARGUMENTS contains "unit": Unit Test Writing Guide

What Are Unit Tests?

Unit tests are fast, isolated tests that:

• Run without starting Galaxy server
• Use in-memory SQLite database
• Mock external dependencies
• Test individual manager/service methods
• Are located in test/unit/

Unit Test Structure

Location: test/unit/<module>/test_<class>.py

Base class: BaseTestCase from test.unit.app.managers.base

Example unit test:

"""
Unit tests for MyResourceManager.
"""
from galaxy import model
from galaxy.managers.myresources import MyResourceManager
from test.unit.app.managers.base import BaseTestCase


class TestMyResourceManager(BaseTestCase):
    """Unit tests for MyResourceManager."""

    def setUp(self):
        super().setUp()
        self.set_up_managers()

    def set_up_managers(self):
        """Set up managers under test."""
        self.manager = MyResourceManager(self.app)

    def test_create_myresource(self):
        """Test creating a resource."""
        # Arrange
        trans = self.trans  # MockTrans from BaseTestCase
        name = "Test Resource"

        # Act
        resource = self.manager.create(trans, name=name)
        self.session.flush()

        # Assert
        assert resource.name == name
        assert resource.user_id == trans.user.id
        assert resource.id is not None

    def test_get_myresource():
        
        
        resource = ._create_resource()

        
        retrieved = .manager.get(.trans, resource.)

        
         retrieved. == resource.
         retrieved.name == resource.name

     ():
        
         galaxy.exceptions  ObjectNotFound

         .assertRaises(ObjectNotFound):
            .manager.get(.trans, )

     ():
        
        
        ._create_resource()
        ._create_resource()

        
        resources = .manager.list_for_user(.trans)

        
         (resources) >= 
        names = [r.name  r  resources]
           names
           names

     ():
        
        
        resource = ._create_resource()
        new_name = 

        
        updated = .manager.update(.trans, resource., name=new_name)
        .session.flush()

        
         updated. == resource.
         updated.name == new_name

     ():
        
        
        resource = ._create_resource()

        
        .manager.delete(.trans, resource.)
        .session.flush()

        
         resource.deleted  

     ():
        
         galaxy.exceptions  ItemAccessibilityException

        
        other_user = ._create_user()
        other_trans = ._create_trans(user=other_user)
        resource = .manager.create(other_trans, name=)
        .session.flush()

        
         .assertRaises(ItemAccessibilityException):
            .manager.get(.trans, resource.)

     ():
        
        resource = .manager.create(.trans, name=name, **kwargs)
        .session.flush()
         resource

     ():
        
        user = model.User(email=email, username=email.split()[])
        .session.add(user)
        .session.flush()
         user

     ():
        
         galaxy_mock  MockTrans
         MockTrans(app=.app, user=user  .user)

Key Points for Unit Tests

• Extend BaseTestCase from test.unit.app.managers.base
• Use self.trans - Pre-configured MockTrans with test user
• Use self.session - SQLAlchemy session (in-memory SQLite)
• Call self.session.flush() after creates/updates to persist
• Override set_up_managers() to instantiate managers under test
• Use helper methods like _create_resource() for test data
• Test error cases with self.assertRaises()
• Follow AAA pattern - Arrange, Act, Assert

Available from BaseTestCase

self.app          # Galaxy application mock
self.trans        # MockTrans with test user
self.user         # Test user (admin)
self.session      # SQLAlchemy session
self.history      # Default test history

Running Unit Tests

# Run all unit tests for a manager
./run_tests.sh -unit test/unit/managers/test_myresources.py

# Run specific test
./run_tests.sh -unit test/unit/managers/test_myresources.py::TestMyResourceManager::test_create_myresource

# Run with coverage
./run_tests.sh --coverage -unit test/unit/managers/test_myresources.py

---

If $ARGUMENTS contains "api": API Test Writing Guide

What Are API Tests?

API tests:

• Start a Galaxy server
• Make HTTP requests to API endpoints
• Test request/response handling
• Verify status codes and response schemas
• Are located in lib/galaxy_test/api/

API Test Structure

Location: lib/galaxy_test/api/test_<resource>s.py

Base class: ApiTestCase from lib/galaxy_test/api/_framework

Example API test:

"""
API tests for MyResource endpoints.
"""
from galaxy_test.base.populators import DatasetPopulator
from ._framework import ApiTestCase


class TestMyResourcesApi(ApiTestCase):
    """Tests for /api/myresources endpoints."""

    def setUp(self):
        super().setUp()
        self.dataset_populator = DatasetPopulator(self.galaxy_interactor)

    def test_create_myresource(self):
        """Test POST /api/myresources."""
        payload = {
            "name": "Test Resource",
            "description": "Test description",
        }
        response = self._post("myresources", data=payload, json=True)
        self._assert_status_code_is(response, 201)

        resource = response.json()
        self._assert_has_keys(resource, "id", "name", "description", "create_time")
        assert resource["name"] == "Test Resource"
        assert resource["description"] == "Test description"

    def test_list_myresources(self):
        """Test GET /api/myresources."""
        # Create test data
        ._create_myresource()
        ._create_myresource()

        
        response = ._get()
        ._assert_status_code_is_ok(response)

        data = response.json()
           data
           data
         data[] >= 
         (data[]) >= 

     ():
        
        resource_id = ._create_myresource()

        response = ._get()
        ._assert_status_code_is_ok(response)

        resource = response.json()
         resource[] == resource_id
         resource[] == 

     ():
        
        resource_id = ._create_myresource()

        payload = {: }
        response = ._put(, data=payload, json=)
        ._assert_status_code_is_ok(response)

        updated = response.json()
         updated[] == 

     ():
        
        resource_id = ._create_myresource()

        response = ._delete()
        ._assert_status_code_is(response, )

        
        response = ._get()
        ._assert_status_code_is(response, )

     ():
        
        response = ._get()
        ._assert_status_code_is(response, )

     ():
        
        payload = {}  
        response = ._post(, data=payload, json=)
        ._assert_status_code_is(response, )

     ():
        
        
        resource_id = ._create_myresource()

        
         ._different_user():
            response = ._get()
            ._assert_status_code_is(response, )

     ():
        
        
        resource_id = ._create_myresource()

        
        response = ._get(, admin=)
        ._assert_status_code_is_ok(response)

     () -> :
        
        payload = {
            : name,
            : kwargs.get(, ),
        }
        response = ._post(, data=payload, json=)
        ._assert_status_code_is(response, )
         response.json()[]

Key Points for API Tests

• Extend ApiTestCase from lib/galaxy_test/api/_framework
• HTTP methods:
  • self._get(path) - GET request
  • self._post(path, data=..., json=True) - POST request
  • self._put(path, data=..., json=True) - PUT request
  • self._delete(path) - DELETE request
  • Paths are relative to /api/ (e.g., "myresources" → /api/myresources)
• Assertions:
  • self._assert_status_code_is(response, 200) - Check specific status
  • self._assert_status_code_is_ok(response) - Check 2xx status
  • self._assert_has_keys(obj, "key1", "key2") - Verify response structure
• User context:
  • Default: Regular user
  • admin=True parameter: Make request as admin
  • self._different_user() context manager: Switch to different user
• Test data:
  • Use helper methods like _create_myresource()
  • Use DatasetPopulator for creating datasets/histories

Additional ApiTestCase Features

Create test datasets:

def setUp(self):
    super().setUp()
    self.dataset_populator = DatasetPopulator(self.galaxy_interactor)

def test_with_dataset(self):
    history_id = self.dataset_populator.new_history()
    dataset = self.dataset_populator.new_dataset(history_id, content="test data")
    # Use dataset["id"] in your test

Test as different user:

with self._different_user():
    response = self._get("myresources")
    # This request is made as a different user

Test with admin privileges:

response = self._get("myresources/admin/all", admin=True)

Running API Tests

# Run all API tests for an endpoint
./run_tests.sh -api lib/galaxy_test/api/test_myresources.py

# Run specific test
./run_tests.sh -api lib/galaxy_test/api/test_myresources.py::TestMyResourcesApi::test_create_myresource

# Run with verbose output
./run_tests.sh -api lib/galaxy_test/api/test_myresources.py --verbose_errors

---

If $ARGUMENTS contains "integration": Integration Test Writing Guide

What Are Integration Tests?

Integration tests:

• Test full system integration
• Use real database (PostgreSQL optional)
• Test complex workflows and interactions
• Can customize Galaxy configuration
• Are located in test/integration/

Integration Test Structure

Location: test/integration/test_<feature>.py

Base class: IntegrationTestCase from lib/galaxy_test/driver/integration_util

Example integration test:

"""
Integration tests for MyResource with vault integration.
"""
from galaxy_test.driver import integration_util


class TestMyResourceIntegration(integration_util.IntegrationTestCase):
    """Integration tests for MyResource."""

    @classmethod
    def handle_galaxy_config_kwds(cls, config):
        """Customize Galaxy configuration for these tests."""
        super().handle_galaxy_config_kwds(config)
        config["vault_config_file"] = cls.vault_config_file
        config["enable_vault"] = True

    def setUp(self):
        super().setUp()

    def test_myresource_with_vault(self):
        """Test creating resource with vault backend."""
        payload = {
            "name": "Vault Resource",
            "vault_type": "hashicorp",
            "username": "vaultuser",
            "password": "vaultpass",
        }
        response = self.galaxy_interactor.post("myresources", data=payload)
        response.raise_for_status()

        resource = response.json()
        assert resource["vault_type"] == "hashicorp"

        # Verify stored in vault
        vault_data = self._get_from_vault(resource["id"])
        assert vault_data["username"] == 

     ():
        
        
        resource_id = ._create_myresource()

        
        workflow_id = ._create_workflow_with_resource(resource_id)

        
        history_id = .dataset_populator.new_history()
        response = .galaxy_interactor.post(
            ,
            data={
                : workflow_id,
                : history_id,
                : resource_id,
            }
        )
        response.raise_for_status()

        
        .dataset_populator.wait_for_history(history_id)

        
        datasets = .dataset_populator.get_history_datasets(history_id)
         (datasets) > 

     () -> :
        
        response = .galaxy_interactor.post(
            ,
            data={: name, : }
        )
        response.raise_for_status()
         response.json()[]

     ():
        
        
        vault = ._app.vault
         vault.read_secret()

Key Points for Integration Tests

• Extend IntegrationTestCase from lib/galaxy_test/driver.integration_util
• Customize config: Override handle_galaxy_config_kwds() to set Galaxy config
• HTTP requests: Use self.galaxy_interactor.get(), .post(), etc.
• Direct app access: self._app gives access to Galaxy application internals
• Populators: Same as API tests - DatasetPopulator, WorkflowPopulator
• Database access: self._app.model.context for SQLAlchemy session

Configuration Mixins

Use mixins to add common configuration:

from galaxy_test.driver.integration_util import (
    IntegrationTestCase,
    ConfiguresDatabaseVault,
)

class TestMyResourceWithVault(IntegrationTestCase, ConfiguresDatabaseVault):
    """Test with database vault configured."""

    @classmethod
    def handle_galaxy_config_kwds(cls, config):
        super().handle_galaxy_config_kwds(config)
        # Additional config here

Available mixins:

• ConfiguresDatabaseVault - Set up database vault
• ConfiguresObjectStores - Configure object stores
• UsesToolshed - Set up Tool Shed integration

Skip Decorators

Skip tests based on environment:

from galaxy_test.driver.integration_util import skip_unless_postgres, skip_unless_docker

@skip_unless_postgres()
def test_postgres_specific_feature(self):
    """Test that requires PostgreSQL."""
    pass

@skip_unless_docker()
def test_docker_specific_feature(self):
    """Test that requires Docker."""
    pass

Running Integration Tests

# Run integration tests
./run_tests.sh -integration test/integration/test_myresources.py

# Run specific test
./run_tests.sh -integration test/integration/test_myresources.py::TestMyResourceIntegration::test_myresource_with_vault

# Run with PostgreSQL
./run_tests.sh -integration test/integration/test_myresources.py --postgres

# Run with coverage
./run_tests.sh --coverage -integration test/integration/test_myresources.py

---

Test Best Practices

General Guidelines

1. Test naming: Use descriptive names that explain what is being tested

   • Good: test_create_myresource_with_valid_data
   • Bad: test_1, test_myresource

2. One assertion per test: Test one thing at a time

   • Good: Separate test_create, test_update, test_delete
   • Bad: One test_crud that does everything

3. Test error cases: Test both success and failure paths

   • Test 404, 403, 400, 422 responses
   • Test validation errors
   • Test access control

4. Use helper methods: Extract common setup into helper methods

   • _create_myresource(), _create_user(), etc.

5. Clean test data: Tests should be independent and repeatable

6. Follow AAA pattern:

   • Arrange: Set up test data
   • Act: Perform the operation
   • Assert: Verify the result

Common Patterns

Testing lists:

resources = response.json()["items"]
assert len(resources) >= 2
names = [r["name"] for r in resources]
assert "Resource 1" in names

Testing timestamps:

from datetime import datetime
resource = response.json()
assert resource["create_time"] is not None
create_time = datetime.fromisoformat(resource["create_time"])
assert create_time < datetime.now()

Testing pagination:

response = self._get("myresources?limit=10&offset=0")
data = response.json()
assert len(data["items"]) <= 10
assert data["total_count"] >= len(data["items"])

---

Additional Resources

Key test infrastructure files:

• lib/galaxy_test/api/_framework.py - ApiTestCase base class
• lib/galaxy_test/driver/integration_util.py - IntegrationTestCase base class
• test/unit/app/managers/base.py - Unit test base class
• galaxy_test/base/populators.py - Test data populators

Example test files to reference:

# Find recent API tests
ls -t lib/galaxy_test/api/test_*.py | head -5

# Find recent integration tests
ls -t test/integration/test_*.py | head -5

# Find unit tests
ls test/unit/managers/test_*.py

Running test suites:

# All unit tests
./run_tests.sh -unit test/unit/

# All API tests (slow)
./run_tests.sh -api lib/galaxy_test/api/

# All integration tests (very slow)
./run_tests.sh -integration test/integration/

---

Troubleshooting Tests

Test fails with "database locked"

• Cause: Multiple tests accessing SQLite concurrently
• Solution: Use pytest-xdist with -n flag or run serially

Test fails with "port already in use"

• Cause: Previous test server didn't shut down
• Solution: Kill Galaxy processes: pkill -f 'python.*galaxy'

Test fails with "fixture not found"

• Cause: Missing test dependency
• Solution: Check imports and base class

Integration test timeout

• Cause: Test waiting for long-running job
• Solution: Use wait_for_history() with longer timeout

Cannot import test module

• Cause: Python path not set correctly
• Solution: Always use ./run_tests.sh, not direct pytest

---

Quick Reference

Test Type	Location	Base Class	Use When
Unit	test/unit/	BaseTestCase	Testing manager/service logic
API	lib/galaxy_test/api/	ApiTestCase	Testing API endpoints
Integration	test/integration/	IntegrationTestCase	Testing full system integration
Selenium	test/integration_selenium/	SeleniumTestCase	Testing browser UI

Running tests:

• Unit: ./run_tests.sh -unit test/unit/...
• API: ./run_tests.sh -api lib/galaxy_test/api/...
• Integration: ./run_tests.sh -integration test/integration/...

Common assertions:

• self._assert_status_code_is(response, 200)
• self._assert_status_code_is_ok(response)
• self._assert_has_keys(obj, "key1", "key2")
• self.assertRaises(ExceptionType)

Common helpers:

• self._get(path), self._post(path, data=...), self._put(...), self._delete(...)
• self._different_user() - Context manager for different user
• DatasetPopulator(self.galaxy_interactor) - Create test datasets