galaxy-testing

Persona: You are a senior Galaxy QA engineer specializing in pytest and Galaxy's test infrastructure. Arguments: - $ARGUMENTS - Optional: "run", "write", "unit", "api", "integration" Examples: "", "run", "write unit", "api" Parse $ARGUMENTS to determine which guidance to provide. --- ## Galaxy Testing Guide Galaxy uses **pytest** with a custom test runner script that sets up the proper environment. **CRITICAL:** Always use `./run_tests.sh`, never run `pytest` directly. --- ## If $ARGUMENTS is empty or "run": Test Running Reference ### Basic Test Running **Run integration tests (most common):** ```bash ./run_tests.sh -integration test/integration/test_credentials.py ``` **Run specific test method:** ```bash ./run_tests.sh -integration test/integration/test_credentials.py::TestCredentialsApi::test_list_credentials ``` **Run all tests in a directory:** ```bash ./run_tests.sh -integration test/integration/ ``` ### Test Type Flags - **`-unit`** - Fast unit tests (no server, mocked dependencies) ```bash ./run_tests.sh -unit test/unit/managers/test_workflows.py ``` - **`-api`** - API endpoint tests (starts Galaxy server) ```bash ./run_tests.sh -api lib/galaxy_test/api/test_workflows.py ``` - **`-integration`** - Integration tests (full Galaxy setup) ```bash ./run_tests.sh -integration test/integration/test_vault.py ``` - **`-selenium`** - Browser-based E2E tests ```bash ./run_tests.sh -selenium test/integration_selenium/test_workflow_editor.py ``` - **`-framework`** - Test infrastructure tests ```bash ./run_tests.sh -framework test/framework/ ``` ### Useful Flags **Show detailed output:** ```bash ./run_tests.sh -integration test/integration/test_credentials.py --verbose_errors ``` **Generate coverage report:** ```bash ./run_tests.sh --coverage -integration test/integration/test_credentials.py ``` **Debug mode (drop into pdb on failure):** ```bash ./run_tests.sh --debug -integration test/integration/test_credentials.py ``` **Run tests matching pattern:** ```bash ./run_tests.sh -integration test/integration/test_credentials.py -k "test_create" ``` **Show print statements:** ```bash ./run_tests.sh -integration test/integration/test_credentials.py -s ``` **Run with specific number of workers (parallel):** ```bash ./run_tests.sh -integration test/integration/ -n 4 ``` ### When Using pytest Directly (Advanced) If you must use pytest directly (e.g., for IDE integration), use markers: ```bash pytest -m "not slow" test/unit/ pytest -m "unit" test/unit/managers/test_workflows.py pytest -m "integration" test/integration/test_credentials.py ``` But prefer `./run_tests.sh` for normal usage. --- ## If $ARGUMENTS is "write": Guide to Writing Tests Ask the user what type of test they want to write: 1. **Unit tests** - Fast, isolated tests with mocked dependencies 2. **API tests** - Test API endpoints with Galaxy server 3. **Integration tests** - Full system tests with real database Then provide guidance based on their choice (see sections below). --- ## If $ARGUMENTS contains "unit": Unit Test Writing Guide ### What Are Unit Tests? Unit tests are fast, isolated tests that: - Run without starting Galaxy server - Use in-memory SQLite database - Mock external dependencies - Test individual manager/service methods - Are located in `test/unit/` ### Unit Test Structure **Location:** `test/unit/<module>/test_<class>.py` **Base class:** `BaseTestCase` from `test.unit.app.managers.base` **Example unit test:** ```python """ Unit tests for MyResourceManager. """ from galaxy import model from galaxy.managers.myresources import MyResourceManager from test.unit.app.managers.base import BaseTestCase class TestMyResourceManager(BaseTestCase): """Unit tests for MyResourceManager.""" def setUp(self): super().setUp() self.set_up_managers() def set_up_managers(self): """Set up managers under test.""" self.manager = MyResourceManager(self.app) def test_create_myresource(self): """Test creating a resource.""" # Arrange trans = self.trans # MockTrans from BaseTestCase name = "Test Resource" # Act resource = self.manager.create(trans, name=name) self.session.flush() # Assert assert resource.name == name assert resource.user_id == trans.user.id assert resource.id is not None def test_get_myresource(self): """Test getting a resource by ID.""" # Arrange resource = self._create_resource("Test Resource") # Act retrieved = self.manager.get(self.trans, resource.id) # Assert assert retrieved.id == resource.id assert retrieved.name == resource.name def test_get_nonexistent_myresource_raises_not_found(self): """Test that getting nonexistent resource raises exception.""" from galaxy.exceptions import ObjectNotFound with self.assertRaises(ObjectNotFound): self.manager.get(self.trans, 99999) def test_list_myresources_for_user(self): """Test listing resources for current user.""" # Arrange self._create_resource("Resource 1") self._create_resource("Resource 2") # Act resources = self.manager.list_for_user(self.trans) # Assert assert len(resources) >= 2 names = [r.name for r in resources] assert "Resource 1" in names assert "Resource 2" in names def test_update_myresource(self): """Test updating a resource.""" # Arrange resource = self._create_resource("Original Name") new_name = "Updated Name" # Act updated = self.manager.update(self.trans, resource.id, name=new_name) self.session.flush() # Assert assert updated.id == resource.id assert updated.name == new_name def test_delete_myresource(self): """Test soft-deleting a resource.""" # Arrange resource = self._create_resource("To Delete") # Act self.manager.delete(self.trans, resource.id) self.session.flush() # Assert assert resource.deleted is True def test_cannot_access_other_user_resource(self): """Test access control for other users' resources.""" from galaxy.exceptions import ItemAccessibilityException # Arrange other_user = self._create_user("other@example.com") other_trans = self._create_trans(user=other_user) resource = self.manager.create(other_trans, name="Other User Resource") self.session.flush() # Act & Assert with self.assertRaises(ItemAccessibilityException): self.manager.get(self.trans, resource.id) def _create_resource(self, name: str, **kwargs): """Helper to create a test resource.""" resource = self.manager.create(self.trans, name=name, **kwargs) self.session.flush() return resource def _create_user(self, email: str): """Helper to create a test user.""" user = model.User(email=email, username=email.split("@")[0]) self.session.add(user) self.session.flush() return user def _create_trans(self, user=None): """Helper to create a transaction context for a user.""" from galaxy_mock import MockTrans return MockTrans(app=self.app, user=user or self.user) ``` ### Key Points for Unit Tests - **Extend `BaseTestCase`** from `test.unit.app.managers.base` - **Use `self.trans`** - Pre-configured MockTrans with test user - **Use `self.session`** - SQLAlchemy session (in-memory SQLite) - **Call `self.session.flush()`** after creates/updates to persist - **Override `set_up_managers()`** to instantiate managers under test - **Use helper methods** like `_create_resource()` for test data - **Test error cases** with `self.assertRaises()` - **Follow AAA pattern** - Arrange, Act, Assert ### Available from BaseTestCase ```python self.app # Galaxy application mock self.trans # MockTrans with test user self.user # Test user (admin) self.session # SQLAlchemy session self.history # Default test history ``` ### Running Unit Tests ```bash # Run all unit tests for a manager ./run_tests.sh -unit test/unit/managers/test_myresources.py # Run specific test ./run_tests.sh -unit test/unit/managers/test_myresources.py::TestMyResourceManager::test_create_myresource # Run with coverage ./run_tests.sh --coverage -unit test/unit/managers/test_myresources.py ``` --- ## If $ARGUMENTS contains "api": API Test Writing Guide ### What Are API Tests? API tests: - Start a Galaxy server - Make HTTP requests to API endpoints - Test request/response handling - Verify status codes and response schemas - Are located in `lib/galaxy_test/api/` ### API Test Structure **Location:** `lib/galaxy_test/api/test_<resource>s.py` **Base class:** `ApiTestCase` from `lib/galaxy_test/api/_framework` **Example API test:** ```python """ API tests for MyResource endpoints. """ from galaxy_test.base.populators import DatasetPopulator from ._framework import ApiTestCase class TestMyResourcesApi(ApiTestCase): """Tests for /api/myresources endpoints.""" def setUp(self): super().setUp() self.dataset_populator = DatasetPopulator(self.galaxy_interactor) def test_create_myresource(self): """Test POST /api/myresources.""" payload = { "name": "Test Resource", "description": "Test description", } response = self._post("myresources", data=payload, json=True) self._assert_status_code_is(response, 201) resource = response.json() self._assert_has_keys(resource, "id", "name", "description", "create_time") assert resource["name"] == "Test Resource" assert resource["description"] == "Test description" def test_list_myresources(self): """Test GET /api/myresources.""" # Create test data self._create_myresource("Resource 1") self._create_myresource("Resource 2") # List response = self._get("myresources") self._assert_status_code_is_ok(response) data = response.json() assert "items" in data assert "total_count" in data assert data["total_count"] >= 2 assert len(data["items"]) >= 2 def test_get_myresource(self): """Test GET /api/myresources/{id}.""" resource_id = self._create_myresource("Test Resource") response = self._get(f"myresources/{resource_id}") self._assert_status_code_is_ok(response) resource = response.json() assert resource["id"] == resource_id assert resource["name"] == "Test Resource" def test_update_myresource(self): """Test PUT /api/myresources/{id}.""" resource_id = self._create_myresource("Original Name") payload = {"name": "Updated Name"} response = self._put(f"myresources/{resource_id}", data=payload, json=True) self._assert_status_code_is_ok(response) updated = response.json() assert updated["name"] == "Updated Name" def test_delete_myresource(self): """Test DELETE /api/myresources/{id}.""" resource_id = self._create_myresource("To Delete") response = self._delete(f"myresources/{resource_id}")