feast-testing

# Testing and Debugging Feast ## Test Organization ``` sdk/python/tests/ ├── unit/ # Fast tests, no external deps, run locally │ ├── infra/ │ │ ├── online_store/ # Per-store unit tests (redis, dynamodb, etc.) │ │ ├── offline_stores/ # Offline store unit tests │ │ └── registry/ # Registry unit tests │ ├── test_unit_feature_store.py # Core FeatureStore behavior │ ├── test_feature_views.py # Feature view validation │ └── test_on_demand_*.py # On-demand transformation tests └── integration/ # Requires real infrastructure (run in CI) ├── feature_repos/ # Sample repos used as test fixtures └── test_universal_*.py # Cross-store compatibility tests ``` **Rule of thumb**: if a test needs mocks for an external service, it belongs in `unit/`. If it spins up real infra (Redis, DynamoDB, BigQuery), it belongs in `integration/`. --- ## Running Tests ### Targeted unit tests ```bash # Run a single test file python -m pytest sdk/python/tests/unit/infra/online_store/test_dynamodb_online_store.py -v # Run a single test by name python -m pytest sdk/python/tests/unit/test_unit_feature_store.py -k "test_apply" -v # Run all unit tests for one subsystem python -m pytest sdk/python/tests/unit/infra/online_store/ -v # Run all unit tests (fast, no infra needed) make test-python-unit ``` ### Integration tests (local) ```bash # Run local integration tests (SQLite online, file offline) make test-python-integration-local # Run specific integration test with verbose output python -m pytest sdk/python/tests/integration/ -k "test_online_retrieval" -v -s ``` ### Running a single test file fast (skip slow markers) ```bash python -m pytest sdk/python/tests/unit/infra/online_store/test_redis.py -v --no-header -q ``` --- ## Writing Unit Tests for a New Component ### Pattern: testing an online store Follow `sdk/python/tests/unit/infra/online_store/test_dynamodb_online_store.py` or `test_redis.py`. ```python import asyncio from unittest.mock import AsyncMock, MagicMock, patch from feast.infra.online_stores.mystore import MyStore, MyStoreConfig from feast.repo_config import RepoConfig def make_config(store_config: dict) -> RepoConfig: return RepoConfig( project="test_project", registry="data/registry.db", provider="local", online_store=store_config, ) @patch("feast.infra.online_stores.mystore.MyClient") def test_online_write_batch(mock_client_cls): mock_client = MagicMock() mock_client_cls.return_value = mock_client config = make_config({"type": "mystore", "host": "localhost"}) store = MyStore() feature_view = MagicMock() feature_view.name = "driver_stats" store.online_write_batch(config, feature_view, data=[...], progress=None) mock_client.set.assert_called_once() # For async methods: def test_online_write_batch_async(): async def _run(): store = MyStore() await store.online_write_batch_async(config, feature_view, data, progress=None) asyncio.run(_run()) ``` ### Pattern: testing registry operations Follow `sdk/python/tests/unit/infra/registry/test_registry.py`. ```python from feast.infra.registry.registry import Registry from feast.repo_config import RegistryConfig def test_apply_feature_view(tmp_path): config = RegistryConfig(path=str(tmp_path / "registry.db")) registry = Registry("test_project", config, repo_path=None) entity = Entity(name="driver_id", value_type=ValueType.INT64) registry.apply_entity(entity, project="test_project") stored = registry.get_entity("driver_id", project="test_project") assert stored.name == "driver_id" ``` ### Pattern: SQL registry — dialect DDL and startup-check tests See `sdk/python/tests/unit/infra/registry/test_sql_registry.py`. - **Assert dialect-specific DDL without a live DB**: compile a table for a target dialect and check the emitted column type. Useful for column-type rules like `ProtoBytes` → `LONGBLOB` on MySQL/MariaDB. ```python from sqlalchemy.dialects import mysql from sqlalchemy.schema import CreateTable ddl = str(CreateTable(feature_views).compile(dialect=mysql.dialect())) assert "feature_view_proto LONGBLOB" in ddl ``` - **Unit-test a startup diagnostic with a mock engine**: stub `engine.dialect.name` and the `engine.connect()` context manager so no DB is needed. ```python engine = MagicMock() engine.dialect.name = "mysql" engine.connect.return_value.__enter__.return_value.execute.return_value.fetchall.return_value = [ ("feature_views", "feature_view_proto"), ] SqlRegistry._warn_if_narrow_blob_columns(engine) ``` - **Live round-trip behavior** (e.g. a >64 KB proto surviving the column) belongs in an integration test against the `mysql_registry` fixture in `tests/integration/registration/test_universal_registry.py` (DDL-compile tests can't catch runtime truncation). ### Pattern: testing FeatureStore end-to-end (unit level) ```python from feast import FeatureStore from feast.repo_config import RepoConfig def test_get_online_features(tmp_path): config = RepoConfig( project="test", registry=str(tmp_path / "registry.db"), provider="local", online_store={"type": "sqlite", "path": str(tmp_path / "online.db")}, offline_store={"type": "file"}, ) store = FeatureStore(config=config) # apply objects, write data, then assert retrieval ``` --- ## Debugging Common Issues ### Registry out of sync / stale metadata ```bash # Inspect the live registry feast registry-dump # prints full proto as JSON # Or from Python: from feast import FeatureStore store = FeatureStore(repo_path=".") store.registry.list_feature_views(project=store.project) store.registry.list_entities(project=store.project) ``` **Symptom**: `FeatureView not found` or stale feature definitions. **Fix**: Re-run `feast apply`. If using a cached registry (TTL > 0), wait for cache expiry or force refresh by restarting the process. ### Online store returns None for all features ```bash # Check if materialization has run feast feature-views list # shows last_updated_timestamp # Check entity key format # Entity keys are serialized — if the serialization_version doesn't match, lookups fail. # Check entity_key_serialization_version in feature_store.yaml ``` **Symptom**: `get_online_features` returns all `None` values despite materialization running. **Cause**: Entity key serialization version mismatch, wrong project name, or wrong TTL causing data to expire. ### Type errors from mypy ```bash # Type-check only the file you changed uv run bash -c "cd sdk/python && mypy feast/infra/online_stores/mystore.py" # Common fix: add Optional[] around return types, use cast() for dynamic lookups ``` ### Test imports fail / module not found ```bash # Ensure you're running from repo root with dev dependencies installed make install-python-dependencies-dev # Run with uv to use the pinned venv uv run python -m pytest sdk/python/tests/unit/... -v ``` ### Protobuf deserialization errors **Symptom**: `DecodeError` or unexpected field values when reading registry. **Cause**: Proto schema changed without recompiling. **Fix**: ```bash make compile-protos-python ``` ### Feature server returns 500 ```bash # Start the feature server locally and check logs feast serve --host 0.0.0.0 --port 6566 # Test directly curl -X POST http://localhost:6566/get-online-features \ -H "Content-Type: application/json" \ -d '{"features": ["driver_stats:conv_rate"], "entities": {"driver_id": [1001]}}' ``` --- ## Useful CLI Commands for Inspection ```bash feast entities list # list all registered entities feast feature-views list # list feature views + last materialization time feast feature-services list # list feature services feast on-demand-feature-views list # list ODFVs feast registry-dump # dump full registry as JSON (debug) feast plan # preview what feast apply would change (dry run) ``` --- ## Test Utilities | Utility | Location | Purpose | |---|---|---| | `get_feature_view()` helpers | `tests/unit/infra/online_store/test_*.py` | Build minimal FeatureView mocks | | `LocalRegistry` fixture | `tests/unit/infra/registry/test_registry.py` | In-memory registry for unit tests | | `integration/feature_repos/` | `tests/integration/` | Sample repos for integration test scenarios | | `conftest.py` fixtures | Various `tests/` subdirs | Shared pytest fixtures per subsystem |