메뉴
Use when loading market data, working with the Parquet catalog, fetching from IBKR, converting CSV data, or diagnosing data availability issues. Covers the DataCatalogService API, catalog structure, and IBKR client configuration.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | data-pipeline |
| description | Use when loading market data, working with the Parquet catalog, fetching from IBKR, converting CSV data, or diagnosing data availability issues. Covers the DataCatalogService API, catalog structure, and IBKR client configuration. |
See also:
docs/agent/nautilus.md(IBKR client, Parquet catalog)
CSV files / IBKR API → nautilus_converter → ParquetDataCatalog → BacktestEngine.add_data(bars)
{SYMBOL}.{VENUE}-{STEP}-{STEP_TYPE}-{PRICE_TYPE}-EXTERNAL
Example: QQQ.NASDAQ-1-DAY-LAST-EXTERNAL
| Component | Examples | Notes |
|---|---|---|
| SYMBOL.VENUE | AAPL.NASDAQ, QQQ.NASDAQ | Instrument identifier |
| STEP | 1, 5, 15 | Bar duration number |
| STEP_TYPE | MINUTE, DAY, HOUR | Bar duration unit |
| PRICE_TYPE | LAST, MID, BID, ASK | Price aggregation type |
| Source | EXTERNAL (real data), INTERNAL (mock) | Data origin |
from src.services.data_catalog import DataCatalogService
service = DataCatalogService() # Uses NAUTILUS_PATH env or ./data/catalog
# Load from catalog, fetch from IBKR if missing (primary method)
bars = await service.fetch_or_load(
instrument_id="AAPL.NASDAQ",
start=datetime(2024, 1, 1, tzinfo=timezone.utc),
end=datetime(2024, 12, 31, tzinfo=timezone.utc),
bar_type_spec="1-MINUTE-LAST",
)
# Query bars directly from catalog (no IBKR fallback)
bars = service.query_bars(instrument_id="AAPL.NASDAQ", ...)
# Other methods
service.write_bars(bars, correlation_id="backtest-123")
avail = service.get_availability("AAPL.NASDAQ", "1-MINUTE-LAST")
gaps = service.detect_gaps("AAPL.NASDAQ", "1-MINUTE-LAST", start_date=..., end_date=...)
catalog_data = service.scan_catalog()
instrument = service.load_instrument("AAPL.NASDAQ")
catalog.bars() Parameter# WRONG — Using BarType objects returns ALL bar types
bars = catalog.bars(bar_types=[BarType.from_str("...")])
# CORRECT — Must be list of STRINGS
bars = catalog.bars(bar_types=["AAPL.NASDAQ-1-MINUTE-LAST-EXTERNAL"])
All connection settings via environment variables through IBKRSettings.
See docs/agent/nautilus.md for full IBKR configuration details.
The client is created lazily on first access — no connection until data is actually needed from IBKR.
src/services/data_catalog.py — Catalog facade (primary entry point)src/services/ibkr_client.py — IBKR connection and historical datasrc/services/csv_loader.py — CSV file loadingsrc/services/nautilus_converter.py — Data format conversionsrc/models/catalog_metadata.py — CatalogAvailability modelUse when the user wants to design, redesign, shape, critique, audit, polish, clarify, distill, harden, optimize, adapt, animate, colorize, extract, or otherwise improve a frontend interface. Covers websites, landing pages, dashboards, product UI, app shells, components, forms, settings, onboarding, and empty states. Handles UX review, visual hierarchy, information architecture, cognitive load, accessibility, performance, responsive behavior, theming, anti-patterns, typography, fonts, spacing, layout, alignment, color, motion, micro-interactions, UX copy, error states, edge cases, i18n, and reusable design systems or tokens. Also use for bland designs that need to become bolder or more delightful, loud designs that should become quieter, live browser iteration on UI elements, or ambitious visual effects that should feel technically extraordinary. Not for backend-only or non-UI tasks.
Use when running backtests, debugging engine crashes, working with BacktestEngine lifecycle, or encountering segfaults and LogGuard panics. Covers engine setup sequence, result extraction, and C extension isolation.
Use when diagnosing segfaults, test failures, data issues, or engine crashes. Covers LogGuard panics, C extension state leaks, common error messages with fixes, and debugging commands.
Use when writing tests, choosing test tiers, using test fixtures, or following TDD workflow. Covers the test pyramid (unit/component/integration/e2e), available test doubles, fixture catalog, and market scenarios.
Use when creating a new trading strategy, modifying strategy logic, adding indicators, or working on position sizing. Covers the 4-file registration pattern, on_bar() flow, and Nautilus indicator gotchas.
Use when adding web UI pages, API endpoints, chart routes, or working with HTMX/Jinja2 templates. Covers the FastAPI app structure, router registration, HTMX patterns, and Tailwind CSS build process.