Adjust a working dlt pipeline for production — remove dev limits, verify pagination, configure incremental loading, expand date ranges. Use when the user wants to remove .add_limit(), load more data, fix pagination, or set up incremental loading.
Installation
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Adjust a working dlt pipeline for production — remove dev limits, verify pagination, configure incremental loading, expand date ranges. Use when the user wants to remove .add_limit(), load more data, fix pagination, or set up incremental loading.
argument-hint
[pipeline-name] [adjustments]
Adjust endpoint for production
Parse $ARGUMENTS:
pipeline-name (optional): the dlt pipeline name. If omitted, infer from session context. If ambiguous, ask the user and stop.
hints (optional, after --): specific adjustments to make
.add_limit(1) during development masks pagination problems — only one page is fetched, so a broken paginator never loops. Removing it without explicit pagination causes stuck pipelines.
Before removing .add_limit():
Check every resource has an explicit "paginator" config. If any rely on auto-detection, add one first.
Use debug-pipeline with INFO logging for the first unlimited run to watch pagination progress and catch loops early.
Real example: OpenAI Usage API
Pipeline worked with .add_limit(1). After removing the limit, it hung forever — dlt's auto-detected paginator looped. Fix: added explicit "paginator": {"type": "cursor", "cursor_path": "next_page", "cursor_param": "page"}. Full load then completed in 5 seconds.
Next steps
Full load complete → invoke explore-data to chart and analyze the data
