implement-connector

name	implement-connector
description	Implement all template methods in a connector's connector.py — the main agent workflow
argument-hint	<connector-name> [hybrid]
disable-model-invocation	true

Implement Connector: Full Agent Workflow

Implement all template methods and remaining BaseConnector methods in connectors/<name>/connector.py. The connection is already working — create_connection and create_cursor are implemented and passing -m connection.

Arguments

$ARGUMENTS contains:

• connector_name (required): e.g., snowflake, bigquery
• hybrid (optional): if present, run hybrid mode (skip metadata and query logs)

Ground Rules

These rules are non-negotiable — follow them throughout the entire workflow:

1. Check access requirements before implementing. When researching system views, catalog tables, or database features, check what permissions they require (admin roles, special privileges, enterprise/paid licenses, etc.). If elevated access is needed, stop and surface this to the user before proceeding. They may not be willing to grant that access to Monte Carlo — in which case find an alternative that works with standard permissions, or skip that capability. Do not assume the user can or will grant admin access.
2. Only edit connectors/<name>/connector.py. Do not modify any files in tests/, conftest.py, or the plugin. The tests are the spec.
3. Every template method must return a template string, not raw SQL and not Python logic. Templates come in two flavors:
   • Placeholder templates (most common): Return format-string placeholders like {x}. Example: return "AVG({x})". The backend substitutes {x} later via .format(). Single braces pass through Jinja untouched.
   • Parameterized templates: Use Jinja {{ variable }} syntax for values the backend passes at render time. Example: return "CAST({{ expression }} AS NUMERIC)".
   • Read the docstring to know which pattern each method uses.
4. Do not edit manifest.json in output/. It is auto-generated by --export.
5. Read each method's docstring before implementing it. The docstring documents which variable pattern the template uses (placeholder vs Jinja), example implementations for common databases, and which metrics the method enables.
6. Read failures carefully. A template rendering error means your Jinja string has the wrong variables. A SQL error means the rendered query is invalid for your database. An assertion error means the query ran but returned the wrong result. A KeyError from .format() means a placeholder name (like {x}) doesn't match what the caller passes.

Step 1: Read source files

Read all of these before writing any code:

1. connectors/<name>/connector.py — the file you'll be editing
2. connectors/_base/connector.py — the base template with all docstrings (this is your API reference)

Understand the database dialect you're implementing for. Use web search if you need to look up SQL syntax for this database.

Step 2: Implement remaining BaseConnector methods

Implement these three methods — they must work before any template test can run:

execute_query(self, query: str) -> None

Execute the query using the cursor. Handle database-specific error cases if needed (e.g., Teradata wraps "object doesn't exist" errors on DROP to allow idempotent drops).

fetch_all_results(self) -> List[Any]

Return all rows from the last executed query. Usually self.cursor.fetchall().

close_connection(self)

Clean up cursor and connection. Usually self.cursor.close(); self.connection.close().

Step 3: Build the Docker image

After editing connector.py, rebuild so the container picks up your changes:

docker compose build

Step 4: Implement templates section by section

Work through each section below in order. After implementing each section, run its test marker. Fix failures before moving on.

---

Section A: MetadataQueryTemplates [Full mode only — skip in hybrid]

Implement: get_databases_query_template, get_schemas_query_template, get_tables_query_template, get_columns_query_template

Access check: Metadata queries often rely on system catalog views (information_schema, pg_catalog, sys.*, etc.). Some databases restrict these to admin roles or require paid features. Before implementing, research what permissions the system views require and confirm with the user that MC's database user will have access. If not, look for alternative views that work with standard read-only permissions, or skip the capability.

Key requirements from the docstrings:

• get_tables_query_template must return columns: database_name, schema_name, table_name, table_type, and optionally row_count, byte_count, last_update_time, view_query
• get_tables_query_template receives offset and limit for pagination, plus optional table_names filter
• get_columns_query_template must return: full_table_id, column_name, column_type

Run tests:

CONNECTOR=<name> docker compose run --rm test -m metadata

---

Section B: QueryLogCollectionTemplates [Full mode only — skip in hybrid]

Implement: get_query_logs_query_template

Access check: Query log/history views frequently require elevated privileges (e.g., ACCOUNTADMIN in Snowflake, pg_read_all_stats in PostgreSQL, ACTIVITY_ADMIN in Teradata). Confirm with the user what access the MC database user will have before choosing which system view to query.

Key requirements from the docstring:

• Receives start_time, end_time (Python datetime objects), offset, limit
• Use .strftime() in Jinja to format timestamps for your database

Run tests:

CONNECTOR=<name> docker compose run --rm test -m query_logs

---

Section C: CustomSQLMonitorTemplates

Implement: transform_into_count_query_template, add_row_limit_template, get_count_all_expression_template

These are usually straightforward. Check if your database uses TOP N vs LIMIT N vs FETCH FIRST N ROWS ONLY.

Run tests:

CONNECTOR=<name> docker compose run --rm test -m custom_monitors

---

Section D: QueryLanguageTemplates — Prerequisites

These are the core building blocks that must pass before metric-specific templates are tested.

Implement all methods in these subsections of QueryLanguageTemplates:

• Core Query Building: build_cte_template, add_select_clause_template, add_from_clause_template, union_queries_template, alias_field_template, all_fields_expression_template, escape_field_name_template, get_table_identifier_template, get_arbitrary_where_clause_template, ascending_order_template, descending_order_template, get_case_when_func_template, negate_expression_template
• String and Literal Handling: escape_string_template, string_literal_template, literal_value_template, literal_datetime_template, literal_time_of_day_template, literal_regex_template, literal_table_from_value_list_template, date_literal_template, utc_literal_template
• Type Casting: all cast_* and get_casting_* methods
• Date/Time Functions: all time-related methods (current_date_func_template, add_days_func_template, time_truncate_func_template, truncate_to_*_template, etc.)
• Dialect Capability Flags: supports_literal_select_template, supports_literal_group_by_template, supports_group_by_on_subquery_template, parses_timestamp_with_trailing_text_template, supports_as_keyword_for_table_alias_template, supports_limit_0_template, requires_subquery_alias_template
• Null and NaN Handling: is_null_template, is_not_null_template, nan_expr_template, get_isnan_expression_template
• Comparison Operators: all get_is_*_expression_template methods
• Aggregation Functions: all aggregation methods (get_avg_function_template, get_stddev_function_template, etc.)
• String Functions: get_length_template, substring_func_template, get_is_empty_string_expression_template, get_regexp_expression_template, get_regexp_count_expression_template
• Array and Timestamp Validation: array_expr_template, get_array_length_func_template, get_is_timestamp_expression_template, get_not_is_timestamp_expression_template, get_epoch_seconds_expression_template, get_epoch_seconds_parameter_template
• Math Functions: get_absolute_value_function_template, rand_func_template
• RCA and Advanced Functions: max_time_func_template, unpivot_template
• Field Operations: get_field_or_alias_template

Important: Methods that return pass (None) are treated as "not supported" — the test will skip. Only implement methods your database actually supports. For example, most databases don't have native NaN, so nan_expr_template returns pass.

Run tests:

CONNECTOR=<name> docker compose run --rm test -m ql_prerequisites

---

Section E: QueryLanguageTemplates — Metrics

These tests verify that the prerequisite templates work together correctly to produce metric queries. No new methods to implement — if prerequisites all pass, metrics should too.

Run tests:

CONNECTOR=<name> docker compose run --rm test -m ql_metrics

If metric tests fail but prerequisites pass, the issue is usually in a prerequisite template producing subtly wrong SQL in combination with others. Read the failing query carefully.

---

Section F: FunctionalTestOperations [Optional]

Implement if the user wants functional validation. This verifies metadata queries reflect real-time database changes.

Implement: get_test_table_identifier, create_test_table_template, insert_rows_template, add_column_template, drop_column_template, drop_test_table_template, create_lineage_query_template

All templates automatically receive {{ database }}, {{ schema }}, {{ table }} from get_test_table_identifier().

Run tests:

CONNECTOR=<name> docker compose run --rm test -m functional

---

Step 5: Export capabilities

After all target tests pass, run the full suite with --export:

CONNECTOR=<name> docker compose run --rm test --export

This generates output/<name>/manifest.json and output/<name>/templates/.

Step 6: Report summary

Report:

• Which sections were implemented and their test results
• Total number of template methods implemented vs stubs remaining
• Any methods left as pass (unsupported by the database) and why
• Suggest /build-agent-image <name> as the next step

Useful test commands

# Run all tests
CONNECTOR=<name> docker compose run --rm test

# Run a single test file
CONNECTOR=<name> docker compose run --rm test tests/test_ql_prerequisites.py

# Run a single test
CONNECTOR=<name> docker compose run --rm test tests/test_ql_prerequisites.py::test_equality -v

Hybrid Mode Workflow

When hybrid is in the arguments, follow this reduced workflow:

1. Step 2: Implement execute_query, fetch_all_results, close_connection (same as full)
2. Step 3: docker compose build (same as full)
3. Section C: Implement CustomSQLMonitorTemplates → run -m custom_monitors
4. Section D: Implement QueryLanguageTemplates prerequisites → run -m ql_prerequisites
5. Section E: Run -m ql_metrics
6. Step 5: Export with --export
7. Step 6: Report summary

Skip Sections A (metadata), B (query logs), and F (functional).