mock-data

name	mock-data
description	Generate deterministic synthetic CSV files under data/sandbox/ using the data_mock_generate MCP tool. Use when no real source file exists for a new or demo topic and you need staged data to build the download/bronze/dq notebook pipeline against.

mock-data

When to use

• A new topic has no real source file yet and you need something to ingest
• A demo needs repeatable, PII-free staged data
• You are writing or testing a Bronze notebook and need a staged CSV with a specific shape

Do not use real source extracts. Always generate synthetic data and run it through the normal masking and DQ pipeline.

Tool — data_mock_generate (MCP)

Call the data_mock_generate MCP tool on the fabric-server. It runs inside the container, so pass target_dir = the project root mounted into the container; output defaults to data/sandbox/<topic>.csv under it. Always pass a schema derived from the target table.

Argument	Default	Notes
target_dir	—	Project root (required); the container writes under it
topic	orders	Output file stem
rows	1000	Row count
seed	42	Deterministic seed
engine	stdlib	One of stdlib, faker, mimesis, sklearn
schema	—	Inline JSON array of {name,type} column defs
schema_file	—	Project-relative path to a JSON schema (mutually exclusive with schema)
output	—	Explicit project-relative CSV path

Examples (arguments shown as JSON for clarity):

// Custom schema matching your target table
{"target_dir": ".", "topic": "orders", "rows": 1000,
 "schema": "[{\"name\":\"id\",\"type\":\"id\"},{\"name\":\"customer_id\",\"type\":\"int\",\"min\":1,\"max\":500},{\"name\":\"email\",\"type\":\"email\"},{\"name\":\"amount\",\"type\":\"decimal\",\"min\":2.5,\"max\":2500,\"decimals\":2},{\"name\":\"order_date\",\"type\":\"date\",\"start\":\"2025-01-01\",\"end\":\"2025-12-31\"}]"}

// Schema from a reusable JSON file
{"target_dir": ".", "topic": "orders", "rows": 5000, "schema_file": "schemas/orders.json"}

// Faker engine for richer PII-shaped values
{"target_dir": ".", "engine": "faker", "topic": "customers", "rows": 1000,
 "schema": "[{\"name\":\"id\",\"type\":\"id\"},{\"name\":\"full_name\",\"type\":\"name\"},{\"name\":\"email\",\"type\":\"email\"},{\"name\":\"phone\",\"type\":\"phone\"},{\"name\":\"address\",\"type\":\"address\"}]"}

// ML fixtures — feature columns + a target column
{"target_dir": ".", "engine": "sklearn", "rows": 5000,
 "schema": "[{\"name\":\"id\",\"type\":\"id\"},{\"name\":\"price\",\"type\":\"float\",\"decimals\":4},{\"name\":\"quantity\",\"type\":\"float\",\"decimals\":4},{\"name\":\"target\",\"type\":\"int\"}]"}

Output lands at data/sandbox/<topic>.csv.

Schema column types

Type	Options	Notes
id	—	Sequential integer starting at 1
int	min, max	Random integer
float / decimal	min, max, decimals	Random float
string / word / str	—	Random word
sentence / text	—	Random sentence
name	—	Full name
first_name / last_name	—	Name parts
email	—	Email address
address	—	Street address
date	start, end	ISO date (YYYY-MM-DD)
datetime / timestamp	start, end	ISO datetime
boolean	—	True or False
uuid	—	UUID v4
phone	—	Phone number
company	—	Company name
url	—	URL

Engine selection guide

Engine	Install	When to use
stdlib (default)	none	No-dependency fallback; string types produce placeholders (name_1, [e-mail])
faker	pip install Faker	Realistic PII-shaped values for UI tests or simple databases
mimesis	pip install mimesis	Very high-volume generation (millions of rows) at speed
sklearn	pip install scikit-learn	Controlled ML classification fixtures with known class balance

After generating

The staged CSV still needs the full three-notebook pipeline:

1. download_<source>.py — skip if file already staged; otherwise simulate a fetch
2. bronze_<source>.py — ingest only new files; mask any PII-shaped columns
3. dq_bronze_<source>.py — Great Expectations checks on the Bronze table

Synthetic PII-shaped columns (email, name, address, phone) must go through the same masking barrier as real data.