| name | pengdows-crud |
| description | Help with pengdows.crud - a SQL-first, strongly-typed data access layer for .NET. Use when implementing CRUD operations, entity mapping, database connections, transactions, or testing with FakeDb. Covers TableGateway, SqlContainer, DatabaseContext, attributes ([Table], [Column], [Id], [PrimaryKey]), and multi-database support. |
| allowed-tools | Read, Grep, Glob, Bash |
pengdows.crud Development Guide
pengdows.crud is a SQL-first, strongly-typed, testable data access layer for .NET 8+. No LINQ, no tracking, no surprises - explicit SQL control with database-agnostic features.
Quick Start
[Table("orders")]
public class Order
{
[Id(false)]
[Column("id", DbType.Int64)]
public long Id { get; set; }
[PrimaryKey(1)]
[Column("order_number", DbType.String, 50)]
public string OrderNumber { get; set; }
[Column("customer_id", DbType.Int64)]
public long CustomerId { get; set; }
[Column("total", DbType.Decimal)]
public decimal Total { get; set; }
}
public interface IOrderGateway : ITableGateway<Order, long>
{
Task<Order?> GetByOrderNumberAsync(string orderNumber);
Task<List<Order>> GetCustomerOrdersAsync(long customerId, DateTime? since = null);
}
public class OrderGateway : TableGateway<Order, long>, IOrderGateway
{
public OrderGateway(IDatabaseContext context) : base(context)
{
}
public async Task<Order?> GetByOrderNumberAsync(string orderNumber)
{
var lookup = new Order { OrderNumber = orderNumber };
return await RetrieveOneAsync(lookup);
}
public async Task<List<Order>> GetCustomerOrdersAsync(long customerId, DateTime? since = null)
{
var sc = BuildBaseRetrieve("o");
sc.Query.Append(" WHERE ");
sc.Query.Append(sc.WrapObjectName("o.customer_id"));
sc.Query.Append(" = ");
var param = sc.AddParameterWithValue("customerId", DbType.Int64, customerId);
sc.Query.Append(sc.MakeParameterName(param));
if (since.HasValue)
{
sc.Query.Append(" AND ");
sc.Query.Append(sc.WrapObjectName("o.created_at"));
sc.Query.Append(" >= ");
var sinceParam = sc.AddParameterWithValue("since", DbType.DateTime, since.Value);
sc.Query.Append(sc.MakeParameterName(sinceParam));
}
sc.Query.Append(" ORDER BY ");
sc.Query.Append(sc.WrapObjectName("o.created_at"));
sc.Query.Append(" DESC");
return await LoadListAsync(sc);
}
}
services.AddSingleton<IDatabaseContext>(sp =>
new DatabaseContext(connectionString, SqlClientFactory.Instance));
services.AddSingleton<IOrderGateway>(sp =>
new OrderGateway(sp.GetRequiredService<IDatabaseContext>()));
public class OrdersController : ControllerBase
{
private readonly IOrderGateway _orderGateway;
public OrdersController(IOrderGateway orderGateway)
{
_orderGateway = orderGateway;
}
[HttpGet("{orderNumber}")]
public async Task<IActionResult> Get(string orderNumber)
{
var order = await _orderGateway.GetByOrderNumberAsync(orderNumber);
return order is null ? NotFound() : Ok(order);
}
}
SQL Building: The Three-Tier API
TableGateway has three tiers of methods. Understanding which tier to use is key:
Tier 1: Build Methods (SQL generation only, no execution)
Build* methods return an ISqlContainer holding generated SQL and parameters. Nothing is sent to the database. You inspect, modify, or execute the container yourself.
ISqlContainer BuildCreate(entity);
ISqlContainer BuildBaseRetrieve("alias");
ISqlContainer BuildRetrieve(ids, "alias");
ISqlContainer BuildRetrieve(ids);
ISqlContainer BuildRetrieve(entities, "a");
ISqlContainer BuildRetrieve(entities);
ISqlContainer BuildDelete(id);
ISqlContainer BuildUpsert(entity);
IReadOnlyList<ISqlContainer> BuildBatchCreate(entities);
IReadOnlyList<ISqlContainer> BuildBatchUpdate(entities);
IReadOnlyList<ISqlContainer> BuildBatchUpsert(entities);
IReadOnlyList<ISqlContainer> BuildBatchDelete(ids);
IReadOnlyList<ISqlContainer> BuildBatchDelete(entities);
ISqlContainer sc = await BuildUpdateAsync(entity);
ISqlContainer sc = await BuildUpdateAsync(entity, loadOriginal: true);
BuildBaseRetrieve vs BuildRetrieve: BuildBaseRetrieve generates SELECT columns FROM table with NO WHERE clause - it's the starting point when you need to add your own custom filtering. BuildRetrieve generates a complete query with a WHERE clause filtering by IDs or primary key values.
Tier 2: Load Methods (execute a pre-built container, map results)
Load* methods take an ISqlContainer you already have (from a Build method or custom SQL) and execute it, mapping rows to entities:
TEntity? result = await LoadSingleAsync(container);
List<TEntity> list = await LoadListAsync(container);
IAsyncEnumerable<TEntity> stream = LoadStreamAsync(container);
Tier 3: Convenience Methods (Build + Execute in one call)
These combine Tier 1 + Tier 2 into a single call. Use when you don't need to customize the SQL:
TEntity? order = await RetrieveOneAsync(id);
TEntity? order = await RetrieveOneAsync(entityLookup);
List<TEntity> orders = await RetrieveAsync(ids);
IAsyncEnumerable<TEntity> orders = RetrieveStreamAsync(ids);
bool created = await CreateAsync(entity, context);
int affected = await UpdateAsync(entity);
int affected = await DeleteAsync(id);
int affected = await DeleteAsync(ids);
int affected = await UpsertAsync(entity);
int affected = await CreateAsync(entities);
int affected = await UpdateAsync(entities);
int affected = await UpsertAsync(entities);
int affected = await DeleteAsync(entities);
int affected = await BatchCreateAsync(entities);
int affected = await BatchUpdateAsync(entities);
int affected = await BatchUpsertAsync(entities);
int affected = await BatchDeleteAsync(ids);
int affected = await BatchDeleteAsync(entities);
WHERE Clause Helpers (modify an existing container)
These append WHERE clauses to a container you already have:
BuildWhere("e.Id", ids, existingContainer);
BuildWhereByPrimaryKey(entities, existingContainer, "e");
Typical Workflow
var order = await RetrieveOneAsync(orderId);
var sc = BuildBaseRetrieve("o");
sc.Query.Append(" WHERE ");
sc.Query.Append(sc.WrapObjectName("o.status"));
sc.Query.Append(" = ");
var p = sc.AddParameterWithValue("status", DbType.String, "Active");
sc.Query.Append(sc.MakeParameterName(p));
var results = await LoadListAsync(sc);
var sc = BuildBaseRetrieve("o");
sc.Query.Append(" ORDER BY ");
sc.Query.Append(sc.WrapObjectName("o.created_at"));
await foreach (var order in LoadStreamAsync(sc))
{
await ProcessAsync(order);
}
ISqlContainer: Execution Methods
All execution methods return ValueTask (not Task) for reduced allocations:
ValueTask<int> ExecuteNonQueryAsync(commandType);
ValueTask<T> ExecuteScalarRequiredAsync<T>(commandType);
ValueTask<T?> ExecuteScalarOrNullAsync<T>(commandType);
ValueTask<ScalarResult<T>> TryExecuteScalarAsync<T>(commandType);
ValueTask<ITrackedReader> ExecuteReaderAsync(commandType);
All have CancellationToken overloads.
ISqlContainer: Clone for Reuse
Clone a container to reuse its SQL structure with different parameter values or contexts:
var template = BuildCreate(entity);
var clone = template.Clone();
var clone = template.Clone(transactionContext);
DI Lifetime Rules - CRITICAL
| Component | Lifetime | Why |
|---|
DatabaseContext | Singleton | Manages connection pool, metrics, DbMode state |
TableGateway<T,TId> | Singleton | Stateless, caches compiled accessors |
IAuditValueResolver | Singleton | Must be thread-safe/AsyncLocal-based (e.g. IHttpContextAccessor) |
services.AddSingleton<IDatabaseContext>(sp =>
new DatabaseContext(connectionString, SqlClientFactory.Instance));
services.AddSingleton<IOrderGateway>(sp =>
new OrderGateway(sp.GetRequiredService<IDatabaseContext>(), sp.GetRequiredService<IAuditValueResolver>()));
services.AddSingleton<IAuditValueResolver, OidcAuditContextProvider>();
Extending TableGateway - THE CORRECT PATTERN
Inherit from TableGateway to add custom query methods. Don't wrap it in a separate service class.
public interface ICustomerGateway : ITableGateway<Customer, long>
{
Task<Customer?> GetByEmailAsync(string email);
Task<List<Customer>> GetActiveCustomersAsync();
Task<List<Customer>> SearchByNameAsync(string namePattern);
}
public class CustomerGateway : TableGateway<Customer, long>, ICustomerGateway
{
public CustomerGateway(IDatabaseContext context, IAuditValueResolver resolver) : base(context, resolver)
{
}
public async Task<Customer?> GetByEmailAsync(string email)
{
var lookup = new Customer { Email = email };
return await RetrieveOneAsync(lookup);
}
public async Task<List<Customer>> GetActiveCustomersAsync()
{
var sc = BuildBaseRetrieve("c");
sc.Query.Append(" WHERE ");
sc.Query.Append(sc.WrapObjectName("c.is_active"));
sc.Query.Append(" = ");
var param = sc.AddParameterWithValue("active", DbType.Boolean, true);
sc.Query.Append(sc.MakeParameterName(param));
sc.Query.Append(" ORDER BY ");
sc.Query.Append(sc.WrapObjectName("c.name"));
return await LoadListAsync(sc);
}
public async Task<List<Customer>> SearchByNameAsync(string namePattern)
{
var sc = BuildBaseRetrieve("c");
sc.Query.Append(" WHERE ");
sc.Query.Append(sc.WrapObjectName("c.name"));
sc.Query.Append(" LIKE ");
var param = sc.AddParameterWithValue("pattern", DbType.String, $"%{namePattern}%");
sc.Query.Append(sc.MakeParameterName(param));
return await LoadListAsync(sc);
}
}
Audit Handling
IAuditValueResolver
Register as singleton and pass to TableGateway for entities with audit fields:
services.AddHttpContextAccessor();
services.AddSingleton<IAuditValueResolver, OidcAuditContextProvider>();
public class OidcAuditContextProvider : IAuditValueResolver
{
private readonly IHttpContextAccessor _accessor;
public OidcAuditContextProvider(IHttpContextAccessor accessor)
{
_accessor = accessor;
}
public IAuditValues Resolve()
{
var user = _accessor.HttpContext?.User;
var id = user?.FindFirst(ClaimTypes.NameIdentifier)?.Value ?? "anonymous";
return new AuditValues { UserId = id };
}
}
Audit Fields on Entity
[Table("orders")]
public class Order
{
[Id] [Column("id")] public long Id { get; set; }
[CreatedBy] [Column("created_by")] public string CreatedBy { get; set; }
[CreatedOn] [Column("created_at")] public DateTime CreatedAt { get; set; }
[LastUpdatedBy] [Column("updated_by")] public string UpdatedBy { get; set; }
[LastUpdatedOn] [Column("updated_at")] public DateTime UpdatedAt { get; set; }
}
Important: Both CreatedBy/On AND LastUpdatedBy/On are SET on CREATE.
Multi-Tenancy
Pattern 1: TenantContextRegistry
Use TenantContextRegistry as singleton to manage per-tenant DatabaseContext instances:
services.AddSingleton<TenantContextRegistry>();
public class TenantService
{
private readonly TenantContextRegistry _registry;
public TenantService(TenantContextRegistry registry)
{
_registry = registry;
}
public IDatabaseContext GetContextForTenant(string tenantId)
{
return _registry.GetContext(tenantId);
}
}
Pattern 2: Per-Tenant Database Contexts (Different Database Types)
How the optional context parameter works:
- Without context parameter: Uses the default context from constructor (simple single-database apps)
- With context parameter: Uses the passed context instead (multi-tenant scenarios)
CRITICAL: Each tenant can use a different database type (SQL Server, PostgreSQL, SQLite, etc.). Pass the tenant's context to CRUD methods to route operations to the tenant's database:
services.AddSingleton<ITableGateway<Order, long>>(sp =>
new OrderGateway(defaultContext));
var order = await gateway.RetrieveOneAsync(orderId);
var registry = services.GetRequiredService<ITenantContextRegistry>();
var tenantCtx = registry.GetContext("enterprise-client");
var gateway = services.GetRequiredService<ITableGateway<Order, long>>();
var order = await gateway.RetrieveOneAsync(orderId, tenantCtx);
await gateway.CreateAsync(newOrder, tenantCtx);
await gateway.UpdateAsync(order, tenantCtx);
await gateway.DeleteAsync(orderId, tenantCtx);
All CRUD methods accept optional context parameter:
CreateAsync(entity, tenantContext) - Insert to tenant's database
RetrieveOneAsync(id, tenantContext) - Select from tenant's database
RetrieveAsync(ids, tenantContext) - Select multiple from tenant's database
UpdateAsync(entity, tenantContext) - Update in tenant's database
DeleteAsync(id, tenantContext) - Delete from tenant's database
UpsertAsync(entity, tenantContext) - Upsert to tenant's database
This pattern enables:
- Single TableGateway instance (singleton) for all tenants
- Each tenant uses different database type (PostgreSQL, SQL Server, MySQL, etc.)
- SQL automatically generated with tenant's dialect (parameter markers, quoting)
- Connection pooling per tenant context
- No tenant_id filtering needed - physical database separation
Example: Multi-Tenant Controller
public class OrdersController : ControllerBase
{
private readonly ITableGateway<Order, long> _orderGateway;
private readonly ITenantContextRegistry _tenantRegistry;
public OrdersController(
ITableGateway<Order, long> orderGateway,
ITenantContextRegistry tenantRegistry)
{
_orderGateway = orderGateway;
_tenantRegistry = tenantRegistry;
}
[HttpGet("{id}")]
public async Task<IActionResult> Get(long id)
{
var tenantId = User.FindFirst("tenant_id")?.Value;
var tenantCtx = _tenantRegistry.GetContext(tenantId);
var order = await _orderGateway.RetrieveOneAsync(id, tenantCtx);
return order is null ? NotFound() : Ok(order);
}
[HttpPost]
public async Task<IActionResult> Create([FromBody] Order order)
{
var tenantId = User.FindFirst("tenant_id")?.Value;
var tenantCtx = _tenantRegistry.GetContext(tenantId);
await _orderGateway.CreateAsync(order, tenantCtx);
return CreatedAtAction(nameof(Get), new { id = order.Id }, order);
}
}
Core Concepts
[Id] vs [PrimaryKey] - CRITICAL DISTINCTION
[Id] - Surrogate/Pseudo Key:
- Single column only
- Used by TableGateway for CRUD operations
[Id(false)] = DB-generated (identity/autoincrement)
[Id(true)] or [Id] = client-provided value
[PrimaryKey] - Business/Natural Key:
- Can be composite (multiple columns with order)
- Enforced as UNIQUE constraint
- Used by
RetrieveOneAsync(entity) for lookup
- NEVER put on same column as [Id]
[Table("order_items")]
public class OrderItem
{
[Id(false)]
[Column("id")]
public long Id { get; set; }
[PrimaryKey(1)]
[Column("order_id")]
public int OrderId { get; set; }
[PrimaryKey(2)]
[Column("product_id")]
public int ProductId { get; set; }
}
Custom SQL with SqlContainer
IMPORTANT: Always use WrapObjectName() for column names and aliases to ensure proper quoting per database dialect.
public async Task<List<Order>> GetRecentLargeOrdersAsync(decimal minTotal)
{
var sc = BuildBaseRetrieve("o");
sc.Query.Append(" WHERE ");
sc.Query.Append(sc.WrapObjectName("o.total"));
sc.Query.Append(" >= ");
var totalParam = sc.AddParameterWithValue("minTotal", DbType.Decimal, minTotal);
sc.Query.Append(sc.MakeParameterName(totalParam));
sc.Query.Append(" AND ");
sc.Query.Append(sc.WrapObjectName("o.created_at"));
sc.Query.Append(" >= ");
var dateParam = sc.AddParameterWithValue("since", DbType.DateTime, DateTime.UtcNow.AddDays(-30));
sc.Query.Append(sc.MakeParameterName(dateParam));
sc.Query.Append(" ORDER BY ");
sc.Query.Append(sc.WrapObjectName("o.total"));
sc.Query.Append(" DESC");
return await LoadListAsync(sc);
}
WrapObjectName behavior by database:
- SQL Server:
[o].[total]
- PostgreSQL:
"o"."total"
- MySQL:
`o`.`total`
- Oracle:
"o"."total"
Connection Management (DbMode)
Use lowest number possible:
| Mode | Use Case |
|---|
Standard (0) | Production default - pool per operation |
KeepAlive (1) | Embedded DBs needing sentinel connection |
SingleWriter (2) | File-based SQLite/DuckDB |
SingleConnection (4) | In-memory :memory: databases |
Best (15) | Auto-select optimal mode for the database |
services.AddSingleton<IDatabaseContext>(sp =>
new DatabaseContext(connStr, factory, null, DbMode.SingleWriter));
Transactions
Transactions are operation-scoped - create inside methods, never store as fields.
public async Task<bool> CancelOrderAsync(long orderId)
{
await using var txn = await Context.BeginTransactionAsync();
try
{
var order = await RetrieveOneAsync(orderId);
if (order == null || order.Status == OrderStatus.Shipped)
{
await txn.RollbackAsync();
return false;
}
order.Status = OrderStatus.Cancelled;
var affected = await UpdateAsync(order);
if (affected > 0)
{
await txn.CommitAsync();
return true;
}
await txn.RollbackAsync();
return false;
}
catch
{
await txn.RollbackAsync();
throw;
}
}
using var txn = Context.BeginTransaction(IsolationLevel.Serializable);
await using var txn = await Context.BeginTransactionAsync(IsolationProfile.SafeNonBlockingReads);
Resource Safety:
Always use await using for ITransactionContext and ITrackedReader to ensure connections are released correctly.
CRITICAL: Do NOT use TransactionScope
TransactionScope is incompatible with pengdows.crud's connection management. The "open late, close early" philosophy means each operation opens/closes its own connection, which causes:
- Distributed transaction promotion - Second connection within
TransactionScope promotes to MSDTC
- Performance overhead - MSDTC has significant overhead and may not work in cloud environments
- Broken semantics - Connections closing between operations lose transactional guarantees
Always use Context.BeginTransaction() which pins the connection for the transaction's lifetime.
Version Column (Optimistic Concurrency)
[Version]
[Column("version")]
public int Version { get; set; }
- Create: Auto-set to 1
- Update: Increments and adds
WHERE version = @current
- Conflict: Returns 0 rows affected
Parameter Naming Convention
| Operation | Pattern | Example |
|---|
| INSERT | i{n} | i0, i1, i2 |
| UPDATE SET | s{n} | s0, s1, s2 |
| WHERE | w{n} | w0, w1, w2 |
| VERSION | v{n} | v0, v1 |
container.SetParameterValue("w0", newId);
container.SetParameterValue("s0", newValue);
Testing with FakeDb
public class OrderGatewayTests
{
private readonly fakeDbFactory _factory;
private readonly IDatabaseContext _context;
private readonly OrderGateway _gateway;
public OrderGatewayTests()
{
_factory = new fakeDbFactory(SupportedDatabase.Sqlite);
_context = new DatabaseContext("Data Source=test", _factory);
_gateway = new OrderGateway(_context);
}
[Fact]
public async Task GetCustomerOrdersAsync_BuildsCorrectSql()
{
var orders = await _gateway.GetCustomerOrdersAsync(123);
Assert.NotNull(orders);
}
[Fact]
public void ConnectionFailure_ThrowsException()
{
var factory = fakeDbFactory.CreateFailingFactory(
SupportedDatabase.PostgreSql,
ConnectionFailureMode.FailOnOpen);
var context = new DatabaseContext("test", factory);
Assert.Throws<InvalidOperationException>(() =>
{
using var container = context.CreateSqlContainer("SELECT 1");
container.ExecuteScalarRequiredAsync<int>().GetAwaiter().GetResult();
});
}
}
Supported Databases
SQL Server, PostgreSQL, Oracle, MySQL, MariaDB, SQLite, Firebird, CockroachDB, DuckDB
Each uses optimal SQL syntax (MERGE vs ON CONFLICT vs ON DUPLICATE KEY UPDATE).
TDD Requirements
ALL code changes MUST follow TDD:
- Write test FIRST
- Run test - verify it fails
- Write minimal implementation
- Refactor while green
- Repeat
- Minimum 83% coverage (CI enforced)
- Target 95%+ for new features
- NO skipped tests
Core Invariants
- DatabaseContext is SINGLETON - one per connection string
- TableGateway is SINGLETON - stateless, caches compiled accessors
- Extend TableGateway - put custom query methods in inherited class, not wrapper service
- IAuditValueResolver is SINGLETON - must be thread-safe/AsyncLocal-based to avoid captive dependencies in singleton gateways
- TenantContextRegistry is SINGLETON - manages per-tenant contexts
- Transactions are operation-scoped - create inside methods, never store as fields
- ITrackedReader is a lease - pins connection until disposed, dispose promptly
- DbMode.Best auto-selects - SQLite :memory: = SingleConnection
- Always use WrapObjectName() - for column names and aliases in custom SQL
- NEVER use TransactionScope - incompatible with connection management, use Context.BeginTransaction()
- Execution methods return ValueTask - not Task, for reduced allocations
- All async methods have CancellationToken overloads - pass tokens through for proper cancellation