| name | schema-composition |
| description | Master Effect Schema composition patterns including Schema.compose vs Schema.pipe, transformations, filters, and validation. Use this skill when working with complex schema compositions, multi-step transformations, or when you need to validate and transform data through multiple stages. |
Schema Composition Skill
Expert guidance for composing, transforming, and validating data with Effect Schema.
Core Concepts
The Schema Type
Every schema in Effect has the type signature Schema<Type, Encoded, Context> where:
- Type: The validated, decoded output type (what you get after successful decoding)
- Encoded: The raw input type (what you provide for decoding)
- Context: External dependencies required for encoding/decoding (often
never)
Example:
import { Schema } from "effect"
const NumberFromString = Schema.NumberFromString
Decoding vs Encoding
- Decoding: Transform
Encoded � Type (e.g., string "123" � number 123)
- Encoding: Transform
Type � Encoded (e.g., number 123 � string "123")
Effect Schema follows "parse, don't validate" - schemas transform data into the desired format, not just check validity.
Schema.compose vs Schema.pipe
Understanding when to use compose vs pipe is fundamental to schema composition.
Schema.compose - Chaining Transformations
Use Schema.compose to chain schemas with different types at each stage. It connects the output type of one schema to the input type of another.
Type Signature:
Schema.compose: <A, B, R1>(from: Schema<B, A, R1>) =>
<C, R2>(to: Schema<C, B, R2>) => Schema<C, A, R1 | R2>
When to Use:
- Multi-step transformations where each stage changes the type
- Connecting parsing and validation steps
- Building pipelines from
Encoded � Intermediate � Type
Example - Parse and Validate:
import { Schema } from "effect"
const schema = Schema.compose(
Schema.split(","),
Schema.Array(Schema.NumberFromString)
)
console.log(Schema.decodeUnknownSync(schema)("1,2,3"))
Example - Boolean from String via Literal:
const BooleanFromString = Schema.compose(
Schema.Literal("on", "off"),
Schema.transform(
Schema.Literal("on", "off"),
Schema.Boolean,
{
strict: true,
decode: (s) => s === "on",
encode: (b) => b ? "on" : "off"
}
)
)
Non-strict Composition:
When type boundaries don't align perfectly, use { strict: false }:
Schema.compose(
Schema.Union(Schema.Null, Schema.Literal("0")),
Schema.NumberFromString,
{ strict: false }
)
Schema.pipe - Sequential Refinements
Use Schema.pipe to apply filters and refinements to the same type. It doesn't change the type, just adds validation constraints.
When to Use:
- Adding validation rules to an existing schema
- Chaining multiple filters on the same type
- Refining without transformation
Example - Number Validation:
import { Schema } from "effect"
const PositiveInt = Schema.Number.pipe(
Schema.int(),
Schema.positive()
)
Example - String Validation:
const ValidEmail = Schema.String.pipe(
Schema.trimmed(),
Schema.lowercased(),
Schema.minLength(5),
Schema.pattern(/^[^\s@]+@[^\s@]+\.[^\s@]+$/)
)
Key Differences
| Aspect | Schema.compose | Schema.pipe |
|---|
| Purpose | Chain transformations | Apply refinements |
| Type Change | Changes type at each stage | Type stays the same |
| Example | string � array � numbers | number � positive number |
| Use Case | Multi-step parsing | Validation constraints |
Built-in Filters
Filters add validation constraints without changing the schema's type. They use Schema.filter() under the hood.
String Filters
import { Schema } from "effect"
Schema.String.pipe(Schema.maxLength(5))
Schema.String.pipe(Schema.minLength(5))
Schema.String.pipe(Schema.nonEmptyString())
Schema.String.pipe(Schema.length(5))
Schema.String.pipe(Schema.length({ min: 2, max: 4 }))
Schema.String.pipe(Schema.pattern(/^[a-z]+$/))
Schema.String.pipe(Schema.startsWith("prefix"))
Schema.String.pipe(Schema.endsWith("suffix"))
Schema.String.pipe(Schema.includes("substring"))
Schema.String.pipe(Schema.trimmed())
Schema.String.pipe(Schema.lowercased())
Schema.String.pipe(Schema.uppercased())
Schema.String.pipe(Schema.capitalized())
Schema.String.pipe(Schema.uncapitalized())
Number Filters
Schema.Number.pipe(Schema.greaterThan(5))
Schema.Number.pipe(Schema.greaterThanOrEqualTo(5))
Schema.Number.pipe(Schema.lessThan(5))
Schema.Number.pipe(Schema.lessThanOrEqualTo(5))
Schema.Number.pipe(Schema.between(-2, 2))
Schema.Number.pipe(Schema.int())
Schema.Number.pipe(Schema.nonNaN())
Schema.Number.pipe(Schema.finite())
Schema.Number.pipe(Schema.positive())
Schema.Number.pipe(Schema.nonNegative())
Schema.Number.pipe(Schema.negative())
Schema.Number.pipe(Schema.nonPositive())
Schema.Number.pipe(Schema.multipleOf(5))
Schema.Uint8
Schema.NonNegativeInt
Array Filters
Schema.Array(Schema.Number).pipe(Schema.maxItems(2))
Schema.Array(Schema.Number).pipe(Schema.minItems(2))
Schema.Array(Schema.Number).pipe(Schema.itemsCount(2))
Date Filters
Schema.DateFromSelf.pipe(Schema.validDate())
Schema.Date.pipe(Schema.greaterThanDate(new Date()))
Schema.Date.pipe(Schema.greaterThanOrEqualToDate(new Date()))
Schema.Date.pipe(Schema.lessThanDate(new Date()))
Schema.Date.pipe(Schema.lessThanOrEqualToDate(new Date()))
Schema.Date.pipe(Schema.betweenDate(new Date(0), new Date()))
BigInt Filters
Schema.BigInt.pipe(Schema.greaterThanBigInt(5n))
Schema.BigInt.pipe(Schema.greaterThanOrEqualToBigInt(5n))
Schema.BigInt.pipe(Schema.lessThanBigInt(5n))
Schema.BigInt.pipe(Schema.lessThanOrEqualToBigInt(5n))
Schema.BigInt.pipe(Schema.betweenBigInt(-2n, 2n))
Schema.BigInt.pipe(Schema.positiveBigInt())
Schema.BigInt.pipe(Schema.nonNegativeBigInt())
Schema.BigInt.pipe(Schema.negativeBigInt())
Schema.BigInt.pipe(Schema.nonPositiveBigInt())
BigDecimal Filters
import { BigDecimal } from "effect"
Schema.BigDecimal.pipe(Schema.greaterThanBigDecimal(BigDecimal.unsafeFromNumber(5)))
Schema.BigDecimal.pipe(Schema.lessThanBigDecimal(BigDecimal.unsafeFromNumber(5)))
Schema.BigDecimal.pipe(Schema.betweenBigDecimal(
BigDecimal.unsafeFromNumber(-2),
BigDecimal.unsafeFromNumber(2)
))
Schema.BigDecimal.pipe(Schema.positiveBigDecimal())
Schema.BigDecimal.pipe(Schema.nonNegativeBigDecimal())
Schema.BigDecimal.pipe(Schema.negativeBigDecimal())
Schema.BigDecimal.pipe(Schema.nonPositiveBigDecimal())
Duration Filters
Schema.Duration.pipe(Schema.greaterThanDuration("5 seconds"))
Schema.Duration.pipe(Schema.lessThanDuration("5 seconds"))
Schema.Duration.pipe(Schema.betweenDuration("5 seconds", "10 seconds"))
Custom Filters
Define custom validation logic using Schema.filter():
import { Schema } from "effect"
const LongString = Schema.String.pipe(
Schema.filter(
(s) => s.length >= 10 || "a string at least 10 characters long"
)
)
Filter Return Types
The filter predicate can return:
| Return Type | Meaning |
|---|
true or undefined | Validation passes |
false | Validation fails (no error message) |
string | Validation fails with error message |
ParseResult.ParseIssue | Validation fails with detailed error |
FilterIssue | Validation fails with path and message |
ReadonlyArray<FilterOutput> | Multiple validation errors |
Filter Annotations
Add metadata to filters for better error messages:
const LongString = Schema.String.pipe(
Schema.filter(
(s) => s.length >= 10 ? undefined : "a string at least 10 characters long",
{
identifier: "LongString",
jsonSchema: { minLength: 10 },
description: "A string with at least 10 characters"
}
)
)
Error Paths for Form Validation
Associate errors with specific fields using path:
import { Schema } from "effect"
const Password = Schema.Trim.pipe(Schema.minLength(2))
const MyForm = Schema.Struct({
password: Password,
confirm_password: Password
}).pipe(
Schema.filter((input) => {
if (input.password !== input.confirm_password) {
return {
path: ["confirm_password"],
message: "Passwords do not match"
}
}
})
)
Multiple Error Reporting
Return an array of issues to report multiple errors:
Schema.Struct({
password: Password,
confirm_password: Password,
name: Schema.optional(Schema.String),
surname: Schema.optional(Schema.String)
}).pipe(
Schema.filter((input) => {
const issues: Array<Schema.FilterIssue> = []
if (input.password !== input.confirm_password) {
issues.push({
path: ["confirm_password"],
message: "Passwords do not match"
})
}
if (!input.name && !input.surname) {
issues.push({
path: ["surname"],
message: "Surname must be present if name is not present"
})
}
return issues
})
)
Effectful Filters
Use Schema.filterEffect for async validation:
import { Effect, Schema } from "effect"
async function validateUsername(username: string) {
return Promise.resolve(username === "gcanti")
}
const ValidUsername = Schema.String.pipe(
Schema.filterEffect((username) =>
Effect.promise(() =>
validateUsername(username).then(
(valid) => valid || "Invalid username"
)
)
)
).annotations({ identifier: "ValidUsername" })
Built-in Transformations
Transformations change data from one type to another, unlike filters which only validate.
String Transformations
import { Schema } from "effect"
Schema.Trim
Schema.Lowercase
Schema.Uppercase
Schema.Capitalize
Schema.Uncapitalize
Schema.split(",")
Schema.parseJson()
Schema.parseJson(schema)
Schema.StringFromBase64
Schema.StringFromBase64Url
Schema.StringFromHex
Schema.StringFromUriComponent
Example:
const decode = Schema.decodeUnknownSync(Schema.Trim)
console.log(decode(" hello "))
Number Transformations
Schema.NumberFromString
Boolean Transformations
Schema.Not
Common Transformation Patterns
URL Parsing:
const schema = Schema.URL
Schema.decodeUnknownSync(schema)("https://example.com")
Date Parsing:
const schema = Schema.Date
Schema.decodeUnknownSync(schema)("2020-01-01")
Custom Transformations
Schema.transform - Simple Transformations
Use Schema.transform when the transformation always succeeds:
import { Schema } from "effect"
const BooleanFromString = Schema.transform(
Schema.Literal("on", "off"),
Schema.Boolean,
{
strict: true,
decode: (literal) => literal === "on",
encode: (bool) => bool ? "on" : "off"
}
)
Key Points:
decode transforms from source output to target input
encode transforms from target type back to source type
- Use
strict: true for better TypeScript error messages
Schema.transformOrFail - Transformations That Can Fail
Use Schema.transformOrFail when transformation might fail:
import { ParseResult, Schema } from "effect"
const NumberFromString = Schema.transformOrFail(
Schema.String,
Schema.Number,
{
strict: true,
decode: (input, options, ast) => {
const parsed = parseFloat(input)
if (isNaN(parsed)) {
return ParseResult.fail(
new ParseResult.Type(
ast,
input,
"Failed to convert string to number"
)
)
}
return ParseResult.succeed(parsed)
},
encode: (input, options, ast) => ParseResult.succeed(input.toString())
}
)
Async Transformations
Return an Effect for async transformations:
import { Effect, Schema, ParseResult } from "effect"
const get = (url: string): Effect.Effect<unknown, Error> =>
Effect.tryPromise({
try: () => fetch(url).then((res) => res.json()),
catch: (e) => new Error(String(e))
})
const PeopleId = Schema.String.pipe(Schema.brand("PeopleId"))
const PeopleIdFromString = Schema.transformOrFail(
Schema.String,
PeopleId,
{
strict: true,
decode: (s, _, ast) =>
Effect.mapBoth(get(`https://swapi.dev/api/people/${s}`), {
onFailure: (e) => new ParseResult.Type(ast, s, e.message),
onSuccess: () => s
}),
encode: ParseResult.succeed
}
)
One-Way Transformations
Use ParseResult.Forbidden to prevent encoding:
import { Schema, ParseResult, Redacted } from "effect"
import { createHash } from "node:crypto"
const PlainPassword = Schema.String.pipe(
Schema.minLength(6),
Schema.brand("PlainPassword")
)
const HashedPassword = Schema.String.pipe(
Schema.brand("HashedPassword")
)
const PasswordHashing = Schema.transformOrFail(
PlainPassword,
Schema.RedactedFromSelf(HashedPassword),
{
strict: true,
decode: (plainPassword) => {
const hash = createHash("sha256")
.update(plainPassword)
.digest("hex")
return ParseResult.succeed(Redacted.make(hash))
},
encode: (hashedPassword, _, ast) =>
ParseResult.fail(
new ParseResult.Forbidden(
ast,
hashedPassword,
"Encoding hashed passwords back to plain text is forbidden."
)
)
}
)
Streamlined Effect Patterns
Direct flatMap with Schema.decodeUnknown
Schema.decodeUnknown(schema) returns a function that can be passed directly to Effect.flatMap:
import { Effect, Schema } from "effect"
self.pipe(
Effect.flatMap((value) =>
Schema.decodeUnknown(schema)(value).pipe(
Effect.mapError(toError)
)
)
)
self.pipe(
Effect.flatMap(Schema.decodeUnknown(schema)),
Effect.mapError(toError)
)
Extract Schema Factories
Create reusable schema factories for common patterns:
const createGreaterThanSchema = (n: number) =>
Schema.Number.pipe(Schema.greaterThan(n))
export const beGreaterThan = (n: number) =>
<E, R>(self: Effect.Effect<number, E, R>) =>
self.pipe(
Effect.flatMap(Schema.decodeUnknown(createGreaterThanSchema(n))),
Effect.mapError(toAssertionError)
)
Reuse Composed Schemas
Define schemas once and reuse them:
const TruthySchema = Schema.compose(Schema.BooleanFromUnknown, Schema.Literal(true))
export const beTruthy = () =>
<E, R>(self: Effect.Effect<unknown, E, R>) =>
self.pipe(
Effect.flatMap(Schema.decodeUnknown(TruthySchema)),
Effect.mapError(toAssertionError)
)
Decoding and Encoding
Decoding APIs
| API | Return Type | Use Case |
|---|
decodeUnknownSync | Type (throws on error) | Sync decoding, immediate error |
decodeUnknownOption | Option<Type> | Sync decoding, no error details |
decodeUnknownEither | Either<ParseError, Type> | Sync decoding, error handling |
decodeUnknownPromise | Promise<Type> | Async decoding |
decodeUnknown | Effect<Type, ParseError, Context> | Full Effect-based decoding |
Example:
import { Schema, Either } from "effect"
const Person = Schema.Struct({
name: Schema.String,
age: Schema.Number
})
const person1 = Schema.decodeUnknownSync(Person)({ name: "Alice", age: 30 })
const result = Schema.decodeUnknownEither(Person)({ name: "Alice", age: 30 })
if (Either.isRight(result)) {
console.log(result.right)
}
const asyncResult = Schema.decodeUnknown(asyncSchema)(data)
Effect.runPromise(asyncResult).then(console.log)
Encoding APIs
| API | Return Type | Use Case |
|---|
encodeSync | Encoded (throws on error) | Sync encoding, immediate error |
encodeOption | Option<Encoded> | Sync encoding, no error details |
encodeEither | Either<ParseError, Encoded> | Sync encoding, error handling |
encodePromise | Promise<Encoded> | Async encoding |
encode | Effect<Encoded, ParseError, Context> | Full Effect-based encoding |
Example:
const Person = Schema.Struct({
name: Schema.NonEmptyString,
age: Schema.NumberFromString
})
console.log(Schema.encodeSync(Person)({ name: "Alice", age: 30 }))
Advanced Composition Patterns
Combining Arrays and Transformations
import { Schema } from "effect"
const ReadonlySetFromArray = <A, I, R>(
itemSchema: Schema.Schema<A, I, R>
): Schema.Schema<ReadonlySet<A>, ReadonlyArray<I>, R> =>
Schema.transform(
Schema.Array(itemSchema),
Schema.ReadonlySetFromSelf(Schema.typeSchema(itemSchema)),
{
strict: true,
decode: (items) => new Set(items),
encode: (set) => Array.from(set.values())
}
)
const schema = ReadonlySetFromArray(Schema.String)
Multi-Stage Transformations
const BooleanFromNumericString = Schema.transform(
Schema.NumberFromString,
BooleanFromString,
{
strict: true,
decode: (n) => n > 0 ? "on" : "off",
encode: (bool) => bool === "on" ? 1 : -1
}
)
Conditional Transformations (Non-strict)
When types don't align perfectly, use strict: false:
import { Schema, Number } from "effect"
const clamp = (minimum: number, maximum: number) =>
<A extends number, I, R>(self: Schema.Schema<A, I, R>) =>
Schema.transform(
self,
self.pipe(
Schema.typeSchema,
Schema.filter((a) => a >= minimum && a <= maximum)
),
{
strict: false,
decode: (a) => Number.clamp(a, { minimum, maximum }),
encode: (a) => a
}
)
Struct and Object Schemas
Basic Struct
import { Schema } from "effect"
const Person = Schema.Struct({
name: Schema.String,
age: Schema.Number
})
Optional Fields
const User = Schema.Struct({
username: Schema.String,
email: Schema.optional(Schema.String)
})
Nullable Fields
const Data = Schema.Struct({
value: Schema.NullOr(Schema.String)
})
Partial and Required
const PartialUser = Schema.partial(User)
const RequiredUser = Schema.required(PartialUser)
Picking and Omitting
const Recipe = Schema.Struct({
id: Schema.String,
name: Schema.String,
ingredients: Schema.Array(Schema.String)
})
const JustTheName = Recipe.pick("name")
const NoIDRecipe = Recipe.omit("id")
Extending Structs
const Dog = Schema.Struct({
name: Schema.String,
age: Schema.Number
})
const DogWithBreed = Dog.pipe(
Schema.extend(Schema.Struct({ breed: Schema.String }))
)
const DogWithBreed2 = Schema.Struct({
...Dog.fields,
breed: Schema.String
})
Excess Property Handling
const person = Schema.Struct({
name: Schema.String
})
Schema.decodeUnknownSync(person)(
{ name: "bob dylan", extraKey: 61 },
{ onExcessProperty: "preserve" }
)
Schema.decodeUnknownSync(person)(
{ name: "bob dylan", extraKey: 61 },
{ onExcessProperty: "error" }
)
Common Patterns
Email Validation
const Email = Schema.String.pipe(
Schema.lowercased(),
Schema.trimmed(),
Schema.pattern(/^[^\s@]+@[^\s@]+\.[^\s@]+$/)
)
UUID Validation
const UserId = Schema.UUID.pipe(
Schema.brand("UserId")
)
Clamping Numbers
const Percentage = Schema.Number.pipe(
Schema.between(0, 100),
Schema.brand("Percentage")
)
Template Literal Parsing
const AuthToken = Schema.TemplateLiteralParser(
"Bearer ",
Schema.String.pipe(Schema.brand("Token"))
)
Branded Types
const PositiveInt = Schema.Number.pipe(
Schema.int(),
Schema.positive(),
Schema.brand("PositiveInt")
)
Form Validation
const LoginForm = Schema.Struct({
email: Schema.String.pipe(
Schema.lowercased(),
Schema.pattern(/^[^\s@]+@[^\s@]+\.[^\s@]+$/)
),
password: Schema.String.pipe(
Schema.minLength(8),
Schema.pattern(/^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)/)
)
})
API Response Parsing
const User = Schema.Struct({
id: Schema.NumberFromString,
name: Schema.String,
email: Schema.String,
createdAt: Schema.DateFromString
})
const UsersResponse = Schema.Struct({
users: Schema.Array(User),
total: Schema.Number
})
Quality Checklist
When creating schemas, ensure:
Key Principles
- Composition over custom logic - Leverage
Schema.compose and Schema.pipe instead of manual validation
- Reusability - Extract schemas as constants or factory functions
- Type safety - Let Schema handle type inference and refinement
- Streamlined Effect chains - Minimize lambda wrappers, use direct function passing
- Built-in schemas first - Use Effect's built-in schemas before creating custom ones
- Parse, don't validate - Transform data into the desired format, not just check it
- Fail fast, fail clearly - Provide detailed error messages with paths and context
References
- Effect Schema is imported from
effect/Schema or { Schema } from "effect"
- Schema API signature:
Schema<Type, Encoded, Context>
- All schemas return
readonly types by default
- Use
Schema.asSchema to view any schema as Schema<Type, Encoded, Context>
- Access base schema before filter with
.from property
- Access struct fields with
.fields property