observability-instrumentation

name	observability-instrumentation
description	Add business metrics using AWS EMF (Embedded Metric Format) to API endpoints. Focus on domain-specific metrics only - AWS AppRunner provides default SLO/SLA metrics. Use when implementing new endpoints, adding command handlers, or instrumenting business events.

Business Metrics with AWS EMF

Instrument API endpoints with business metrics using AWS CloudWatch Embedded Metric Format (EMF). This skill focuses exclusively on domain-specific metrics - AWS AppRunner already provides infrastructure SLO/SLA metrics automatically.

What This Skill Covers

• Business metrics - Domain events (customers created, orders placed, payments processed)
• AWS EMF format - Logs that automatically become CloudWatch metrics
• Event subscribers - Metrics emitted via domain event subscribers (not in handlers)
• Type-safe metrics - Concrete metric classes instead of arrays
• SOLID principles - Single Responsibility (subscribers) + Open/Closed (new metric classes)

What This Skill Does NOT Cover

• Infrastructure metrics - Latency, error rates, RPS (AWS AppRunner provides these)
• SLO/SLA metrics - Availability, response times (AWS AppRunner provides these)
• Distributed tracing - Use AWS X-Ray integration instead

When to Use This Skill

Use this skill when:

• Implementing new API endpoints that have business significance
• Adding domain events that should trigger metric emission
• Tracking domain events for analytics and business intelligence
• Building dashboards for business KPIs

---

Architecture Overview

Business metrics follow these patterns:

1. Metric classes - Each metric type is a concrete class extending BusinessMetric
2. Event subscribers - Metrics are emitted via domain event subscribers (not hardcoded in handlers)
3. Symfony logger - EMF output goes through Monolog with a custom EMF formatter
4. No arrays - All metric configuration uses typed objects, not arrays
5. Collections - Multiple metrics use MetricCollection, not arrays

---

SOLID Principles in Observability

Single Responsibility Principle (SRP)

Each class has ONE responsibility:

Class	Responsibility
CustomersCreatedMetric	Define metric name, value, dimensions
CustomerCreatedMetricsSubscriber	Listen to event, emit metric
AwsEmfBusinessMetricsEmitter	Format and write EMF logs
MetricCollection	Hold multiple metrics for batch emission

Anti-pattern: Metrics emitted directly in command handlers (violates SRP - handler should only handle commands)

Open/Closed Principle (OCP)

• Open for extension: Add new metrics via new classes
• Closed for modification: Don't change existing metric/emitter code

// ✅ GOOD: Add new metric by creating new class
final readonly class OrdersPlacedMetric extends EndpointOperationBusinessMetric { ... }

// ❌ BAD: Modify existing emitter to handle new metric type

Why Event Subscribers (Not Handler Injection)

// ❌ BAD: Metrics in command handler (violates SRP)
final class CreateCustomerHandler
{
    public function __construct(
        private CustomerRepository $repository,
        private BusinessMetricsEmitterInterface $metrics  // Wrong!
    ) {}
}

// ✅ GOOD: Metrics in dedicated event subscriber
final class CustomerCreatedMetricsSubscriber implements DomainEventSubscriberInterface
{
    public function __invoke(CustomerCreatedEvent $event): void
    {
        $this->metricsEmitter->emit($this->metricFactory->create());
    }
}

Benefits:

• Handler focuses on domain logic only
• Metrics emission is decoupled and testable
• Easy to add/remove metrics without touching business logic
• Multiple subscribers can react to same event

---

Type-Safe Metric Class Hierarchy

BusinessMetric (abstract)
├── EndpointOperationBusinessMetric (abstract) - for metrics with Endpoint/Operation dimensions
│   ├── CustomersCreatedMetric
│   ├── CustomersUpdatedMetric
│   ├── CustomersDeletedMetric
│   └── EndpointInvocationsMetric
└── (other base classes for different dimension patterns)

MetricDimensionsInterface
├── EndpointOperationMetricDimensions - Endpoint + Operation
└── (custom dimensions for specific metrics)

MetricDimensions - typed collection of MetricDimension objects
MetricDimension - key/value pair

MetricUnit (enum)
├── COUNT, NONE, SECONDS, MILLISECONDS, BYTES, PERCENT

MetricCollection - typed collection implementing IteratorAggregate, Countable

Why no arrays?

Arrays	Typed Classes
No type safety	Full type checking
No IDE autocomplete	IDE support
Runtime errors	Compile-time errors
Hard to refactor	Easy to refactor
No encapsulation	Validation in constructor

---

Current Implementation

Metric Base Class (Application Layer)

// src/Shared/Application/Observability/Metric/BusinessMetric.php
abstract readonly class BusinessMetric
{
    public function __construct(
        private float|int $value,
        private MetricUnit $unit
    ) {}

    abstract public function name(): string;
    abstract public function dimensions(): MetricDimensionsInterface;

    public function value(): float|int { return $this->value; }
    public function unit(): MetricUnit { return $this->unit; }
}

Concrete Metric Example

// src/Core/Customer/Application/Metric/CustomersCreatedMetric.php
final readonly class CustomersCreatedMetric extends EndpointOperationBusinessMetric
{
    private const ENDPOINT = 'Customer';
    private const OPERATION = 'create';

    public function __construct(
        MetricDimensionsFactoryInterface $dimensionsFactory,
        float|int $value = 1
    ) {
        parent::__construct($dimensionsFactory, $value, MetricUnit::COUNT);
    }

    public function name(): string
    {
        return 'CustomersCreated';
    }

    protected function endpoint(): string
    {
        return self::ENDPOINT;
    }

    protected function operation(): 
    {
         ::;
    }
}

Emitter Interface (Application Layer)

// src/Shared/Application/Observability/Emitter/BusinessMetricsEmitterInterface.php
interface BusinessMetricsEmitterInterface
{
    public function emit(BusinessMetric $metric): void;
    public function emitCollection(MetricCollection $metrics): void;
}

Metrics Event Subscriber

// src/Core/Customer/Application/EventSubscriber/CustomerCreatedMetricsSubscriber.php
/**
 * Error handling is automatic via DomainEventMessageHandler in async workers.
 * Subscribers stay clean - failures are logged + emit metrics automatically.
 * This ensures observability never breaks the main request (AP from CAP theorem).
 */
final readonly class CustomerCreatedMetricsSubscriber implements DomainEventSubscriberInterface
{
    public function __construct(
        private BusinessMetricsEmitterInterface $metricsEmitter,
        private CustomersCreatedMetricFactoryInterface $metricFactory
    ) {
    }

    public function __invoke(CustomerCreatedEvent $event): void
    {
        $this->metricsEmitter->emit($this->metricFactory->create());
    }

    public function subscribedTo(): array
    {
        return [CustomerCreatedEvent::class];
    }
}

---

AWS EMF Format

AWS Embedded Metric Format allows you to embed custom metrics in structured log events. CloudWatch automatically extracts metrics from EMF-formatted logs.

EMF Log Structure

{
  "_aws": {
    "Timestamp": 1702425600000,
    "CloudWatchMetrics": [
      {
        "Namespace": "CCore/BusinessMetrics",
        "Dimensions": [["Endpoint", "Operation"]],
        "Metrics": [{ "Name": "CustomersCreated", "Unit": "Count" }]
      }
    ]
  },
  "Endpoint": "Customer",
  "Operation": "create",
  "CustomersCreated": 1
}

When this log is written to stdout via the EMF Monolog channel, CloudWatch automatically:

1. Extracts CustomersCreated as a metric
2. Associates it with the CCore/BusinessMetrics namespace
3. Applies dimensions Endpoint and Operation

---

Creating New Business Metrics

Step 1: Create the Metric Class

// src/Core/Order/Application/Metric/OrdersPlacedMetric.php
namespace App\Core\Order\Application\Metric;

use App\Shared\Application\Observability\Metric\BusinessMetric;
use App\Shared\Application\Observability\Metric\MetricDimension;
use App\Shared\Application\Observability\Metric\MetricDimensions;
use App\Shared\Application\Observability\Metric\MetricDimensionsFactoryInterface;
use App\Shared\Application\Observability\Metric\MetricDimensionsInterface;
use App\Shared\Application\Observability\Metric\MetricUnit;

final readonly class OrdersPlacedMetricDimensions implements 
{
     {
    }

     {
         ->dimensionsFactory->(
            ,
            ,
             (, ->paymentMethod)
        );
    }
}

  {
     {
        ::(, ::);
    }

     {
         ;
    }

     {
          (
            dimensionsFactory: ->dimensionsFactory,
            paymentMethod: ->paymentMethod
        );
    }
}

Step 2: Create the Event Subscriber

// src/Core/Order/Application/EventSubscriber/OrderPlacedMetricsSubscriber.php
namespace App\Core\Order\Application\EventSubscriber;

use App\Core\Order\Application\Factory\OrdersPlacedMetricFactoryInterface;
use App\Core\Order\Domain\Event\OrderPlacedEvent;
use App\Shared\Application\Observability\Emitter\BusinessMetricsEmitterInterface;
use App\Shared\Domain\Bus\Event\DomainEventSubscriberInterface;

final readonly class OrderPlacedMetricsSubscriber implements DomainEventSubscriberInterface
{
    public function __construct(
        private BusinessMetricsEmitterInterface $metricsEmitter,
        private OrdersPlacedMetricFactoryInterface $metricFactory
    ) {}

    public  (): 
    {
        ->metricsEmitter->(->metricFactory->(->()));
    }

    
     {
         [::];
    }
}

Step 3: For Multiple Metrics - Use MetricCollection

// Emit multiple metrics together (dimensionsFactory injected via constructor)
$this->metricsEmitter->emitCollection(new MetricCollection(
    $this->ordersPlacedMetricFactory->create($event->paymentMethod()),
    $this->orderValueMetricFactory->create($event->totalAmount())
));

---

Dimension Best Practices

Recommended Dimensions

Dimension	Description	Cardinality
Endpoint	API resource name	Low
Operation	CRUD action	Very Low
PaymentMethod	Payment type	Low
CustomerType	Customer segment	Low

Avoid High-Cardinality Dimensions

Don't use:

• Customer IDs
• Order IDs
• Session IDs
• Timestamps

These create too many unique metric streams and increase CloudWatch costs.

---

Metric Naming Conventions

Format

{Entity}{Action}   # PascalCase

Examples

Good	Bad
CustomersCreated	customer_created
OrdersPlaced	orders.placed.count
PaymentsProcessed	payment-processed

Guidelines

• Use PascalCase for metric names
• Use plural nouns for counters (CustomersCreated not CustomerCreated)
• Use past tense for completed actions

---

Testing Business Metrics

Use the Spy in Tests

use App\Shared\Application\Observability\Metric\MetricDimension;
use App\Shared\Infrastructure\Observability\Factory\MetricDimensionsFactory;
use App\Tests\Unit\Shared\Infrastructure\Observability\BusinessMetricsEmitterSpy;

final class CustomerCreatedMetricsSubscriberTest extends TestCase
{
    public function testEmitsMetricOnCustomerCreated(): void
    {
        $metricsSpy = new BusinessMetricsEmitterSpy();
        $dimensionsFactory = new MetricDimensionsFactory();
        $metricFactory = new CustomersCreatedMetricFactory($dimensionsFactory);
        $logger = $this->createMock(LoggerInterface::class);

        $subscriber = new CustomerCreatedMetricsSubscriber(
            ,
            ,
            
        );

         =  (, );
        ()();

        ::(, ->());

         (->()  ) {
            ::(, ->());
            ::(, ->());
            ::(, ->()->()->());
            ::(, ->()->()->());
        }

        
        ->(
            ,
             (, ),
             (, )
        );
    }
}

Test Service Configuration

In config/services_test.yaml, the spy is configured:

App\Shared\Application\Observability\Emitter\BusinessMetricsEmitterInterface: '@App\Tests\Unit\Shared\Infrastructure\Observability\BusinessMetricsEmitterSpy'

App\Tests\Unit\Shared\Infrastructure\Observability\BusinessMetricsEmitterSpy:
  public: true

---

CloudWatch Queries

After deploying, query your business metrics:

-- Total endpoint invocations by resource
SELECT SUM(EndpointInvocations)
FROM "CCore/BusinessMetrics"
GROUP BY Endpoint

-- Customers created over time
SELECT SUM(CustomersCreated)
FROM "CCore/BusinessMetrics"
WHERE Endpoint = 'Customer'

---

What NOT to Track

Remember: AWS AppRunner already provides infrastructure metrics.

Don't track:

• Request latency
• Error rates
• Response times
• HTTP status codes
• Memory usage
• CPU usage

Do track:

• Business events (orders placed, customers created)
• Business values (order amounts, payment totals)
• Domain-specific actions (logins, uploads, exports)

---

Success Criteria

After implementing business metrics:

• Each domain event that needs tracking has a corresponding metric subscriber
• Metrics use typed classes (not arrays)
• Metrics are emitted via event subscribers (not hardcoded in handlers)
• Dimensions provide meaningful segmentation
• Unit tests verify metric emission
• No infrastructure metrics (AppRunner handles those)

SOLID Compliance Checklist

• SRP: Each metric class has single purpose (define one metric type)
• SRP: Event subscriber only emits metrics (no business logic)
• OCP: New metrics added via new classes (no modification to emitter)
• OCP: New event subscribers added without changing existing code
• LSP: All metrics properly extend BusinessMetric base class
• ISP: MetricDimensionsInterface is minimal (only values())
• DIP: Handlers depend on EventBusInterface, not concrete metrics

Type Safety Checklist

• NO arrays for metric configuration - use typed classes
• NO arrays for metric collections - use MetricCollection
• All dimensions via MetricDimensionsInterface implementations
• Arrays are allowed only at infrastructure boundaries (JSON serialization, PSR-3 log context)
• Unit enum MetricUnit used for all units

---

Files Reference

Metric Classes

• src/Shared/Application/Observability/Metric/BusinessMetric.php - Base class
• src/Shared/Application/Observability/Metric/MetricUnit.php - Unit enum
• src/Shared/Application/Observability/Metric/MetricDimension.php - Dimension key/value
• src/Shared/Application/Observability/Metric/MetricDimensions.php - Dimension collection
• src/Shared/Application/Observability/Metric/MetricCollection.php - Metrics collection
• src/Shared/Application/Observability/Metric/EndpointInvocationsMetric.php - Endpoint metric
• src/Core/Customer/Application/Metric/CustomersCreatedMetric.php - Customer create metric
• src/Core/Customer/Application/Metric/CustomersUpdatedMetric.php - Customer update metric
• src/Core/Customer/Application/Metric/CustomersDeletedMetric.php - Customer delete metric

Infrastructure

• src/Shared/Application/Observability/Emitter/BusinessMetricsEmitterInterface.php - Interface
• src/Shared/Infrastructure/Observability/AwsEmfBusinessMetricsEmitter.php - EMF implementation
• src/Shared/Infrastructure/Observability/EmfLogFormatter.php - Monolog formatter

Event Subscribers

• src/Shared/Infrastructure/Observability/ApiEndpointBusinessMetricsSubscriber.php - HTTP metrics
• src/Core/Customer/Application/EventSubscriber/CustomerCreatedMetricsSubscriber.php
• src/Core/Customer/Application/EventSubscriber/CustomerUpdatedMetricsSubscriber.php
• src/Core/Customer/Application/EventSubscriber/CustomerDeletedMetricsSubscriber.php

Configuration

• config/packages/monolog.yaml - EMF channel configuration
• config/services.yaml - Production wiring
• config/services_test.yaml - Test spy wiring

---

AWS Documentation

• CloudWatch Embedded Metric Format
• AWS App Runner Metrics