platform-engineering

# Platform Engineering Platform engineering is the discipline of building and maintaining internal developer platforms (IDPs) that enable self-service capabilities for software engineering teams. The goal is to reduce cognitive load, standardize infrastructure provisioning, and accelerate delivery while maintaining governance and security guardrails. --- ## 1. When to Use Adopt platform engineering practices when your organization experiences: - **Cognitive overload on dev teams** -- developers spend more time on infrastructure wiring than writing business logic. - **Inconsistent environments** -- every team provisions infrastructure differently, causing drift and outages. - **Slow onboarding** -- new engineers take weeks to get a working development environment. - **Repeated toil** -- the same Terraform/Helm/CI boilerplate is copy-pasted across dozens of repos. - **Compliance bottlenecks** -- security and ops reviews gate every deployment, slowing release cadence. - **Scale inflection points** -- you have 5+ teams and shared infrastructure concerns (networking, observability, secrets). Platform engineering is NOT about replacing ops with a portal. It is about encoding organizational standards into reusable, self-service abstractions that dev teams consume through golden paths. --- ## 2. Backstage Setup [Backstage](https://backstage.io) is the leading open-source developer portal framework, originally created at Spotify. ### Installation ```bash # Prerequisites: Node.js 18+, yarn 1.x npx @backstage/create-app@latest # Follow the prompts -- name your app, e.g., "internal-platform" cd internal-platform # Start the development server yarn dev ``` ### Production Docker Build ```dockerfile # Dockerfile for Backstage production image FROM node:18-bookworm-slim AS build WORKDIR /app COPY package.json yarn.lock ./ COPY packages/ packages/ COPY plugins/ plugins/ RUN yarn install --frozen-lockfile RUN yarn tsc RUN yarn build:backend FROM node:18-bookworm-slim WORKDIR /app COPY --from=build /app/packages/backend/dist/ ./ COPY --from=build /app/node_modules/ ./node_modules/ COPY app-config.yaml app-config.production.yaml ./ ENV NODE_ENV=production CMD ["node", "packages/backend", "--config", "app-config.production.yaml"] ``` ### Core app-config.yaml ```yaml # app-config.yaml app: title: Internal Developer Platform baseUrl: http://localhost:3000 organization: name: MyOrg backend: baseUrl: http://localhost:7007 listen: port: 7007 database: client: pg connection: host: ${POSTGRES_HOST} port: ${POSTGRES_PORT} user: ${POSTGRES_USER} password: ${POSTGRES_PASSWORD} integrations: github: - host: github.com token: ${GITHUB_TOKEN} catalog: import: entityFilename: catalog-info.yaml pullRequestBranchName: backstage-integration rules: - allow: [Component, System, API, Resource, Location, Template] locations: - type: url target: https://github.com/myorg/software-catalog/blob/main/catalog-info.yaml - type: url target: https://github.com/myorg/backstage-templates/blob/main/all-templates.yaml ``` --- ## 3. Crossplane for Self-Service Infrastructure Crossplane extends Kubernetes to provision and manage cloud infrastructure through declarative YAML. ### Install Crossplane ```bash # Add the Crossplane Helm repo helm repo add crossplane-stable https://charts.crossplane.io/stable helm repo update # Install Crossplane into its own namespace helm install crossplane crossplane-stable/crossplane \ --namespace crossplane-system \ --create-namespace \ --set args='{"--enable-composition-revisions"}' # Install the AWS provider kubectl apply -f - <<EOF apiVersion: pkg.crossplane.io/v1 kind: Provider metadata: name: provider-aws spec: package: xpkg.upbound.io/upbound/provider-family-aws:v1.1.0 EOF # Configure AWS credentials kubectl create secret generic aws-creds \ -n crossplane-system \ --from-file=creds=./aws-credentials.txt kubectl apply -f - <<EOF apiVersion: aws.upbound.io/v1beta1 kind: ProviderConfig metadata: name: default spec: credentials: source: Secret secretRef: namespace: crossplane-system name: aws-creds key: creds EOF ``` ### CompositeResourceDefinition (XRD) This defines a new platform API that developers consume without knowing the underlying cloud resources. ```yaml # xrd-application-database.yaml apiVersion: apiextensions.crossplane.io/v1 kind: CompositeResourceDefinition metadata: name: xapplicationdatabases.platform.myorg.io spec: group: platform.myorg.io names: kind: XApplicationDatabase plural: xapplicationdatabases claimNames: kind: ApplicationDatabase plural: applicationdatabases versions: - name: v1alpha1 served: true referenceable: true schema: openAPIV3Schema: type: object properties: spec: type: object properties: parameters: type: object properties: engine: type: string enum: ["postgres", "mysql"] default: "postgres" engineVersion: type: string default: "15" storageGB: type: integer minimum: 20 maximum: 500 default: 20 instanceSize: type: string enum: ["small", "medium", "large"] default: "small" environment: type: string enum: ["dev", "staging", "prod"] required: - environment ``` ### Composition (AWS RDS) ```yaml # composition-aws-database.yaml apiVersion: apiextensions.crossplane.io/v1 kind: Composition metadata: name: applicationdatabase-aws labels: provider: aws spec: compositeTypeRef: apiVersion: platform.myorg.io/v1alpha1 kind: XApplicationDatabase resources: - name: rds-instance base: apiVersion: rds.aws.upbound.io/v1beta1 kind: Instance spec: forProvider: region: us-east-1 allocatedStorage: 20 autoMinorVersionUpgrade: true backupRetentionPeriod: 7 dbName: appdb deletionProtection: false publiclyAccessible: false skipFinalSnapshot: true storageEncrypted: true storageType: gp3 vpcSecurityGroupIdSelector: matchLabels: platform.myorg.io/network: shared dbSubnetGroupNameSelector: matchLabels: platform.myorg.io/network: shared patches: - type: FromCompositeFieldPath fromFieldPath: spec.parameters.engine toFieldPath: spec.forProvider.engine - type: FromCompositeFieldPath fromFieldPath: spec.parameters.engineVersion toFieldPath: spec.forProvider.engineVersion - type: FromCompositeFieldPath fromFieldPath: spec.parameters.storageGB toFieldPath: spec.forProvider.allocatedStorage - type: CombineFromComposite combine: variables: - fromFieldPath: spec.parameters.instanceSize strategy: map map: small: db.t3.micro medium: db.t3.medium large: db.r6g.large toFieldPath: spec.forProvider.instanceClass - name: db-secret base: apiVersion: secretstores.aws.upbound.io/v1beta1 kind: Secret spec: forProvider: region: us-east-1 patches: - type: FromCompositeFieldPath fromFieldPath: metadata.name toFieldPath: spec.forProvider.name transforms: - type: string string: fmt: "platform/%s/db-credentials" ``` ### Developer Claim (what devs actually write) ```yaml # my-app-database.yaml apiVersion: platform.myorg.io/v1alpha1 kind: ApplicationDatabase metadata: name: orders-db namespace: team-commerce spec: parameters: engine: postgres engineVersion: "15" storageGB: 50 instanceSize: medium environment: staging ``` --- ## 4. Golden Paths Golden paths are opinionated, well-supported paths through your tech stack that teams can follow to ship quickly. ### Backstage Software Template ```yaml # template-nodejs-service.yaml apiVersion: scaffolder.backstage.io/v1beta3 kind: Template metadata: name: nodejs-service title: Node.js Microservice description: Create a production-ready Node.js service with CI/CD, monitoring, and Kubernetes manifests. tags: - recommended - nodejs spec: owner: platform-team type: service parameters: - title: Service Details required: - name - owner - system properties: name: title: Service Name type: string pattern: "^[a-z0-9-]+$" ui:autofocus: true description: title: Description type: string owner: title: Owner Team type: string ui:field: OwnerPicker ui:options: catalogFilter: kind: Group system: title: System type: string ui:field: EntityPicker ui:options: catalogFilter: kind: System - title: Infrastructure properties: database: title: Database type: string enum: ["none", "postgres", "mysql"] default: "none" cacheLayer: title: Cache type: string enum: ["none", "redis"] default: "none" environment: title: Target Environment type: string enum: ["dev", "staging", "prod"] default: "dev" steps: - id: fetch-template name: Fetch Skeleton action: fetch:template input: url: ./skeleton values: name: ${{ parameters.name }} owner: ${{ parameters.owner }} system: ${{ parameters.system }} description: ${{ parameters.description }} database: ${{ parameters.database }} cacheLayer: ${{ parameters.cacheLayer }} - id: publish name: Publish to GitHub action: publish:github input: allowedHosts: ["github.com"] repoUrl: github.com?owner=myorg&repo=${{ parameters.name }} repoVisibility: internal defaultBranch: main protectDefaultBranch: true requireCodeOwnerReviews: true - id: create-argocd-app name: Register with ArgoCD action: argocd:create-resources input: appName: ${{ parameters.name }} argoInstance: main namespace: ${{ parameters.name }} repoUrl: https://github.com/myorg/${{ parameters.name }} path: deploy/k8s - id: register name: Register in Catalog action: catalog:register input: repoContentsUrl: ${{ steps.publish.output.repoContentsUrl }} catalogInfoPath: /catalog-info.yaml output: links: - title: Repository url: ${{ steps.publish.output.remoteUrl }} - title: Open in Backstage icon: catalog entityRef: ${{ steps.register.output.entityRef }} ``` ### Cookiecutter Template Structure ``` golden-path-nodejs/ cookiecutter.json {{cookiecutter.service_name}}/ Dockerfile package.json tsconfig.json src/ index.ts health.ts deploy/ k8s/ deployment.yaml service.yaml ingress.yaml .github/ workflows/ ci.yaml deploy.yaml catalog-info.yaml ``` ```json // cookiecutter.json { "service_name": "my-service", "description": "A new microservice", "owner_team": "platform", "port": "3000", "database": ["none", "postgres", "mysql"], "node_version": "18" } ``` --- ## 5. Service Catalog ### catalog-info.yaml for a Microservice ```yaml # catalog-info.yaml apiVersion: backstage.io/v1alpha1 kind: Component metadata: name: orders-service description: Handles order creation, fulfillment, and tracking. annotations: backstage.io/techdocs-ref: dir:. github.com/project-slug: myorg/orders-service backstage.io/kubernetes-id: orders-service backstage.io/kubernetes-namespace: team-commerce argocd/app-name: orders-service pagerduty.com/integration-key: ${PAGERDUTY_KEY} grafana/dashboard-selector: "app=orders-service" tags: - nodejs - grpc links: - url: https://grafana.internal/d/orders-service title: Grafana Dashboard icon: dashboard - url: https://runbooks.internal/orders-service title: Runbook icon: docs spec: type: service lifecycle: production owner: team-commerce system: commerce-platform providesApis: - orders-api consumesApis: - inventory-api - payments-api dependsOn: - resource:orders-db - resource:orders-cache ``` ### API Entity ```yaml # orders-api.yaml apiVersion: backstage.io/v1alpha1 kind: API metadata: name: orders-api description: Order management API annotations: backstage.io/techdocs-ref: dir:. spec: type: openapi lifecycle: production owner: team-commerce system: commerce-platform definition: $text: ./api/openapi.yaml ``` ### TechDocs Configuration ```yaml # mkdocs.yml (in the service repo root) site_name: Orders Service site_description: Technical documentation for the Orders Service nav: - Home: index.md - Architecture: architecture.md - API Reference: api-reference.md - Runbook: runbook.md - ADRs: - adr/001-choose-grpc.md - adr/002-event-sourcing.md plugins: - techdocs-core markdown_extensions: - admonition - pymdownx.details - pymdownx.superfences: custom_fences: - name: mermaid class: mermaid format: !!python/name:pymdownx.superfences.fence_code_format ``` Enable TechDocs in `app-config.yaml`: ```yaml techdocs: builder: external generator: runIn: docker publisher: type: awsS3 awsS3: bucketName: myorg-techdocs region: us-east-1 credentials: roleArn: arn:aws:iam::123456789012:role/techdocs-publisher ``` --- ## 6. Developer Portal -- Backstage Plugins ### Kubernetes Plugin ```bash # Install Kubernetes plugin yarn --cwd packages/app add @backstage/plugin-kubernetes yarn --cwd packages/backend add @backstage/plugin-kubernetes-backend ``` Backend configuration in `app-config.yaml`: ```yaml kubernetes: serviceLocatorMethod: type: multiTenant