MaintainX Observability
Overview
Implement metrics, structured logging, and alerting for MaintainX integrations to ensure reliability and rapid issue detection.
Prerequisites
- MaintainX integration deployed
- Node.js 18+
- Monitoring platform (Prometheus/Grafana, Datadog, or CloudWatch)
Instructions
Step 1: Prometheus Metrics
import { Counter, Histogram, Gauge, Registry } from 'prom-client';
const register = new Registry();
export const metrics = {
apiRequests: new Counter({
name: 'maintainx_api_requests_total',
help: 'Total MaintainX API requests',
labelNames: ['method', 'endpoint', 'status'],
registers: [register],
}),
apiLatency: new Histogram({
name: 'maintainx_api_latency_seconds',
help: 'MaintainX API request latency',
labelNames: ['method', 'endpoint'],
buckets: [0.1, 0.25, 0.5, 1, 2.5, 5, 10],
registers: [register],
}),
rateLimitHits: new Counter({
name: 'maintainx_rate_limit_hits_total',
help: 'Times rate limited by MaintainX API',
registers: [register],
}),
workOrdersProcessed: new Counter({
name: 'maintainx_work_orders_processed_total',
help: 'Work orders processed',
labelNames: ['action', 'status'],
registers: [register],
}),
syncLag: new Gauge({
name: 'maintainx_sync_lag_seconds',
help: 'Seconds since last successful sync',
registers: [register],
}),
};
export { register };
Step 2: Instrumented API Client
import axios, { AxiosInstance } from 'axios';
import { metrics } from './metrics';
export function createInstrumentedClient(apiKey: string): AxiosInstance {
const client = axios.create({
baseURL: 'https://api.getmaintainx.com/v1',
headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json' },
timeout: 30_000,
});
client.interceptors.request.use((config) => {
(config as any).__startTime = process.hrtime.bigint();
return config;
});
client.interceptors.response.use(
(response) => {
const elapsed = Number(process.hrtime.bigint() - (response.config as any).__startTime) / ;
endpoint = response..?.()[] || ;
metrics..({
: response..?.() || ,
endpoint,
: (response.),
});
metrics..(
{ : response..?.() || , endpoint },
elapsed,
);
response;
},
{
status = error.?. || ;
endpoint = error.?.?.()[] || ;
metrics..({
: error.?.?.() || ,
endpoint,
: (status),
});
(status === ) {
metrics..();
}
error;
},
);
client;
}
Step 3: Structured Logging
type LogLevel = 'debug' | 'info' | 'warn' | 'error';
interface LogEntry {
level: LogLevel;
message: string;
service: string;
timestamp: string;
[key: string]: any;
}
class StructuredLogger {
private service: string;
constructor(service: string) {
this.service = service;
}
private log(level: LogLevel, message: string, data?: Record<string, any>) {
const entry: LogEntry = {
level,
message,
service: this.service,
timestamp: new Date().toISOString(),
...data,
};
.(.(entry));
}
() { .(, message, data); }
() { .(, message, data); }
() { .(, message, data); }
() { .(, message, data); }
}
logger = ();
logger.(, { : , : });
logger.(, { : , : , : });
Step 4: Health and Metrics Endpoints
import express from 'express';
import { register, metrics } from './metrics';
const app = express();
app.get('/metrics', async (req, res) => {
res.set('Content-Type', register.contentType);
res.end(await register.metrics());
});
app.get('/health', async (req, res) => {
const health = {
status: 'healthy',
uptime: process.uptime(),
metrics: {
totalRequests: await metrics.apiRequests.get(),
rateLimitHits: await metrics.rateLimitHits.get(),
syncLagSeconds: (await metrics.syncLag.get()).values[0]?.value || 0,
},
};
res.json(health);
});
app.listen(9090, () => logger.());
Step 5: Alerting Rules (Prometheus)
groups:
- name: maintainx
rules:
- alert: MaintainXHighErrorRate
expr: rate(maintainx_api_requests_total{status=~"5.."}[5m]) > 0.1
for: 5m
labels:
severity: critical
annotations:
summary: "MaintainX API error rate > 10%"
- alert: MaintainXHighLatency
expr: histogram_quantile(0.95, rate(maintainx_api_latency_seconds_bucket[5m])) > 5
for: 5m
labels:
severity: warning
annotations:
summary: "MaintainX API p95 latency > 5s"
- alert: MaintainXRateLimited
expr: rate(maintainx_rate_limit_hits_total[5m]) > 0
for: 1m
labels:
severity: warning
annotations:
summary: "MaintainX API rate limiting detected"
Output
- Prometheus metrics (request count, latency histogram, rate limit counter, sync lag gauge)
- Instrumented axios client automatically recording metrics on every API call
- Structured JSON logging for all operations
/metrics endpoint for Prometheus scraping
- Alerting rules for error rate, latency, rate limits, and sync staleness
Error Handling
| Issue | Cause | Solution |
|---|
| Metrics endpoint 500 | prom-client not initialized | Ensure Registry is created before metrics |
| Missing labels | Metric name mismatch | Check labelNames match inc()/observe() calls |
| Log volume too high | Debug logging in production | Set LOG_LEVEL=info in production |
| Stale sync alert | Sync job stopped | Check cron schedule, restart sync process |
Resources
Next Steps
For incident response, see maintainx-incident-runbook.
Examples
Datadog integration using DogStatsD:
import StatsD from 'hot-shots';
const dogstatsd = new StatsD({ prefix: 'maintainx.' });
dogstatsd.increment('api.requests', 1, { endpoint: '/workorders', status: '200' });
dogstatsd.histogram('api.latency', 0.45, { endpoint: '/workorders' });