We run it after go-live, and you can take it back whenever you want
Monitoring, on-call, model lifecycle, cost attribution and recovery for AI in production — against an SLA, with no exit fee.
The service
We monitor whether the system is right, not only whether it is responding. Live traffic is sampled and scored against the evaluation set used at release, inputs are watched for drift, and alerts fire on the gap between expected and observed quality.
Availability
99.9%
Standard-tier target, measured on responses inside the latency objective
Response
15 min
Severity-one acknowledgement on the 24/7 tier, with an engineer engaged
Lifecycle
Suppliers deprecate versions and change behaviour within one version name. We test candidates against your evaluation set first and release by canary, with rollback in minutes.
Cost
Tokens and GPU-hours tagged by workload and model, reported as cost per resolution or document. What we find is routine: prompts that grew, retries never bounded, oversized models.
Resilience
Vector indexes need a measured rebuild time, adapters need lineage, configuration needs point-in-time recovery, and a provider outage should degrade to a fallback, not go down.
Observability
Incident management
Evaluation
Delivery
Security and resilience
Who we do this for
2–3 weeks
An assessment of monitoring gaps, evaluation coverage, cost attribution, recovery posture and on-call maturity.
Gap analysis, prioritised remediation plan, draft severity matrix
12 months, rolling
We take the pager on an agreed tier, covering incidents, model lifecycle, patching and continuous evaluation.
Service definition, monitoring in your accounts, monthly operations report
Ongoing, with an end date
The same service, run with an explicit plan to give it back: shadow on-call, then primary, on a date you set.
Runbooks, architecture documentation, incident history, training record
Yes, and the service assumes you might. Everything runs in your accounts, with code, prompts, evaluation sets, runbooks and incident history in your repositories from day one. No proprietary agent, no exit fee. Your engineers join the rota, take shadow on-call, then primary, and we step down on a date you choose.
If you already have a capable platform team with an on-call rota and an observability practice, buy the readiness review instead, fix the AI-specific gaps and run it yourself. It is also wrong for a low-volume internal tool where an hour of downtime costs nothing — the overhead exceeds the risk it removes.
Three layers. Live traffic is sampled and scored against the golden set used at release, so a regression moves against a known baseline. Input distributions are watched for drift, because the questions shift before the answers get worse. Escalation, refusal and retry rates act as leading indicators.
Send us the constraint you are actually up against — budget, latency, regulator, deadline.
