model-serving-security

name	model-serving-security
description	Harden a deployed model inference endpoint — authn/authz, rate limiting and abuse control, input validation, and exposure to model-extraction / inversion / membership-inference attacks. Use when reviewing how a model is served (REST/gRPC, Triton/TorchServe/KServe, or a hosted inference API), not how it was trained.

Goal

An inference endpoint that's authenticated, abuse-resistant, and not leaking the model or its training data through its outputs — covering the serving layer beneath LLM/app-level concerns.

What to review

• AuthN/Z & exposure — is the endpoint authenticated and authorized per caller? Not publicly reachable without controls? Tenant isolation for multi-tenant serving?
• Rate limiting & abuse / cost — per-caller quotas to blunt model extraction (stealing the model by querying it) and to cap inference cost/DoS.
• Model extraction — high-volume systematic querying to clone the model; mitigate with rate limits, anomaly detection, output granularity limits, and monitoring.
• Model inversion / membership inference — outputs (esp. confidence scores/logits) that leak training data or reveal whether a record was in the training set; limit exposed confidence detail.
• Input validation & resource safety — malformed/adversarial/oversized inputs; tensor/shape validation; timeouts and resource caps to prevent inference DoS.
• Serving infra — the model server itself (Triton/TorchServe/KServe) patched and hardened; management/metrics endpoints not exposed; no unsafe model auto-loading.

Steps

1. Identify the serving stack, exposure, and who can call it.
2. Check authn/z, rate limiting/quotas, and input validation at the endpoint.
3. Assess extraction/inversion/membership-inference exposure — especially what confidence/score detail is returned.
4. Review the model server's own hardening and management-surface exposure.
5. Recommend controls prioritized by exposure (unauthenticated/public first).

Output

A findings list (surface · risk · evidence · fix) plus controls: authn/z, rate limits/quotas, input validation, output-granularity limits, and monitoring. For LLM-specific output risks (prompt injection, jailbreaks) defer to llm-security; for the app around it, the appsec plugins.

Notes

The model is intellectual property and the training data is a confidentiality boundary — both can leak through a perfectly "working" endpoint via extraction and inversion. Returning raw confidence scores/logits is the quiet enabler of inversion and membership inference; expose the minimum the use case needs. Rate limiting is the single highest-leverage control here.