AI Gateway feature
Edge Models
In progress
Run lightweight models at the edge to add “reflexes” before calling an LLM: classification, redaction, enrichment, and routing decisions.
We’re actively building this. Talk to us if you want to help shape the first use cases.
Capabilities
- Edge-side classification and intent routing
- PII detection and redaction as a pre-inference step
- Lightweight enrichment (language detection, summarization, tagging)
- Policy hooks that can influence provider/model selection
Common use cases
- Detect “sensitive” prompts and route to providers with stricter data policies
- Classify requests and pick a cheaper model for low-stakes tasks
- Auto-redact PII before logging/export and before provider calls
- Gate tool usage by detecting intent and risk
Lower cost without losing quality
Use small models to compress, filter, or pre-process so large models see only what matters.
Faster p95 latency
Make quick decisions close to the user (and the provider) before the expensive call.
Safer inputs by default
Apply privacy layers (e.g., PII detection/redaction) before prompts are forwarded.
FAQ
Answers reflect current direction and may evolve as the platform ships.
