01Routing
Routing by rule, with failover
Rules send requests to different upstreams by model, tools, images, extended thinking and more. When an upstream fails before the answer begins, the next one takes over, and each session stays on one upstream so its prompt cache keeps hitting. Auxiliary requests such as title generation and warm-ups can be answered locally without using any quota.
02Protection
Protection against relays: keys replaced, malicious tool calls cut off
API keys, private keys, connection-string passwords, ID numbers and bank card numbers can be replaced with placeholders before a request leaves, so a relay never holds the real values, and an answer whose tool call downloads and runs code or sends out credentials can be cut off before the client runs it. Hidden characters that smuggle in instructions can be deleted and prompt injection refused; the three protections start in Observe, recording without changing anything, and each switches to Replace, Cut off or Enforce on its own.
03Tracing
Every request, traceable
A request shows the rule it matched, each upstream it tried, any conversion between API formats and how its cost was calculated. A finished request can be replayed against another upstream and the two answers compared side by side, and the whole history can be searched, including the text of requests and answers.
04Cost
Costs stated as they are
Tokens, cost, cache savings, time to first token and generation speed, by model and by upstream. Estimated amounts are marked, and requests without a price are counted separately instead of as zero; prices follow LiteLLM's public list, refreshed daily, or a custom price sheet.