Configuration reference#

ThinkWatch Core reads one file, config.yaml. This page describes every field in it: what it does, its default, the values it takes, and how a change reaches the running process. To run core on a server and control it from the desktop app, see Running core on a server.

The field tables on this page are generated from the code, and a test fails when they disagree, so a field listed here is a field the binary reads.

Where the file is#

THINKWATCH_HOME replaces the directory, and --config <path> names the file for a single command. The directory holds everything else core keeps as well: the request database (data.db), the configuration history (history/), the downloaded price table (model_prices.json) and the local control socket (twcore.sock; on Windows a loopback port recorded in control.port). The directory is private to its owner (0700), the file is 0600: it holds keys in plain text.

These permissions are the file’s only protection on this machine. They keep out other users, not programs running as the same user: such a program can read every key in the file, and with the control key it holds, change the configuration through the control plane. Outbound redaction does not change that; it protects what a request carries off the machine, not what is on disk. The control key is masked where the configuration is shown so that a write through the control plane cannot change it (listen.control), not to keep it from local programs; upstream and gateway keys are shown as written.

Tool-call inspection has a built-in rule for this directory, thinkwatch-data. A tool call whose path or command points into one of the default locations above, or into /var/lib/thinkwatch or /etc/thinkwatch on a server, is recorded, and under enforce the response is cut off, so a model cannot be steered into reading these keys or rewriting its own protections. Mentioning the path, as in a document being edited, does not count. With THINKWATCH_HOME elsewhere, a custom rule in security.inspect_tools can cover that path.

twcore serve writes a starting configuration when there is none, and twcore init writes one on request. Both produce this:

version: 1
listen:
  control:
    key: 6629…753d       # generated
clients:
  - name: default
    key: tw-…            # generated

That is a complete, valid configuration. It has no upstream yet, so the control plane runs and requests are answered with an error saying so. Adding one upstream makes it forward:

providers:
  - name: anthropic
    base_url: https://api.anthropic.com
    key: ${ANTHROPIC_API_KEY}

How the file is read#

• A field name that is not in this reference is an error. A misspelled prot: is not ignored while the gateway quietly starts on the default port; the whole file is refused and the message names the field.
• Defaults are not written into the file. Anything left out has the default in the tables below. The app and the command line add a field only when its value differs from the default, so what is in the file is what someone chose.
• ${VAR} reads an environment variable in fields marked “${VAR} allowed”: upstream keys, header values, proxy passwords. It is read from the environment of the core process when a request is sent; an unset variable fails that upstream’s requests and names the variable. Under systemd, that environment is the unit’s EnvironmentFile.
• Names are references. Rules, groups and keys refer to upstreams, groups, routes and price sheets by name. A name that points nowhere is an error at load time rather than a rule that never matches. Renaming in the app changes every reference in the same write. Names starting with __ are reserved for built-ins.
• version is the format version, 1. A file with a higher version was written by a newer twcore and is refused.

How a change takes effect#

There are three ways to change the configuration, and they all go through the same path: the desktop app, twcore config …, and editing the file in an editor. Core watches the file and reloads it within a second of a save; nothing needs to be restarted.

A new version is applied only if it passes every stage:

1. It parses as YAML.
2. It matches the schema: known fields, the right types.
3. It is consistent: names are unique, references resolve, patterns compile, CIDR ranges are well formed.
4. The runtime objects can be built from it.

If any stage fails, the previous configuration stays in service, and the error says which stage failed and where. A typo never takes the gateway down. The desktop app shows the rejection until a valid version is saved.

Changes to listen.gateway apply live as well: core opens the new listener, and if it cannot (the port is taken) it keeps the old one and reports why. Retention changes are applied on the next hourly clean-up.

When two writers edit at once (the app and a hand edit), the second write is refused with a version mismatch instead of overwriting the first.

History and rollback#

Every version that was in effect is kept in history/ beside the file, with where it came from (the app, the command line, an outside edit, a rollback, a credential rotation). The last 50 are kept.

twcore check                      # validate the file without starting anything
twcore config show                # print it, with its version
twcore config history             # list the versions, newest first
twcore config rollback 3f9a2c     # go back to a version (a prefix is enough)
twcore config set /listen/gateway/port 8790 --int

These commands work on the file directly, so they work when core is not running, which is when a rollback is most needed. A running core picks their changes up like any other save.

twcore config set <path> <value> changes one value that is already written in the file. The path names list items by their name (/providers/anthropic/base_url); a number is an index (/routes/0/rules/1/to). The value is a string unless --int, --bool or --null says otherwise. The result is validated before it is written. To add a field that is not in the file yet, edit the file.

Credentials the gateway writes back#

One write is not made by a person. When an upstream’s OAuth token endpoint issues a new refresh token, the old one stops working, so the gateway writes the new token (and the access token with its expiry) back into providers[].oauth. Only those values change; comments and layout are left as they are.

Reference#

Each table lists every field of one section. ”—” in the Default column means the field is simply absent unless written; the description says what that means.

Top level#

listen#

Where core accepts connections. There are two kinds: the AI gateway that clients send requests to, and the control channel the desktop app and twcore commands use.

listen.gateway#

When bind reaches beyond this machine, allow_from decides who gets in. The gateway has no TLS: expose it on networks you trust, or put it behind a tunnel or VPN.

listen:
  gateway:
    bind: all
    port: 8788
    allow_from: [192.168.1.0/24]

listen.control#

The control channel is how the desktop app, and twcore config and twcore control-key on the same machine, talk to core. Locally it is a socket file in the data directory (a loopback port on Windows); no network port is opened for it unless remote is enabled.

Every control connection, local or remote, starts with a handshake that proves both ends hold key (Noise_NNpsk0_25519_ChaChaPoly_BLAKE2s: the key is the pre-shared key, each connection negotiates fresh session keys, and the traffic is encrypted). There are no certificates. A peer without the key cannot complete the first message.

key is written by twcore serve before the control channel starts listening, if it is missing, and by twcore init. It is masked wherever the configuration is shown or kept in the history; saving a masked value back keeps the real one. A missing or malformed key (not 64 hexadecimal characters) makes the whole file invalid, so it cannot be replaced by a short, guessable one.

twcore control-key            # print the key, to paste into the desktop app
twcore control-key --rotate   # replace it; connected apps have to reconnect

Both run on the machine where core runs.

listen.control.remote#

A network port for a desktop app on another machine. It is opened in addition to the local channel, so a mistake here (a port that is taken, an allow_from that shuts you out) never locks out the machine itself: twcore config and the local app keep working.

listen:
  control:
    key: 9f2c…e41a          # 64 hexadecimal characters
    remote:
      enabled: true
      bind: all
      port: 23483            # random, written when the section is generated
      allow_from: [192.168.1.0/24]

A connection over the remote port cannot do three things, whatever the app asks: shut core down (it is managed by systemd), change listen.control (the door it came in through), or produce a diagnostic bundle (it would be written on the server). Handshakes time out after 5 seconds; five failed handshakes from one source within a minute block that source for a minute.

clients#

Gateway keys: the keys clients such as Claude Code and Codex send to the gateway. A key is an identity. Limits, model scope and route are per key.

clients:
  - name: default
    key: tw-a3f9c8d1e5b2h7k4m6n8p2q4
  - name: build-server
    key: tw-q8r2s4t6u8v2w4x6y8z2a4b6
    max_concurrent: 4
    allow: [claude-sonnet-*]
    route: cheap

clients[].limits#

Usage limits for a key: at most so many requests, tokens or dollars per minute, hour, day, week or month. Each entry counts one of the three; a key can have several, and a request has to pass every one.

minute and hour are rolling: the last 60 seconds, the last 60 minutes. When one is used up, a request waits for the next free slot if it frees within failover.slot_wait_secs (30 seconds by default), and is refused otherwise. Any later wait for a busy upstream comes out of the same time. day, week and month follow the calendar in the time zone of the machine twcore runs on and start again at midnight, on Monday and on the 1st. When one is used up, requests are refused until it starts again. If the machine’s time zone changes, the current day, week and month are added up again from the request records.

A refused request gets HTTP 429 in the client’s own error format, naming the key, the limit, the amount used and when it resets, and it shows in the traffic list. A request that never reaches an upstream counts toward no limit: one refused by a rule, the content filter or a limit, or turned away because every upstream was at its max_concurrent. Cost is what is recorded for each request, so a model without a price and an upstream with billing: free count as $0. A request still running counts with an estimate of its input until it is recorded. After a restart, the day, week and month are added up again from the request records, so the records have to cover the period: retention.row_days of at least 1 for a daily limit, 7 for a weekly one and 31 for a monthly one. Minute and hour limits start empty.

On a Responses WebSocket connection, each response.create is a request of its own: it is recorded with its usage and cost and checked against these limits, and a refused one is answered with response.failed while the connection stays open. A Realtime connection (/v1/realtime) is one request: it is checked against the limits when it opens, and the tokens and cost of all its answers count when it closes.

clients:
  - name: build-server
    key: tw-q8r2s4t6u8v2w4x6y8z2a4b6
    limits:
      - { per: minute, requests: 30 }
      - { per: day, cost: 5 }
      - { per: month, tokens: 20000000, cache_reads: true }

providers#

Upstreams: the APIs requests are forwarded to.

A credential is one of four things: key, which goes in the header the protocol expects; oauth, a token obtained from a refresh token; aws, the access keys a Bedrock upstream signs its requests with; or headers, when the upstream wants something of its own. headers can be combined with the others, except for the header that already carries the credential.

providers:
  - name: anthropic
    base_url: https://api.anthropic.com
    key: ${ANTHROPIC_API_KEY}

  - name: relay
    base_url: https://relay.example.com/v1
    protocol: openai-chat
    headers:
      X-Relay-Token: ${RELAY_TOKEN}
    proxy: office
    models_only: [gpt-4.1*, o3]
    pricing: relay-discount
    max_concurrent: 4

  - name: local
    base_url: http://127.0.0.1:11434/v1
    protocol: openai-chat
    billing: free

A request carries the request itself and the headers its upstream needs, and nothing else from the client: the credential and the headers written in headers; ThinkWatch’s own User-Agent; and, from the client’s request, only the headers the upstream’s protocol uses (anthropic-* for Anthropic, Idempotency-Key and X-Client-Request-Id for OpenAI, none for Gemini). Identity fields that clients fill in themselves, such as Claude Code’s metadata.user_id, are removed from the body. For an upstream that admits only certain clients, turn on forward_client_identity.

A ChatGPT account upstream (protocol: chatgpt) takes only the credential the desktop app obtains by signing in; it cannot be written by hand. Claude and Google subscription sign-ins are not supported; use an API key.

Some relays and accounts accept only a few requests at a time and refuse the rest. max_concurrent keeps the gateway within that number: a request takes a slot on the upstream when it is sent, and gives it back when the answer has been passed on in full or the client has gone. When the upstream is full, a conversation that stays on it to reuse its prompt cache waits for a slot; any other request goes straight to the next upstream. How long a request waits is failover.slot_wait_secs. Waiting is not a failure: the upstream is not set aside. Requests that only count tokens do not take a slot. On a Responses WebSocket connection, each response.create takes a slot from when it is sent until its answer ends, and so does a key’s max_concurrent; an idle connection takes none.

providers[].oauth#

The access token goes in the protocol’s authorization header. To put it somewhere else, write the header in headers with {{access_token}} where the token goes:

    oauth:
      refresh: ${VENDOR_REFRESH_TOKEN}
      endpoint: https://auth.example.com/oauth/token
      client_id: my-client
    headers:
      X-Access: Token {{access_token}}

providers[].aws#

A Bedrock upstream authenticates in one of two ways. A Bedrock API key goes in key and is sent as Authorization: Bearer. AWS access keys go in aws, written there or read with aws.profile from a profile in the AWS credential files: every request is signed with them (SigV4) once its body is final, and the keys themselves are never sent. Keys written in the configuration can be read from the environment with ${VAR}. A profile is read from the files on the machine core runs on, and read again when they change, so a tool that refreshes temporary keys in ~/.aws/credentials needs no restart. Nothing runs a command to obtain a credential, so a profile that signs in through IAM Identity Center (aws sso login), runs a credential_process or assumes a role cannot be used; export the keys it produces instead.

providers:
  - name: bedrock
    base_url: https://bedrock-runtime.us-east-1.amazonaws.com
    key: ${AWS_BEARER_TOKEN_BEDROCK}

  - name: bedrock-keys
    base_url: https://bedrock-runtime.eu-west-1.amazonaws.com
    aws:
      access_key_id: ${AWS_ACCESS_KEY_ID}
      secret_access_key: ${AWS_SECRET_ACCESS_KEY}
      session_token: ${AWS_SESSION_TOKEN}

  - name: bedrock-profile
    base_url: https://bedrock-runtime.us-west-2.amazonaws.com
    aws:
      profile: dev

Requests are converted to Converse. The model list comes from the region’s control plane: the foundation models that can be invoked on demand, the inference profiles AWS defines (us.anthropic.claude-…), and the account’s application inference profiles, listed by the ARN they are invoked by. Listing needs bedrock:ListFoundationModels and bedrock:ListInferenceProfiles; without them requests are still forwarded, and models can list the models by hand. For a VPC endpoint or a proxy, write its address in base_url and the region in aws.region; the model list is asked of that address too.

providers[].model_specs#

A model’s context window and output limit come from the price table. A relay’s own models are often missing from it, and now and then it is wrong. Write the numbers here, for this upstream and by the exact id in its model list. A value written here takes precedence over the price table; one left out still comes from it. At least one of the two is written, and neither can be 0.

The same numbers are used everywhere: in /v1/models for every client format, for an alias this upstream serves, when the gateway judges whether a conversation still fits the model it is on, and as the output limit of a request converted to Anthropic that does not set one. When several upstreams offer the same model, /v1/models describes it by the first of them in providers.

providers:
  - name: relay
    base_url: https://relay.example.com/v1
    protocol: openai-chat
    model_specs:
      glm-5-air: { context_window: 128000, max_output_tokens: 16384 }
      claude-sonnet-4-5: { context_window: 1000000 }

proxies#

Outbound proxies. Different upstreams often need different ones, so there is no global switch: an upstream picks one with proxy.

proxies[].auth#

proxies:
  - name: office
    type: http
    addr: proxy.example.com:3128
    auth:
      user: alice
      pass: ${PROXY_PASSWORD}

on_proxy_fail: fail is the default because falling back silently sends a request by a path you did not intend; you would believe you were on the proxy while you were not.

pricing#

The cost of a request is its usage times the price of the model. Prices come from the default price table (LiteLLM’s public dataset, a copy of which is built into the binary and refreshed daily) or from a price sheet an upstream picks. A change of price applies to requests from then on, never to ones already recorded.

pricing.sheets#

pricing.sheets[].models#

Prices are in US dollars per million tokens, as printed on vendors’ price pages. Every field is written out; nothing is inferred when pricing.

pricing:
  sheets:
    - name: relay-discount
      multiplier: 0.8
      models:
        claude-sonnet-4-5-thinking:
          input: 3
          output: 15
          cache_read: 0.3
          cache_write_5m: 3.75
          cache_write_1h: 6

client_probes#

Some requests clients send are not the user’s: connectivity checks, warm-ups, session titles, topic detection, suggestions. Each class is either answered locally (intercept, nothing is sent upstream) or forwarded (forward). Forwarded ones go through the routing rules like any other request and carry their class, so a rule can catch them with when.intent. The defaults intercept only what nobody would miss.

security#

Three guards, applied to every upstream alike. Each has a mode: off, observe (detect and record, change nothing) or enforce (act). All three start in observe. What enforce does differs per guard: redaction replaces, tool-call inspection cuts the response off, and the content filter does what each rule says (refuse, delete or record).

security.redact#

Before a request leaves, the whole request (system prompt, earlier answers and tool calls included) is searched for credentials and personal information. Under enforce what is found is replaced with placeholders, and put back where the answer repeats them. Images, files and other base64 payloads are not searched.

Built-in rules:

cn-resident-id, bank-card, email and cn-mobile-phone look for personal information rather than credentials. The first two are on out of the box and match only what checks out by structure:

• cn-resident-id: an 18-character resident ID number of the People’s Republic of China whose first two digits are a province-level code, whose date of birth is a real date between 1900-01-01 and today, and whose last character is the right check character (ISO 7064 MOD 11-2). The old 15-digit numbers are not matched.
• bank-card: a card number whose prefix and length belong to UnionPay, Visa, Mastercard, American Express, JCB, Discover or Diners Club and which passes the Luhn check, written as one run of digits or in groups of four separated by single spaces or single hyphens (American Express also 4-6-5, Diners Club also 4-6-4). The test card numbers published by Stripe, Braintree and Adyen are not matched.

email and cn-mobile-phone are off out of the box: they have no structure to check, and code and documents are full of things that look like them.

• email: an address whose domain has at least two parts, the last of them two or more letters. User names in URLs (https://user@host) and file names such as icon@2x.png are not matched.
• cn-mobile-phone: a Chinese mainland mobile number, 11 digits starting with 1 and a second digit from 3 to 9, not part of a longer run of digits.

A number that is part of a longer run of letters or digits is not matched, and neither is one written as a JSON number in the request body (in a tool call’s arguments, for instance), since replacing it would leave the body invalid JSON. The placeholders of these rules say what was there (<<TW_ID_NUMBER_1>>, <<TW_CARD_NUMBER_1>>, <<TW_EMAIL_1>>, <<TW_PHONE_1>>), and the security log shows only the last four characters of a number, and the first character and the domain of an email address. A custom rule replaces with <<TW_SECRET_1>> unless it names its own placeholder (label).

security.inspect_tools#

Tool calls a model returns are checked against the rules. Under enforce, a match with rules set to cut stops the response, so the client never receives a complete call to run.

Built-in rules:

security.content#

Words, patterns or characters in what the caller sends: user messages and the tool results in them, not the system prompt or the model’s own turns. Each rule matches a keyword (contains), a regular expression (regex) or code points (codepoints), and says what happens under enforce: block refuses the request, strip deletes every match from the caller’s text and sends the rest, record only records it. After deleting, the text is checked again, so a keyword split by hidden characters is caught once they are gone.

Built-in rules:

The invisible group matches characters people cannot see and models can read. Unicode tag characters (U+E0000–U+E007F) and bidirectional controls (U+202A–U+202E, U+2066–U+2069) are on out of the box; zero-width characters (U+200B–U+200D, U+2060, U+FEFF) and private-use characters (U+E000–U+F8FF, U+F0000–U+FFFFD, U+100000–U+10FFFD) are off, since emoji, Persian and icon fonts use them too. All four delete what they match under enforce.

security:
  redact:
    mode: enforce
    enable: [internal-ip]
    custom:
      - name: employee-id
        pattern: 'EMP-\d{6}'
        label: EMPLOYEE
  inspect_tools:
    mode: enforce
  content:
    mode: enforce
    enable: [zero-width]
    custom:
      - name: project-x
        pattern: project-x
        action: strip

retention#

Two limits, because the two kinds of data differ in size by three orders of magnitude: request bodies are tens of kilobytes each, a request’s record a few hundred bytes. The byte limit covers bursts.

Each request and response body is kept up to 4 MiB; of a longer one, the beginning is kept. Bodies are written with credentials and personal numbers already taken out. A request stored under enforce carries the placeholders the upstream received; anything else the redaction rules (security.redact) recognize is masked, in every mode, off included.

failover#

An upstream that fails is set aside for a while, so that the next requests go straight to the next candidate. How long depends on the reason the upstream gives: an insufficient balance waits for a top-up, a used-up quota waits until the moment the upstream says it resets, and a rate limit usually passes within seconds. A request with a single candidate is never affected. On a Responses WebSocket connection, each response.create counts here like a request: one that fails because of the upstream before any content arrives counts as a failure, and so does a connection the upstream refuses or that cannot be made.

Before the first content of a streamed answer reaches the client, an error the upstream sends in the stream moves the request to the next candidate, the same as an error status would.

An upstream can also be slow to start: it accepts the request and then sends nothing for a long time. With next_on_slow_start, the request moves on to the next candidate when no content has arrived stream_start_wait_secs after it was sent. It is off by default, because models that think before they write can take long to start; with it on, wait 30 seconds or more. The last candidate always waits, and the upstream given up on is not set aside. A candidate that is at its max_concurrent at that moment does not count as a next one: the slow upstream keeps the request.

failover:
  stream_start_wait_secs: 30
  next_on_slow_start: true

When upstreams are at their max_concurrent, a request waits for a free slot for at most slot_wait_secs in all. The same time also covers waiting for a key’s minute or hour limit, so a request never waits longer than slot_wait_secs for the two together. An ongoing conversation waits for the upstream it stays on and, if no slot frees in time, moves on to the next one, where its cache starts over. A new conversation skips a full upstream at once. When every candidate is full, the request waits for whichever frees first; if none does, the client gets a 429 with Retry-After saying the upstreams are busy. If an upstream did receive the request and failed, the client gets that failure instead.

aliases#

A model alias is another name for the same model. Clients request it like any model; each upstream receives the name it uses for that model.

aliases:
  deepseek-v4.1: DeepSeek-v4.1-flash
  claude-sonnet-5:
    - claude-sonnet-5
    - us.anthropic.claude-sonnet-5-v1:0
    - anthropic/claude-sonnet-5

• An upstream serves an alias when it offers one of the listed names within its models_only, and receives the first such name in the list. Every upstream that serves an alias takes part in failover for it.
• GET /v1/models lists aliases next to the upstream models, and the original names stay available.
• An alias takes precedence over an upstream model of the same name: a request for claude-sonnet-5 above goes only to upstreams offering one of the three names. List an upstream’s own name when it should keep serving it.
• A key’s allow and a rule’s when.model written for an upstream model name also cover the aliases that list it; written for an alias, they cover only the alias.
• When the upstream answers with the model it was sent, the answer carries the name the client asked for. An answer naming a different model is passed on as it is.
• Prices and the upstream check-up use the name sent upstream. The request log keeps the name the client asked for, and each attempt the name it sent.
• To send one key’s requests to a particular upstream model, use a rule with pinned models (routes[].rules[].to) rather than an alias: an alias changes what the name means for every client.

groups#

A group puts several upstreams behind one name. Rules send requests to a group with to.

fallback is the default because a single user’s machine has no load to spread.

In a load-balance group, a member can carry a weight, from 1 to 100; a member written as just its name has weight 1. Weights set how the group’s requests are shared out: with { name: anthropic, weight: 7 } and relay, the official API serves seven requests in ten. Conversations in progress stay on the upstream that answers them (see below) and count toward its share, so the balance is kept by where new conversations start. An upstream that is cooling down after failures, is at its max_concurrent, or cannot serve a request, sits that request out, and the others share it by their weights. A new WebSocket connection is placed the same way and counts as one request; everything sent on it then goes to the upstream it connected to. Other group types take no weights.

groups:
  - name: pool
    type: load-balance
    providers:
      - { name: anthropic, weight: 7 }
      - relay

Whatever the type, a conversation stays on the upstream that last answered it, so that what the upstream holds of it in its prompt cache is read again rather than paid for in full elsewhere. Within a turn (while the client sends tool results back) it always stays; across turns it stays while the previous answer read or wrote at least 1024 tokens of prompt cache and came less than five minutes ago. An upstream that is cooling down after failures releases the conversation, and whichever upstream answered after a failover is the one it stays on. The rule a turn matched at its start also holds for the rest of that turn: rules keyed on input size or images do not move a turn halfway, unless its input no longer fits the context window of a model the rule sends it to. A load-balance weight is therefore the long-run share of requests: conversations in progress stay where they are and count toward that upstream’s share.

balance_by lets a load-balance group also look at how each upstream has been doing lately. Each member’s weight is multiplied by a factor, and the group shares out requests by the result in the same way as above.

• weights (the default): the weights alone.
• latency: faster upstreams get a larger share. Speed is the typical time from sending a request to the first content of the answer, the same measurement url-test uses. An upstream twice as fast as the middle of the group has its weight multiplied by four, by at most ten and at least a tenth.
• health: upstreams that fail less get a larger share. It looks at the last 50 requests within the past 30 minutes. Server errors, rate limits, used-up quota or balance, rejected credentials, timeouts and connection errors count as failures; errors caused by the request itself do not, and neither does a client that cancels, a switch away from a stream that is slow to start, or an upstream skipped because it is at its max_concurrent. An upstream that keeps failing keeps a twentieth of its weight, so it still gets the occasional request and its recovery is noticed; one that fails outright is set aside by failover as before.
• latency-health: both factors, multiplied.

Speed is measured on streamed answers only, from the moment the request is sent to that upstream, so waiting and upstreams that failed before it do not count. An upstream given up on because its stream was slow to start (failover.next_on_slow_start) counts as taking the whole wait. On a Responses WebSocket connection, each response.create counts as one request for both speed and failures, its speed measured from the moment the upstream starts answering it.

An upstream without enough measurements yet counts as average. As with weights alone, conversations in progress stay where they are, and new conversations make up the difference.

groups:
  - name: pool
    type: load-balance
    balance_by: latency-health
    providers:
      - { name: official, weight: 3 }
      - relay

routes#

A route is a list of rules evaluated top to bottom. Each key takes the route named in its route, otherwise default_route, otherwise the route named default; without any, requests fail over across all upstreams in the order they are declared.

routes[].rules#

routes[].rules[].to#

to names an upstream or a group, or pins models: a list of upstreams, each with the model sent to it, tried in order. A pinned model is sent as written, without aliases or set.model, so a rule can send a key’s requests to one upstream’s model even when an alias of the same name points elsewhere.

routes:
  - name: default
    rules:
      - name: Opus on Bedrock
        when: { model: claude-opus-5 }
        to:
          - { provider: bedrock, model: us.anthropic.claude-opus-5-v1:0 }
          - { provider: anthropic, model: claude-opus-5 }

routes[].rules[].when#

A comparison starts with >, >=, <, <= or ==, and the number may end in k or m: ">200k", "<=4k". Without an operator it is an error, not an equality: "200k" alone is refused.

routes[].rules[].set#

set.model may name an alias; each upstream then receives its own name for it. Answers carry the name the client asked for, as with aliases.

groups:
  - name: fast
    type: url-test
    providers: [anthropic, relay]

routes:
  - name: default
    rules:
      - name: long context goes to the official API
        when: { input_tokens: ">200k" }
        to: anthropic
      - name: titles go to the cheap model
        when: { intent: titling }
        set: { model: claude-haiku-4-5 }
      - name: everything else
        to: fast
default_route: default

plugins#

Script plugins change requests before they reach an upstream and answers before they reach the client. They run in a sandbox inside core, without access to files, the network or the real values of secrets. The app installs them: each plugin’s code goes to plugins/<id>.js next to this file, a copy of the approved code to plugins/.approved/<id>.js, and the code’s SHA-256 to sha256.

A plugin’s file holds its settings too. The manifest at the top of the file says what to do when the plugin fails (on_error), which requests it handles (match) and the value of each setting (settings.<name>.value). When the app changes one of these, it rewrites only the manifest in the file and updates sha256 along with it. This list keeps just the plugin, its approved hash and whether it is on. Entries written by core 0.58 also have on_error, scope and settings: they are ignored, and removed the next time the app changes a plugin.

A plugin runs only while its file has exactly the approved hash. When the file changes on disk or disappears, the plugin stops within seconds and the app shows the change for review. Until the change is approved, the requests the plugin covers are refused (on_error: "reject", the default) or pass without it (on_error: "skip"), as the approved file says. A plugin that does not load is handled the same way. Neither keeps the rest of the configuration from taking effect.

Plugins run in the order of this list.

In match, clients, models and upstreams are lists of names or patterns with * anywhere in them, matched regardless of case; a list that is empty or left out matches everything. clients names the client app (claude-code, codex, …), and a request whose app is not recognised matches only an empty list. models matches the model sent to the upstream: when a routing rule renames the model, the new name is the one that matches. upstreams applies to requests and answers alike.

A plugin changes a request after routing, each time the request is sent to an upstream. A request that fails over to another upstream starts again from what the client sent, and the plugin sees which upstream and which model name the request goes to. Routing, model checks and session grouping use what the client sent. A plugin that changes the model name only renames what is sent to that upstream: the request is not routed again, and the new name must still be one of the models the key may use (clients[].allow), or the request is not sent.

A plugin handles the kinds of request its code declares: conversations (Anthropic Messages, OpenAI Chat Completions and Responses, and Gemini, including their token counts and compaction), embeddings (/v1/embeddings, Gemini :embedContent and :batchEmbedContents) and legacy completions (/v1/completions). A plugin that declares none handles conversations only. Requests of a kind a plugin does not handle pass without it, whatever its on_error. Other endpoints, such as images and audio, pass without any plugin.

plugins:
  - id: add-date
    file: plugins/add-date.js
    sha256: 9f2b6c0e4a1d8f3b7c5e2a9d6f1b4c8e3a7d0f5b2c9e6a1d4f8b3c7e0a5d2f9b
    enabled: true

The start of plugins/add-date.js:

export const manifest = {
  name: "Add date",
  api: 1,
  permissions: ["system"],
  match: { clients: ["claude-code"], models: ["claude-*"], upstreams: [] },
  on_error: "reject",
  settings: {
    note: { type: "string", label: "Note", value: "Answer in English." },
  },
};

Environment variables#