Skip to content
ThinkWatch
Start typing to search the docs.
ThinkWatch Core · Configuration reference

Configuration reference#

ThinkWatch Core reads one file, config.yaml. This page describes every field in it: what it does, its default, the values it takes, and how a change reaches the running process. To run core on a server and control it from the desktop app, see Running core on a server.

The field tables on this page are generated from the code, and a test fails when they disagree, so a field listed here is a field the binary reads.

Where the file is#

PlatformDefault location
macOS, Linux~/.thinkwatch/config.yaml
Windows%APPDATA%\ThinkWatch\config.yaml

THINKWATCH_HOME replaces the directory, and --config <path> names the file for a single command. The directory holds everything else core keeps as well: the request database (data.db), the hourly usage totals behind the overview’s year (usage.db, kept when the request database is rebuilt and not trimmed by the retention period), the backups of the configuration (history/), the downloaded price table (model_prices.json) and the local control socket (twcore.sock; on Windows a loopback port recorded in control.port). The directory is private to its owner (0700), the file is 0600: it holds keys in plain text.

These permissions are the file’s only protection on this machine. They keep out other users, not programs running as the same user: such a program can read every key in the file, and with the control key it holds, change the configuration through the control plane. Outbound redaction does not change that; it protects what a request carries off the machine, not what is on disk. The control key is masked where the configuration is shown so that a write through the control plane cannot change it (listen.control), not to keep it from local programs; upstream and gateway keys are shown as written.

Redaction exceptions (security.redact.exceptions) are stored as a plain, unsalted SHA-256 of each value. That is no secret from anyone who can read the file: a short value such as a phone number can be recovered from its hash by trying every number. The file already holds upstream keys in plain text, so the hash adds nothing to what it gives away; but it travels with exports, sync and backups, so list only values that are not sensitive. The request database holds no hash of any value found: the hashes behind the security log’s “list as exception” are kept in memory for the most recent 20,000 entries and dropped when core restarts.

Tool-call inspection has a built-in rule for this directory, thinkwatch-data. A tool call whose path or command points into one of the default locations above, or into /var/lib/thinkwatch or /etc/thinkwatch on a server, is recorded, and under enforce the response is cut off, so a model cannot be steered into reading these keys or rewriting its own protections. Mentioning the path, as in a document being edited, does not count. With THINKWATCH_HOME elsewhere, a custom rule in security.inspect_tools can cover that path.

twcore serve writes a starting configuration when there is none, and twcore init writes one on request. Both produce this:

version: 1
listen:
  control:
    key: 6629…753d       # generated
clients:
  - name: default
    key: tw-…            # generated

That is a complete, valid configuration. It has no upstream yet, so the control plane runs and requests are answered with an error saying so. Adding one upstream makes it forward:

providers:
  - name: anthropic
    base_url: https://api.anthropic.com
    key: ${ANTHROPIC_API_KEY}

How the file is read#

  • A field name that is not in this reference is an error. A misspelled prot: is not ignored while the gateway quietly starts on the default port; the whole file is refused and the message names the field.
  • Defaults are not written into the file. Anything left out has the default in the tables below. The app and the command line add a field only when its value differs from the default, so what is in the file is what someone chose.
  • ${VAR} reads an environment variable in fields marked “${VAR} allowed”: upstream keys, header values, proxy passwords. It is read from the environment of the core process when a request is sent; an unset variable fails that upstream’s requests and names the variable. Under systemd, that environment is the unit’s EnvironmentFile.
  • Names are references. Rules, groups and keys refer to upstreams, groups, routes and price sheets by name. A name that points nowhere is an error at load time rather than a rule that never matches. Renaming in the app changes every reference in the same write. Names starting with __ are reserved for built-ins.
  • version is the format version, 1. A file with a higher version was written by a newer twcore and is refused.

How a change takes effect#

There are three ways to change the configuration, and they all go through the same path: the desktop app, twcore config …, and editing the file in an editor. Core watches the file and reloads it within a second of a save; nothing needs to be restarted.

A new version is applied only if it passes every stage:

  1. It parses as YAML.
  2. It matches the schema: known fields, the right types.
  3. It is consistent: names are unique, references resolve, patterns compile, CIDR ranges are well formed.
  4. The runtime objects can be built from it.

If any stage fails, the previous configuration stays in service, and the error says which stage failed and where. A typo never takes the gateway down. The desktop app shows the rejection until a valid version is saved.

Changes to listen.gateway apply live as well: core opens the new listener, and if it cannot (the port is taken) it keeps the old one and reports why. Retention changes are applied on the next hourly clean-up.

When two writers edit at once (the app and a hand edit), the second write is refused with a version mismatch instead of overwriting the first.

Backups, export and import#

Every change to the configuration leaves a version in history/ beside the file: a configuration pack (.twconfig) with the plugins and the app’s preferences, recording where the change came from (the app, the command line, an outside edit, an import, a restore, sync). The last 10 are kept; a credential rotation does not make one. The control key is masked in them.

twcore check                      # validate the file without starting anything
twcore config show                # print it, with its version
twcore backup list                # list the versions, newest first
twcore backup restore 1792315800000 --dry-run   # what going back would change
twcore backup restore 1792315800000             # go back to a version
twcore export -o all.twconfig     # the whole configuration as a pack
twcore export --providers relay --no-secrets --encrypt -o relay.twconfig
twcore import all.twconfig --dry-run            # what importing would do
twcore config set /listen/gateway/port 8790 --int

A restore keeps the settings that belong to this machine (listening, the control key, retention, sync), the ChatGPT sign-ins and the gateway keys that clients set up on this machine use. An import adds what is not here and asks, item by item, what to do with what differs (--on-conflict answers for all). A pack carries no settings of this machine and no ChatGPT sign-in; without --no-secrets it holds keys in plain text unless --encrypt is given (the passphrase is asked for, or read from TWCORE_PASSPHRASE). Overwriting an item here with one from a pack without keys keeps this machine’s keys only while the item still points at the same place (scheme, host and port; AWS region; OAuth token endpoint; the outbound proxy it goes through; a proxy’s address); otherwise, and for an item imported under another name, the upstream comes in disabled until its key is entered.

These commands work on the files directly, so they work when core is not running, which is when a restore is most needed; when the file does not load at all, a restore brings the whole version back with the current control key. A running core picks their changes up like any other save.

twcore config set <path> <value> changes one value that is already written in the file. The path names list items by their name (/providers/anthropic/base_url); a number is an index (/routes/0/rules/1/to). The value is a string unless --int, --bool or --null says otherwise. The result is validated before it is written. To add a field that is not in the file yet, edit the file.

Credentials the gateway writes back#

One write is not made by a person. When an upstream’s OAuth token endpoint issues a new refresh token, the old one stops working, so the gateway writes the new token (and the access token with its expiry) back into providers[].oauth. Only those values change; comments and layout are left as they are.

Reference#

Each table lists every field of one section. ”—” in the Default column means the field is simply absent unless written; the description says what that means.

Top level#

FieldTypeDefaultDescription
versionintegerrequiredFormat version of this file. The only version is 1. A file with a higher number was written by a newer twcore and is refused rather than half-understood.
listenobject, listen—Where the gateway and the control channel listen.
clientslist of clients[][]Gateway keys. At least one is required; twcore init and the first twcore serve write one named default.
providerslist of providers[][]Upstreams. None is a valid configuration: the control plane runs and requests are answered with an error saying no upstream is configured.
proxieslist of proxies[][]Outbound proxies, declared once and referred to by name from providers[].proxy.
pricingobject, pricing—Refreshing the default price table, and price sheets of your own.
web_toolsobject, web_tools—Gateway-owned search and public-page tools. Configured under Routing → Auxiliary in Lite.
client_probesobject, client_probes—What happens to the helper requests clients send on their own (health checks, warm-ups, titles).
securityobject, security—The three guards. All of them start in observe, so out of the box nothing is changed or refused.
retentionobject, retention—How long request logs are kept.
failoverobject, failover—How long an upstream is set aside after it fails, and how long the start of a stream is awaited.
aliasesmap of alias → string, list of strings or object{}Model aliases: one name for the same model across upstreams, mapped to the name each upstream uses, in order. A request for an alias goes to any upstream offering one of the listed names, under the first of them it offers. An alias cannot list another alias. The object form accepts models and hide_originals (default false); hiding only affects client discovery, not access.
groupslist of groups[][]Strategy groups: several upstreams behind one name, with a way to pick among them.
routeslist of routes[][]Routes. Without any, requests fail over across all upstreams in the order they are declared.
default_routestring—The route for keys that do not name one. Unset: the route named default, or the built-in failover when there is none.
default_keystring—The gateway key for clients that were not given a key of their own. Unset: the key named default, or the first key. It cannot be disabled.
pluginslist of plugins[][]Script plugins, in the order they run. The app installs them; each one’s code and settings are a file next to this one.
syncobject, sync—Multi-device sync. A local setting: configuration packs and other devices never carry it, and restoring a backup keeps the current one. Unset: sync is off.

listen#

Where core accepts connections. There are two kinds: the AI gateway that clients send requests to, and the control channel the desktop app and twcore commands use.

FieldTypeDefaultDescription
gatewayobject, listen.gateway—The AI gateway: the address clients send requests to.
controlobject, listen.control—The control channel: how the desktop app and twcore commands reach core. It holds the control key, so every configuration has it.

listen.gateway#

FieldTypeDefaultDescription
bindloopback | all | interface name | IP addressloopbackloopback is this machine only; all is every interface; an interface name (en0, eth0) is looked up again every few seconds and follows address changes, and while the interface is not there the gateway listens on 127.0.0.1 only and adds it once it appears; a fixed IP address stops working when the address changes. Binding one interface also listens on 127.0.0.1.
additionallist of strings[]Additional interface names or pinned IP addresses, sharing the gateway port and source allowlist. Loopback stays available. A loopback primary bind disables all additional listeners. An offline interface does not remove other available listeners. Omit or clear this list when selecting loopback only.
portinteger8788TCP port of the gateway.
allow_fromlist of strings[10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fc00::/7]Sources other than this machine that may connect, as CIDR ranges or single addresses. This machine is always allowed. [] means this machine only; 0.0.0.0/0 allows everyone and has to be written out.

When bind reaches beyond this machine, allow_from decides who gets in. The gateway has no TLS: expose it on networks you trust, or put it behind a tunnel or VPN.

listen:
  gateway:
    bind: all
    port: 8788
    allow_from: [192.168.1.0/24]

listen.control#

The control channel is how the desktop app, and twcore config and twcore control-key on the same machine, talk to core. Locally it is a socket file in the data directory (a loopback port on Windows); no network port is opened for it unless remote is enabled.

Every control connection, local or remote, starts with a handshake that proves both ends hold key (Noise_NNpsk0_25519_ChaChaPoly_BLAKE2s: the key is the pre-shared key, each connection negotiates fresh session keys, and the traffic is encrypted). There are no certificates. A peer without the key cannot complete the first message.

FieldTypeDefaultDescription
keystringgeneratedThe control key: 64 hexadecimal characters (32 bytes). Every control connection, local or remote, proves it knows this key. twcore serve writes one before listening if it is missing; a configuration where it is malformed is refused. Show it with twcore control-key, replace it with twcore control-key --rotate.
remoteobject, listen.control.remote—A network port for the desktop app on another machine. Additional to the local channel, never instead of it.

key is written by twcore serve before the control channel starts listening, if it is missing, and by twcore init. It is masked wherever the configuration is shown or kept in a backup; saving a masked value back keeps the real one. A missing or malformed key (not 64 hexadecimal characters) makes the whole file invalid, so it cannot be replaced by a short, guessable one.

twcore control-key            # print the key, to paste into the desktop app
twcore control-key --rotate   # replace it; connected apps have to reconnect

Both run on the machine where core runs.

listen.control.remote#

A network port for a desktop app on another machine. It is opened in addition to the local channel, so a mistake here (a port that is taken, an allow_from that shuts you out) never locks out the machine itself: twcore config and the local app keep working.

FieldTypeDefaultDescription
enabledboolfalseListen on the remote port. Unset or false: no network port is opened for control. twcore remote enable and twcore remote disable switch it; a running core follows within a second.
bindloopback | all | interface name | IP addressallInterface to listen on, written as for listen.gateway.bind.
portintegerrequiredTCP port. There is no fixed default: twcore init and twcore remote enable write a random port between 20000 and 32000 (never the gateway’s) when they write this section. It cannot be 0 or the gateway’s port.
allow_fromlist of strings[10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fc00::/7]Sources that may connect, as for listen.gateway.allow_from, except that this machine is not let in automatically (it has the local channel). A connection from anywhere else is closed before the handshake, without a byte in reply; narrowing the list also closes open connections it no longer allows. A source that fails the handshake 5 times within a minute is ignored for a minute.
listen:
  control:
    key: 9f2c…e41a          # 64 hexadecimal characters
    remote:
      enabled: true
      bind: all
      port: 23483            # random, written when the section is generated
      allow_from: [192.168.1.0/24]

A connection over the remote port cannot do three things, whatever the app asks: shut core down (it is managed by systemd), change listen.control (the door it came in through), or produce a diagnostic bundle (it would be written on the server). Handshakes time out after 5 seconds; five failed handshakes from one source within a minute block that source for a minute.

clients#

Gateway keys: the keys clients such as Claude Code and Codex send to the gateway. A key is an identity. Limits, model scope and route are per key.

FieldTypeDefaultDescription
namestringrequiredName of the key; unique. Routing rules match it with when.client.
keystringrequiredThe key clients send (as x-api-key or Authorization: Bearer). Generated keys start with tw- so they are not mistaken for an upstream’s key. Unique.
max_concurrentinteger—Requests with this key that may run at once; the rest wait. Unset: no limit. 0 is refused.
allowlist of strings—Models this key may use, as model ids or globs (claude-*). An upstream model name also allows the aliases that list it; an alias allows only the alias. Unset: every model. []: none at all.
routestring—Name of the route requests with this key take. Unset: default_route.
clientstring—The client this key was made for (claude-code, codex, …), recorded when the desktop app points a client at the gateway. A client has at most one.
disabledboolfalseRefuse every request made with this key, and keep the key.
limitslist of clients[].limits[]—Usage limits: requests, tokens or cost per minute, hour, day, week or month. A request has to pass every one. Unset: no limit.
clients:
  - name: default
    key: tw-a3f9c8d1e5b2h7k4m6n8p2q4
  - name: build-server
    key: tw-q8r2s4t6u8v2w4x6y8z2a4b6
    max_concurrent: 4
    allow: [claude-sonnet-*]
    route: cheap

clients[].limits#

Usage limits for a key: at most so many requests, tokens or dollars per minute, hour, day, week or month. Each entry counts one of the three; a key can have several, and a request has to pass every one.

minute and hour are rolling: the last 60 seconds, the last 60 minutes. With every, the window spans several of them: { per: hour, every: 5 } is the last 5 hours. When one is used up, a request waits for the next free slot if it frees within failover.wait_secs (120 seconds by default), and is refused otherwise. Any later wait for an upstream comes out of the same time. day, week and month follow the calendar in the time zone of the machine twcore runs on and start again at midnight, on Monday and on the 1st. When one is used up, requests are refused until it starts again. If the machine’s time zone changes, the current day, week and month are added up again from the request records.

A refused request gets HTTP 429 in the client’s own error format, naming the key, the limit, the amount used and when it resets, and it shows in the traffic list. A request that never reaches an upstream counts toward no limit: one refused by a rule, the content filter or a limit, or turned away because every upstream was at its max_concurrent. Cost is what is recorded for each request, so a model without a price and an upstream with billing: free count as $0. A request still running counts with an estimate of its input until it is recorded. After a restart, the day, week and month are added up again from the request records, so the records have to cover the period: retention.row_days of at least 1 for a daily limit, 7 for a weekly one and 31 for a monthly one. Minute and hour limits start empty.

On a Responses WebSocket connection, each response.create is a request of its own: it is recorded with its usage and cost and checked against these limits, and a refused one is answered with response.failed while the connection stays open. A Realtime connection (/v1/realtime) is one request: it is checked against the limits when it opens, and the tokens and cost of all its answers count when it closes.

FieldTypeDefaultDescription
perminute | hour | day | week | monthrequiredThe period. minute and hour are rolling (the last 60 seconds, the last 60 minutes, or every of them); day, week and month start again at local midnight, on Monday and on the 1st.
everyinteger—How many minutes or hours the rolling window spans: {per: hour, every: 5} is the last 5 hours. From 1 to 60 for minute, from 1 to 24 for hour; day, week and month do not take it. Unset: 1.
requestsinteger—At most this many requests. Token counts, answers the gateway gives itself and requests that never reach an upstream do not count.
tokensinteger—At most this many tokens: uncached input, cache writes and output.
costnumber—At most this much, in US dollars, as recorded for each request; at least 0.01. Models without a price and upstreams with billing: free count as 0.
cache_readsboolfalseCount cache reads too. Only for a tokens limit.
clients:
  - name: build-server
    key: tw-q8r2s4t6u8v2w4x6y8z2a4b6
    limits:
      - { per: minute, requests: 30 }
      - { per: day, cost: 5 }
      - { per: month, tokens: 20000000, cache_reads: true }

providers#

Upstreams: the APIs requests are forwarded to.

FieldTypeDefaultDescription
namestringrequiredName of the upstream; unique, and not the name of a group. Names starting with __ are reserved.
base_urlstringrequiredEndpoint, http:// or https://, up to the version segment where the provider documents one (https://api.anthropic.com, https://api.openai.com/v1). For Bedrock, the region’s runtime endpoint: https://bedrock-runtime.<region>.amazonaws.com. No backslashes, and no user name or password before the host (credentials go in key or headers).
keystring, ${VAR} allowed—API key. It goes in the header the protocol expects: x-api-key (Anthropic), Authorization: Bearer (OpenAI, and a Bedrock API key), x-goog-api-key (Gemini). Leave it out for upstreams without a key, or when the credential is written in headers. Cannot be combined with oauth or aws.
headersmap of header name → value{}Additional request headers, in the order written; values may use ${VAR}, and {{access_token}} where oauth is set. At most 32. Headers HTTP or the gateway manages (host, content-length, connection, …) cannot be set.
oauthobject, providers[].oauth—OAuth credential: an access token obtained from a refresh token. Instead of key.
awsobject, providers[].aws—AWS access keys of a Bedrock upstream, written there or read from an AWS profile: every request is signed with them (SigV4). Instead of key, which holds a Bedrock API key.
protocolanthropic | openai-chat | openai-responses | gemini | chatgpt | bedrock—API format of the upstream. Unset: recognized from base_url for the official endpoints (a Bedrock runtime endpoint is bedrock), otherwise treated as anthropic.
reasoning_historyauto | reasoning-content | reasoning | content-blocks | disabledautoRequest-side Chat reasoning history: auto uses verified endpoint defaults, otherwise keeps native content blocks for the same source and the existing reasoning_content field convention. Select reasoning-content, reasoning, content-blocks, or disabled for a compatible gateway. Model-specific settings take precedence. Signed history cannot cross an incompatible source or format.
reasoning_history_groupstring—Explicit compatibility group for signed Chat history. Unset: isolated by upstream, model and configured credentials. Only share a group when the backends accept each other’s signatures; blank is treated as unset.
forward_client_identityboolfalseAlso send the client’s own identity: its User-Agent, identity headers such as x-app and originator, and identity fields in the request body such as metadata.user_id. Values are the client’s, never made up. Off: requests carry ThinkWatch’s User-Agent and no client identity. For upstreams that admit only certain clients (Kimi For Coding, Bailian Coding Plan, relays restricted to official clients). Not available for chatgpt.
proxystringdirectdirect; system, the proxy in the core process’s HTTPS_PROXY, HTTP_PROXY or ALL_PROXY environment variables; or the name of an entry in proxies.
on_proxy_failfail | directfailWhen the proxy cannot be reached: fail the request, or go direct.
modelslist of strings[]Models added by hand, by exact id, for those the upstream serves but leaves out of its list. They count as offered together with the models the upstream lists, or are the whole list when it lists none: they appear in /v1/models and requests for them are routed here. models_only still applies. No wildcards, no duplicates, at most 256 characters each.
models_onlylist of strings—Use only these of the upstream’s models, as ids or globs. Others are not listed and are not routed here. Unset: all of them. Empty is refused; use disabled.
billingper-token | freeper-tokenper-token: cost is usage times the price in the upstream’s price sheet, subscription accounts included. free: cost is recorded as 0.
pricingstring—Name of a price sheet under pricing.sheets. Unset: the default price table.
model_specsmap of model id → providers[].model_specs.*{}Context window, output limit, reasoning and image input of single models of this upstream, written by hand, by exact model id. They take precedence over the price table: for models it does not know, or gets wrong.
max_concurrentinteger—Most requests sent to this upstream at the same time, from 1 to 1000. When it is full, a conversation that stays on it waits for a free slot and other requests go to the next upstream; see failover.wait_secs. Unset: no limit.
min_speedobject, providers[].min_speed—End a streamed answer that generates too slowly: from the first token on, once a second, when the average speed over the last window_secs is below tokens_per_sec, the answer ends with an error clients retry (503, overloaded_error for Anthropic clients), and the upstream counts one failure toward failover.failures_to_pause. Long pauses and summarized reasoning partway through are left out of the average; the idle timeout deals with pauses. Answers that are not streamed are not judged. Unset: never.
limitslist of providers[].limits[]—Usage limits of this upstream, written like its own quota: requests, tokens or cost every few minutes or hours, per day, week or month. Every one has to have room before a request is sent to it. When a rolling window is full, requests go to the next upstream or wait for room; when a day, week or month is used up, requests go to the next upstream until the next one starts. Unset: no limit.
disabledboolfalseTake the upstream out of routing and out of the model list, and keep its configuration.
balanceauto | off | openrouter | deepseek | moonshot | sub2api | newapi | thinkwatchautoWhere the upstream’s balance is read from, with its own key and proxy: openrouter, deepseek, moonshot, sub2api, newapi (New API and One API) or thinkwatch (a ThinkWatch Enterprise gateway). auto: recognized from base_url for OpenRouter, DeepSeek and Moonshot; any other address, except Anthropic’s, OpenAI’s and Gemini’s own, is asked once whether it is a Sub2API, ThinkWatch Enterprise or New API relay, and the answer is kept in memory only. off: no balance. ChatGPT and Z.ai accounts and Bedrock have no balance whatever is written here.
signed_inzai | bigmodel—Which account sign-in the key came from: zai (Z.ai) or bigmodel (BigModel). Signing in to the account in the desktop app writes it; replacing the key there removes it. It does not change how requests are sent. Unset: the key was written by hand.

In auto, unknown Chat-compatible endpoints can also learn a narrowly scoped omission from a structured HTTP 400/422 extra_forbidden validation error for messages[n].reasoning_content or messages[n].reasoning. This does not establish support for an alternate format. The gateway re-encodes from the original history, removes only the rejected unsigned field, applies outbound redaction again, and makes at most one request-schema repair per logical request (shared with effort, automatic cache-breakpoint and sealed-history repairs). Explicit overrides and signed history are never changed by this negotiation. Profiles that require reasoning replay do not negotiate omissions. Ambiguous errors, SSE errors, timeouts and failures after generation starts do not initiate negotiation.

Only an accepted adjusted request is remembered. The in-memory cache is bounded at 1,024 entries, expires after 24 hours, is isolated by upstream/resolved credentials, model, actual API path and tool/thinking mode, and resets on configuration reload or restart. These runtime observations are internal and require no user settings. Explicit upstream/model overrides remain available in advanced configuration.

A credential is one of four things: key, which goes in the header the protocol expects; oauth, a token obtained from a refresh token; aws, the access keys a Bedrock upstream signs its requests with; or headers, when the upstream wants something of its own. headers can be combined with the others, except for the header that already carries the credential.

providers:
  - name: anthropic
    base_url: https://api.anthropic.com
    key: ${ANTHROPIC_API_KEY}

  - name: relay
    base_url: https://relay.example.com/v1
    protocol: openai-chat
    headers:
      X-Relay-Token: ${RELAY_TOKEN}
    proxy: office
    models_only: [gpt-4.1*, o3]
    pricing: relay-discount
    max_concurrent: 4

  - name: local
    base_url: http://127.0.0.1:11434/v1
    protocol: openai-chat
    billing: free

An upstream’s models are the ones it lists on /v1/models (for Bedrock, the region’s model list) and the ones written in models. Upstreams often leave models out of their list: a relay lists only some of what it serves, an account backend hides new models from older clients. A model added in models is listed in /v1/models, can be named by an alias, pinned in a rule and allowed for a key, and is routed to this upstream like a listed one; models_only applies to both. An upstream that lists nothing has exactly the models in models; one with neither lists no model and is still sent requests for any model.

A request carries the request itself and the headers its upstream needs, and nothing else from the client: the credential and the headers written in headers; ThinkWatch’s own User-Agent; and, from the client’s request, only the headers the upstream’s protocol uses (anthropic-* for Anthropic, Idempotency-Key and X-Client-Request-Id for OpenAI, none for Gemini). Identity fields that clients fill in themselves, such as Claude Code’s metadata.user_id, are removed from the body. For an upstream that admits only certain clients, turn on forward_client_identity.

A request converted to Anthropic, or to Claude on Bedrock, marks where the upstream may cache the prompt when the client marked nothing itself. Clients in OpenAI or Gemini formats such as Codex cannot mark anything: those providers cache a repeated prompt on their own, while Anthropic caches only what is marked. The marks go at the end of the tools, at the end of the system prompt and at the end of the last two user turns, at most four, each kept for the default five minutes. The earlier of the two user marks is where the previous request ended, so each turn reads back what the turn before wrote and pays the cache price for it instead of the full input price. Cache writes and reads are charged at the price table’s cache prices. A request that carries its own marks, as Claude Code’s do, keeps exactly those, and a request sent on in the upstream’s own format is not changed.

On Bedrock, marks are added only for the Claude models AWS lists as supporting prompt caching: Claude 3.7 Sonnet, Claude 3.5 Sonnet v2, and every Claude from version 4.5 on, including newer ones not yet listed. Older models, such as Claude 3 Haiku, Sonnet 4 and Opus 4.1, get none. An upstream that refuses the marks is sent the request once more without them, and that upstream is not sent marks for that model again until the core restarts.

A ChatGPT account upstream (protocol: chatgpt) takes only the credential the desktop app obtains by signing in; it cannot be written by hand. Claude and Google subscription sign-ins are not supported; use an API key.

Generation requests first use the client’s streaming or non-streaming mode. Known backend requirements take priority: ChatGPT account backends keep their existing stream-only behavior. For other backends, an HTTP 400, 422 or 501 JSON API rejection can trigger one trial of the opposite mode on the same upstream. No particular mode-error code or English message is required. Known history, reasoning-effort and cache-mark repairs run first and share the request’s single automatic repair budget; a trial also respects its wait deadline. Explicit errors about another parameter, recognizable authentication, quota, rate-limit, overload or timeout errors, and responses carrying a retry hint do not trigger a trial. An unrelated opaque request error can therefore cost one extra rejected attempt.

The gateway collects a streaming answer into complete JSON, or writes a complete answer as the stream the client requested. The latter still waits for the whole upstream answer. A successful response already in the other mode is converted without resending it. If the trial is itself rejected over HTTP, or fails before its answer starts, the original error is retained; both attempt outcomes remain in the request record. An original server error can still fail over to another upstream. Partially received answers are never replayed, and errors inside a successful stream do not initiate mode negotiation.

Only a complete successful answer teaches a working preference. It is kept in memory for 30 minutes from that trial, isolated by upstream account, model, API and request options such as tools and reasoning. Using the preference does not extend its expiry: after expiry the next request tries the client’s mode again. Reloading the configuration clears this memory; an eligible rejection invalidates a remembered preference. No upstream setting is required. Native requests for multiple candidates, log probabilities or non-text output are not switched automatically: the current response converters cannot preserve those response fields. WebSocket requests are not negotiated.

Some relays and accounts accept only a few requests at a time and refuse the rest. max_concurrent keeps the gateway within that number: a request takes a slot on the upstream when it is sent, and gives it back when the answer has been passed on in full or the client has gone. When the upstream is full, a conversation that stays on it to reuse its prompt cache waits for a slot; any other request goes straight to the next upstream. How long a request waits is failover.wait_secs. Waiting is not a failure: the upstream is not set aside. Requests that only count tokens do not take a slot. On a Responses WebSocket connection, each response.create takes a slot from when it is sent until its answer ends, and so does a key’s max_concurrent; an idle connection takes none.

providers[].min_speed#

A self-hosted model sometimes drops to a few tokens per second and stays there. Content keeps arriving, so the idle timeout never fires, and retrying usually brings the speed back. With min_speed, a streamed answer is judged once a second from its first token on: when the average speed over the last window_secs seconds is below tokens_per_sec, the gateway ends the answer with an error the client can retry (503: overloaded_error for Anthropic clients, response.failed with server_error for Responses clients, UNAVAILABLE for Gemini clients) and closes the connection to the upstream.

The speed is counted the way the live speed in the app is: text, reasoning and tool arguments, estimated and calibrated against what the upstream reports. Pauses of 5 seconds or more partway through (a quarter of the window when it is under 20 seconds) are left out, since the idle timeout deals with them; when what is left of the window is under half of it, nothing is judged that second. Nothing is judged while the stream is paused (no content for more than a second): tool arguments often arrive in one burst after a pause. Reasoning that arrives as a summary, a few sentences every few seconds while the model thinks fast (Anthropic, Gemini and Bedrock reasoning, the Responses reasoning summary), is not judged while it lasts and its time is left out afterwards; reasoning streamed as it is generated (reasoning_content in OpenAI Chat, the Responses reasoning_text) is judged like text. The wait before the first token is not judged, nor is anything after the answer has ended, nor are answers that are not streamed, including Gemini streams requested without alt=sse.

The request is recorded as failed with the speed it had, the part already generated is billed, and the upstream counts one failure toward failover.failures_to_pause, so a retry goes to another upstream first when there is one.

FieldTypeDefaultDescription
tokens_per_secintegerrequiredThe lowest acceptable speed, in tokens per second, counted the way the live speed in the app is. From 1 to 1000.
window_secsinteger20How many seconds the average is taken over. The first one is taken this long after the first token. From 5 to 120.
providers:
  - name: local-vllm
    base_url: http://127.0.0.1:8000/v1
    min_speed: { tokens_per_sec: 15, window_secs: 20 }

providers[].limits#

Usage limits for an upstream, written like the quota the upstream itself enforces: at most so many requests, tokens or dollars every few minutes or hours, per day, week or month. The entries are written like a key’s limits; an upstream can have several, and every one has to have room before a request is sent to it.

Each attempt sent to the upstream counts as a request, whether it succeeds or fails, unless the upstream answers it with 429. Tokens and cost count for the attempt that is answered. Requests that only count tokens do not count. An attempt still running counts with an estimate of its input until it is recorded.

When a minute or hour window is full, the upstream is passed over like a busy one: a conversation that stays on it waits for room, other requests go to the next upstream, and when no candidate has room a request waits for the first one that does, within the time a request may wait (see failover). When a day, week or month is used up, requests go to the next upstream until the period starts again; none waits for it. Being full is not a failure: the upstream is not set aside. After a restart the day, week and month are added up again from the request records, so retention.row_days has to cover the period, as for a key; minute and hour windows start empty.

FieldTypeDefaultDescription
perminute | hour | day | week | monthrequiredThe period. minute and hour are rolling (the last 60 seconds, the last 60 minutes, or every of them); day, week and month start again at local midnight, on Monday and on the 1st.
everyinteger—How many minutes or hours the rolling window spans: {per: hour, every: 5} is the last 5 hours. From 1 to 60 for minute, from 1 to 24 for hour; day, week and month do not take it. Unset: 1.
requestsinteger—At most this many requests sent to this upstream. Each attempt that reaches it counts, unless it answers 429; token counts do not count.
tokensinteger—At most this many tokens in the answers of this upstream: uncached input, cache writes and output.
costnumber—At most this much, in US dollars, as recorded for the requests this upstream answers; at least 0.01. Models without a price count as 0, and so does everything with billing: free.
cache_readsboolfalseCount cache reads too. Only for a tokens limit.
providers:
  - name: relay
    base_url: https://relay.example.com
    key: sk-relay
    limits:
      - { per: hour, every: 5, requests: 200 }
      - { per: week, cost: 50 }

providers[].oauth#

FieldTypeDefaultDescription
accessstring—Current access token. Written back by the gateway after every refresh; unset means one is obtained on first use.
expires_atstring—When access expires, RFC 3339 in UTC. Written back with it. Unset: used until the upstream answers 401.
refreshstringrequiredRefresh token. When the token endpoint issues a new one, the old one stops working, so the gateway writes the new one back into this file.
endpointstringrequiredToken endpoint URL.
client_idstring—OAuth client id, if the endpoint wants one.
client_secretstring—OAuth client secret, if the endpoint wants one.
refresh_beforeduration (30s, 5m, 1h)—How long before expiry to refresh. Unset or unreadable: 5m.

The access token goes in the protocol’s authorization header. To put it somewhere else, write the header in headers with {{access_token}} where the token goes:

    oauth:
      refresh: ${VENDOR_REFRESH_TOKEN}
      endpoint: https://auth.example.com/oauth/token
      client_id: my-client
    headers:
      X-Access: Token {{access_token}}

providers[].aws#

FieldTypeDefaultDescription
access_key_idstring, ${VAR} allowed—Access key ID. Written together with secret_access_key; profile instead.
secret_access_keystring, ${VAR} allowed—Secret access key.
session_tokenstring, ${VAR} allowed—Session token of temporary credentials, such as those STS issues. When it expires, requests are refused until it is replaced.
profilestring—Profile in the AWS credential files to read the access keys from, instead of writing them here: ~/.aws/credentials and ~/.aws/config, or the files AWS_SHARED_CREDENTIALS_FILE and AWS_CONFIG_FILE name, on the machine core runs on. The files are read again when they change.
regionstring—Region to sign for. Unset: the one in base_url, which must then be a standard runtime endpoint. Required when base_url is a VPC endpoint or a proxy.

A Bedrock upstream authenticates in one of two ways. A Bedrock API key goes in key and is sent as Authorization: Bearer. AWS access keys go in aws, written there or read with aws.profile from a profile in the AWS credential files: every request is signed with them (SigV4) once its body is final, and the keys themselves are never sent. Keys written in the configuration can be read from the environment with ${VAR}. A profile is read from the files on the machine core runs on, and read again when they change, so a tool that refreshes temporary keys in ~/.aws/credentials needs no restart. Nothing runs a command to obtain a credential, so a profile that signs in through IAM Identity Center (aws sso login), runs a credential_process or assumes a role cannot be used; export the keys it produces instead.

providers:
  - name: bedrock
    base_url: https://bedrock-runtime.us-east-1.amazonaws.com
    key: ${AWS_BEARER_TOKEN_BEDROCK}

  - name: bedrock-keys
    base_url: https://bedrock-runtime.eu-west-1.amazonaws.com
    aws:
      access_key_id: ${AWS_ACCESS_KEY_ID}
      secret_access_key: ${AWS_SECRET_ACCESS_KEY}
      session_token: ${AWS_SESSION_TOKEN}

  - name: bedrock-profile
    base_url: https://bedrock-runtime.us-west-2.amazonaws.com
    aws:
      profile: dev

Requests are converted to Converse. The model list comes from the region’s control plane: the foundation models that can be invoked on demand, the inference profiles AWS defines (us.anthropic.claude-…), and the account’s application inference profiles, listed by the ARN they are invoked by. Listing needs bedrock:ListFoundationModels and bedrock:ListInferenceProfiles; without them requests are still forwarded, and models can list the models by hand. For a VPC endpoint or a proxy, write its address in base_url and the region in aws.region; the model list is asked of that address too.

providers[].model_specs#

Model discovery uses upstream-reported capabilities automatically, with the price table supplying known specifications that the upstream omits. Write overrides here for missing or incorrect values, scoped to this upstream and an exact model ID. Each entry must set at least one specification; context and output token counts must be positive.

Manual values also apply when checking context capacity and supplying an omitted Anthropic output limit. For discovery, manual values take precedence over reported metadata, then the price table. Reasoning effort has no price-table fallback: use reasoning_effort to supply its levels and optional default. Nonempty levels imply reasoning support, an empty list explicitly declares no adjustable effort, and removing the override restores automatic discovery. Public effort choices are intersected across all routing candidates and limited to values the required protocol conversions can represent. See Codex model discovery.

FieldTypeDefaultDescription
reasoning_historyauto | reasoning-content | reasoning | content-blocks | disabled—Override this model’s request-side reasoning-history format. Unset: use the upstream setting. This is independent of whether the model can reason.
context_windowinteger—Context window: the most tokens a request can take in. Unset: the price table’s.
max_output_tokensinteger—The most tokens an answer can have. Unset: the price table’s.
reasoningbool—Whether the model reasons. This boolean does not declare adjustable effort levels; use reasoning_effort for those. False conflicts with nonempty manual levels. Unset: use known metadata or the price table.
reasoning_effortobject, providers[].model_specs.*.reasoning_effort—Override the entire reasoning effort profile. Unset: automatic upstream metadata; an empty supported_levels list explicitly disables adjustable effort. Nonempty levels imply reasoning support.
image_inputbool—Whether the model takes images as input. The model list carries it, as input_modalities. Unset: the price table’s; the list leaves it out when the price table does not say.

FieldTypeDefaultDescription
supported_levelslist of stringsrequiredSupported levels: minimal, low, medium, high, xhigh, max, ultra. No duplicates. Empty explicitly means no adjustable effort; protocol conversion may further restrict these choices.
default_levelminimal | low | medium | high | xhigh | max | ultra—Must be one of supported_levels. Unset: prefer medium in the final available set, otherwise its lowest level. The manual profile does not inherit the upstream default.
providers:
  - name: relay
    base_url: https://relay.example.com/v1
    protocol: openai-chat
    model_specs:
      glm-5-air: { context_window: 128000, max_output_tokens: 16384 }
      claude-sonnet-4-5: { context_window: 1000000 }

proxies#

Outbound proxies. Different upstreams often need different ones, so there is no global switch: an upstream picks one with proxy.

FieldTypeDefaultDescription
namestringrequiredName used in providers[].proxy. direct and system are built in.
typesocks5 | socks5h | http | httpssocks5hsocks5h sends the host name to the proxy to resolve; socks5 resolves it locally first. http and https are HTTP proxies.
addrstringrequiredhost:port of the proxy.
authobject, proxies[].auth—User name and password, if the proxy wants them.

proxies[].auth#

FieldTypeDefaultDescription
userstringrequiredUser name.
passstring, ${VAR} allowedrequiredPassword.
proxies:
  - name: office
    type: http
    addr: proxy.example.com:3128
    auth:
      user: alice
      pass: ${PROXY_PASSWORD}

on_proxy_fail: fail is the default because falling back silently sends a request by a path you did not intend; you would believe you were on the proxy while you were not.

pricing#

The cost of a request is its usage times the price of the model. Prices come from the default price table (LiteLLM’s public dataset, a copy of which is built into the binary and refreshed daily) or from a price sheet an upstream picks. A change of price applies to requests from then on, never to ones already recorded.

FieldTypeDefaultDescription
auto_updatebooltrueRefresh the default price table from the network once a day. It is saved as model_prices.json beside config.yaml; the table built into the binary is used until then and when offline.
sheetslist of pricing.sheets[][]Price sheets of your own. An upstream uses one with providers[].pricing.

pricing.sheets#

FieldTypeDefaultDescription
namestringrequiredName of the sheet; unique.
multipliernumber1Applied to every price of the default table, cache and long-context prices included.
modelsmap of model id → pricing.sheets[].models.*{}Prices for single models. They replace the default table’s price for that model and are not multiplied.

pricing.sheets[].models#

Prices are in US dollars per million tokens, as printed on vendors’ price pages. Every field is written out; nothing is inferred when pricing.

FieldTypeDefaultDescription
inputnumberrequiredUS dollars per million input tokens.
outputnumberrequiredUS dollars per million output tokens.
cache_readnumberrequiredUS dollars per million tokens read from the prompt cache.
cache_write_5mnumberrequiredUS dollars per million tokens written to a 5-minute cache.
cache_write_1hnumberrequiredUS dollars per million tokens written to a 1-hour cache.
input_above_200knumber—Input price once a request’s input, cache reads and writes included, exceeds 200K tokens. Written together with output_above_200k, or neither. Cache prices stay the ones above.
output_above_200knumber—Output price once a request’s input, cache reads and writes included, exceeds 200K tokens.
pricing:
  sheets:
    - name: relay-discount
      multiplier: 0.8
      models:
        claude-sonnet-4-5-thinking:
          input: 3
          output: 15
          cache_read: 0.3
          cache_write_5m: 3.75
          cache_write_1h: 6

client_probes#

Some requests clients send are not the user’s: connectivity checks, warm-ups, session titles, topic detection, suggestions. Each class is either answered locally (intercept, nothing is sent upstream) or forwarded (forward). Forwarded ones go through the routing rules like any other request and carry their class, so a rule can catch them with when.intent. The defaults intercept only what nobody would miss.

FieldTypeDefaultDescription
health_checkintercept | forwardinterceptConnectivity checks (max_tokens: 1). Answered locally by default: nothing is lost.
warmupintercept | forwardinterceptWarm-up requests. Answered locally by default.
titlingintercept | forwardforwardRequests that name a session. Forwarded by default: intercepting them gives every session the same title.
topic_detectintercept | forwardforwardTopic detection. Forwarded by default.
suggestionintercept | forwardforwardSuggestions. Forwarded by default.

security#

Three guards, applied to every upstream alike. Each has a mode: off, observe (detect and record, change nothing) or enforce (handle each match by its rule’s action). All three start in observe. Every rule has an action: redaction replaces with a placeholder or only records, tool-call inspection cuts the response off or only records, and the content filter refuses, deletes, replaces or only records.

FieldTypeDefaultDescription
redactobject, security.redact—Outbound redaction (Secrets & privacy in the app): credentials and personal information in the text a request carries (messages, system prompts, reasoning, tool arguments and results, tool descriptions) are replaced with placeholders before it leaves, or only recorded, rule by rule. Identifiers, model names and other protocol fields are sent as they are.
inspect_toolsobject, security.inspect_tools—Tool-call inspection (Dangerous commands in the app): dangerous commands in the tool calls a model returns cut the response off.
contentobject, security.content—Content filter: words, patterns or characters (hidden ones among them) in what the caller sends, and words or patterns in the text of the answer; each rule refuses the request, deletes or replaces what it matched, or only records it.

security.redact#

Before a request leaves, the text written in it is searched for credentials and personal information: system prompts, the messages of every role (earlier answers included), reasoning, tool-call arguments, tool results and tool descriptions. Under enforce what a rule finds is replaced with placeholders, and put back where the answer repeats them; what a rule with action: record finds is sent as it is and only recorded. Values the protocol needs as they are, such as ids, types, roles, tool names, model names and signatures, are never searched, nor are field names, images, files, settings, or parts of a request the gateway does not recognize as written text.

FieldTypeDefaultDescription
modeoff | observe | enforceobserveoff does nothing (Off in the app); observe detects and records only, and changes nothing (Monitor); enforce detects and handles each match by its rule’s action (On).
enablelist of strings[]Built-in rules to switch on that are off out of the box, by id.
disablelist of strings[]Built-in rules to switch off, by id.
actionsmap of built-in rule id → placeholder | record{}What a built-in rule does under enforce, written only where it differs from the factory setting (email: record). placeholder sends a placeholder in place of the value and puts the value back where the answer repeats the placeholder; record sends the value as it is and records the match.
customlist of security.redact.custom[][]Rules of your own: whatever a rule matches is replaced like a credential, or only recorded.
exceptionslist of security.redact.exceptions[][]Values that are never replaced or recorded, under any rule. Each is kept as a hash, never as the value itself; at most 500. A saved secret cannot be an exception: an entry whose rule is thinkwatch-saved-secret is read but has no effect.

FieldTypeDefaultDescription
namestringrequiredName shown in logs and in the app; it identifies the rule and has to be unique within this guard.
patternstringrequiredThe text to look for; a regular expression with match: regex.
matchcontains | regexcontainscontains: pattern is looked for as written; characters such as . and * have no special meaning. regex: pattern is a regular expression.
case_sensitiveboolfalseTell capital letters from small ones. Off by default: ABC and abc both match.
labelstringSECRETPlaceholder name: what the rule matches is replaced with <<TW_label_1>>, numbered per name. Capital letters, digits and underscores, starting with a letter, at most 24 characters.
actionplaceholder | recordplaceholderUnder enforce: replace what the rule matches with a placeholder, or only record the match and send it as it is.
disabledboolfalseSwitches the rule off and keeps it in the file.

Built-in rules:

idNameOut of the box
anthropic-api-keyAnthropic API keyon
openai-project-keyOpenAI project keyon
openai-api-keyOpenAI API keyon
github-personal-tokenGitHub personal access tokenon
github-oauth-tokenGitHub OAuth tokenon
github-server-tokenGitHub server tokenon
github-user-tokenGitHub user tokenon
github-fine-grained-tokenGitHub fine-grained tokenon
slack-bot-tokenSlack bot tokenon
slack-user-tokenSlack user tokenon
slack-app-tokenSlack app tokenon
aws-access-key-idAWS access key IDon
aws-temporary-key-idAWS temporary access key IDon
google-api-keyGoogle API keyon
google-oauth-tokenGoogle OAuth tokenon
gitlab-tokenGitLab tokenon
stripe-live-keyStripe live keyon
stripe-restricted-keyStripe restricted keyon
npm-tokennpm tokenon
digitalocean-tokenDigitalOcean tokenon
sendgrid-keySendGrid keyon
private-keyPrivate keyon
jwtJWTon
conn-string-passwordConnection string passwordon
cn-resident-idChinese resident ID numberon
bank-cardBank card numberon
emailEmail addressoff
cn-mobile-phoneChinese mainland mobile numberoff
internal-ipInternal IP addressoff
internal-domainInternal domainoff
thinkwatch-saved-secretCredentials saved in ThinkWatchon
openai-service-keyOpenAI service account or admin keyon
openrouter-keyOpenRouter API keyon
xai-keyxAI API keyon
groq-keyGroq API keyon
huggingface-tokenHugging Face tokenon
replicate-tokenReplicate tokenon
perplexity-keyPerplexity API keyon
bedrock-api-keyAmazon Bedrock API keyon
fireworks-keyFireworks AI API keyon
cerebras-keyCerebras API keyon
langsmith-keyLangSmith API keyon
langfuse-secret-keyLangfuse secret keyon
pinecone-keyPinecone API keyon
tavily-keyTavily API keyon
minimax-keyMiniMax API keyon
zhipu-api-keyZhipu API keyon
qianfan-api-keyBaidu Qianfan API keyon
sk-compat-keyOpenAI-compatible API keyon
github-refresh-tokenGitHub refresh tokenon
gitlab-other-tokenOther GitLab tokenon
pypi-tokenPyPI tokenon
rubygems-tokenRubyGems tokenon
clojars-tokenClojars tokenon
docker-hub-tokenDocker Hub tokenon
artifactory-keyJFrog Artifactory keyon
databricks-tokenDatabricks tokenon
sentry-tokenSentry tokenon
grafana-tokenGrafana tokenon
postman-keyPostman API keyon
aws-other-key-idOther AWS access key IDon
alibaba-access-key-idAlibaba Cloud AccessKey IDon
tencent-secret-idTencent Cloud SecretIdon
volcengine-access-key-idVolcengine AccessKey IDon
azure-ad-client-secretAzure AD client secreton
digitalocean-other-tokenOther DigitalOcean tokenon
cloudflare-origin-ca-keyCloudflare Origin CA keyon
vault-tokenHashiCorp Vault tokenon
terraform-tokenTerraform Cloud tokenon
doppler-tokenDoppler tokenon
pulumi-tokenPulumi tokenon
heroku-keyHeroku API keyon
flyio-tokenFly.io tokenon
supabase-keySupabase keyon
age-secret-keyage secret keyon
onepassword-token1Password service account tokenon
stripe-webhook-secretStripe webhook signing secreton
shopify-tokenShopify tokenon
twilio-api-keyTwilio API keyon
brevo-keyBrevo API keyon
slack-other-tokenOther Slack tokenon
linear-keyLinear API keyon
notion-tokenNotion tokenon
atlassian-tokenAtlassian API tokenon
auth-header-credentialCredential in a request headeron
command-line-passwordPassword on a command lineon
url-tokenToken in a URLon
bot-webhookBot webhookon
telegram-bot-tokenTelegram bot tokenon
config-assignmentSecret assigned in a config fileon
crypto-mnemonicRecovery phraseon
crypto-wif-keyWIF private keyon
crypto-extended-keyExtended private keyon
uuidUUID / compact identifieron
crypto-hex-keyRaw wallet key / 32-byte hashon
crypto-solana-keySolana keypairon
crypto-sui-keySui private keyon
crypto-stellar-keyStellar secret seedon
crypto-xrp-seedXRP family seedon
crypto-addressBlockchain account / extended public keyon
crypto-secret-fieldWallet secret in a fieldon
exchange-credentialTrading API / signed authentication credentialon
crypto-keystoreWallet keystore / ciphertexton
crypto-rpc-keyBlockchain RPC URL credentialon
cn-resident-id-1515-digit legacy resident ID numberoff
passport-mrzPassport machine-readable zoneon
cn-passportChinese passport numberoff
hk-macau-taiwan-idHong Kong, Macau or Taiwan ID numberoff
mainland-travel-permitMainland travel permit numberoff
us-ssnUS Social Security numberoff
uk-ninoUK National Insurance numberoff
sg-nricSingapore NRIC / FINoff
kr-rrnKorean resident registration numberoff
jp-my-numberJapanese My Numberoff
in-aadhaarIndian Aadhaar numberoff
br-cpfBrazilian CPFoff
ibanIBANon
cn-landlineChinese mainland landline numberoff
intl-phoneInternational phone numberoff
cn-addressChinese mainland street addressoff
geo-coordinatesPrecise coordinatesoff
cn-license-plateChinese mainland license plateoff
vinVehicle identification number (VIN)off
imeiIMEIoff
mac-addressMAC addressoff
public-ipPublic IP addressoff
labeled-nameName by field nameoff
labeled-addressAddress by field nameoff
labeled-birth-dateBirth date by field nameoff
labeled-accountBank account by field nameoff
labeled-healthHealth information by field nameoff

uuid is on by default. It matches hexadecimal UUIDs written as 8-4-4-4-12 or 32 consecutive hexadecimal characters, ignoring case and without restricting the UUID version. Order and trace IDs, and compact hashes with the same shape, are replaced too; longer tokens are not split and JSON numbers stay numbers. Observe mode only records matches. A UUID alone does not trigger the credential-to-network tool check; saved credentials are still checked by value.

cn-resident-id, bank-card, email and cn-mobile-phone look for personal information rather than credentials. The first two are on out of the box and match only what checks out by structure:

  • cn-resident-id: an 18-character resident ID number of the People’s Republic of China whose first two digits are a province-level code, whose date of birth is a real date between 1900-01-01 and today, and whose last character is the right check character (ISO 7064 MOD 11-2). The old 15-digit numbers are not matched.
  • bank-card: a card number whose prefix and length belong to UnionPay, Visa, Mastercard, American Express, JCB, Discover or Diners Club and which passes the Luhn check, written as one run of digits or in groups of four separated by single spaces or single hyphens (American Express also 4-6-5, Diners Club also 4-6-4). The test card numbers published by Stripe, Braintree and Adyen are not matched.

email and cn-mobile-phone are off out of the box: they have no structure to check, and code and documents are full of things that look like them.

  • email: an address whose domain has at least two parts, the last of them two or more letters. User names in URLs (https://user@host) and file names such as icon@2x.png are not matched.
  • cn-mobile-phone: a Chinese mainland mobile number, 11 digits starting with 1 and a second digit from 3 to 9, not part of a longer run of digits.

A number that is part of a longer run of letters or digits is not matched, and neither is one written as a JSON number in the request body (in a tool call’s arguments, for instance), since replacing it would leave the body invalid JSON. The placeholders of these rules say what was there (<<TW_ID_NUMBER_1>>, <<TW_CARD_NUMBER_1>>, <<TW_EMAIL_1>>, <<TW_PHONE_1>>), and the security log shows only the last four characters of a number, and the first character and the domain of an email address. A custom rule replaces with <<TW_SECRET_1>> unless it names its own placeholder (label).

Exceptions: a value that a rule keeps matching but that is not sensitive (a test fixture, a sample key in documentation) can be listed under exceptions. The rules keep working; only that value is left alone, under every rule, in both observe and enforce. Each entry holds the SHA-256 of the value, never the value itself, so the list can travel with exports, sync and backups (see the note on the plain SHA-256 under where the file is). The app adds entries from the security log or from a pasted value; written by hand, sha256 is the hash of the value as it appears in the message, with no surrounding quotes or JSON escapes. Only recent log entries can be listed from the log: the hash of each value is kept in memory for the last 20,000 entries and not across a restart, and a value made of overlapping matches joined into one cannot be listed at all. Older entries show neither the button nor the exception tag; paste the value instead.

FieldTypeDefaultDescription
sha256stringrequiredSHA-256 of the value, as 64 lowercase hexadecimal characters. Escapes in the request body are decoded first: the hash is of the value as written.
maskedstring''The value as shown when it was added, masked the same way as in the security log.
rulestring''The rule that first recognised the value: a built-in rule id or a custom rule name. Empty for a value added by hand that no rule recognises.
added_atinteger0When the value was added, in milliseconds since the Unix epoch.

security.inspect_tools#

Tool calls a model returns are checked against the rules. Under enforce, a match with rules set to cut stops the response, so the client never receives a complete call to run.

FieldTypeDefaultDescription
modeoff | observe | enforceobserveoff does nothing (Off in the app); observe detects and records only, and changes nothing (Monitor); enforce detects and handles each match by its rule’s action (On).
enablelist of strings[]Built-in rules to switch on that are off out of the box, by id.
disablelist of strings[]Built-in rules to switch off, by id.
actionsmap of built-in rule id → cut | record{}What a built-in rule does under enforce, written only where it differs from the factory setting (rm-rf-root: record).
customlist of security.inspect_tools.custom[][]Rules of your own, matched against the arguments of a tool call.

FieldTypeDefaultDescription
namestringrequiredName shown in logs and in the app; it identifies the rule and has to be unique within this guard.
patternstringrequiredRegular expression.
actioncut | recordrecordUnder enforce: cut the response off, or only record the match.
disabledboolfalseSwitches the rule off and keeps it in the file.

Built-in rules:

idNameUnder enforce, out of the box
curl-pipe-shDownload and runcut
base64-decode-execDecode and runcut
exfil-envSend out environment variablescut
exfil-credentialsSend out a credential filecut
exfil-credentials-reversedSend out a credential file (verb first)cut
ssh-key-readRead a private key or cloud credentialcut
secret-to-unknown-hostSend a credential to an unknown hostcut
thinkwatch-dataRead or change ThinkWatch’s own datacut
write-startup-itemWrite a startup itemcut
crontab-installInstall a scheduled jobcut
rm-rf-rootDelete home or rootrecord
chmod-777World-writable permissionsrecord
upload-file-to-hostUpload a local file to an external hostrecord

security.content#

Words, patterns or characters in what the caller sends (user messages and the tool results in them, not the system prompt or the model’s own turns), and, for custom rules with response in their scope, words or patterns in the text of the answer. Each rule matches a keyword (contains), a regular expression (regex) or code points (codepoints, requests only), and says what happens under enforce: block refuses the request (requests only), strip deletes every match and sends the rest, replace puts the rule’s replacement in place of every match and sends the rest, record only records it. In a request, the text is checked again after deleting or replacing, so a keyword split by hidden characters is caught once they are gone; a replacement is not checked again. In an answer, what a rule deletes or replaces never reaches the client, and the request record keeps the answer the client received.

FieldTypeDefaultDescription
modeoff | observe | enforceobserveoff does nothing (Off in the app); observe detects and records only, and changes nothing (Monitor); enforce detects and handles each match by its rule’s action (On).
enablelist of strings[]Built-in rules to switch on that are off out of the box, by id.
disablelist of strings[]Built-in rules to switch off, by id.
actionsmap of built-in rule id → block | strip | replace | record{}What a built-in rule does under enforce, written only where it differs from the factory setting. Built-in rules check requests only; replace puts *** in place of a match.
customlist of security.content.custom[][]Rules of your own, each checking requests, answers or both.

FieldTypeDefaultDescription
namestringrequiredName shown in logs and in the app; it identifies the rule and has to be unique within this guard.
patternstringrequiredA keyword; a regular expression with match: regex; code points with match: codepoints (U+200B, U+E0000–U+E007F).
scopelist of request | response[request]What the rule checks: request is what the caller sends (its messages and the tool results in them); response is the text of the answer the client sees, the input of free-form tools such as Codex’s apply_patch included, and neither tool calls nor reasoning. Both may be listed. A streamed answer is never held back whole: only the last few characters that could still grow into a match wait for the next piece, at most one fewer than the longest keyword, and at most 256 for a regular expression; a match longer than that may have its beginning sent already. Under enforce, rules that delete hold back as many characters again: text that comes together where something was deleted is checked again, until no match is left. A match nested deeper than that may have its beginning sent already, and the rest of it is deleted. Under enforce, when any rule that checks answers deletes or replaces, logprobs are removed from streamed answers, and from a whole answer whose text changed, since they repeat the text token by token. A rule that checks answers cannot block, and cannot match by codepoints.
matchcontains | regex | codepointscontainscontains: the text contains pattern as written; characters such as . and * have no special meaning. regex: pattern is a regular expression. codepoints: the text has a character among the code points or ranges listed in pattern, separated by commas; requests only.
case_sensitiveboolfalseTell capital letters from small ones. Off by default: ABC and abc both match. Code points are what they are either way.
actionblock | strip | replace | recordrecordUnder enforce: block the request (requests only), strip what matched and send the rest, replace what matched with replacement and send the rest, or only record the match. Text that comes together where something was stripped is checked again, until no match is left; a replacement is never checked again, and in an answer a regular expression anchored to the start or end of the text or of a line (^, $, \A, \z) is checked once. Replacing cannot be undone: the answer does not get the original back.
replacementstring"***"What replace puts in place of a match.
disabledboolfalseSwitches the rule off and keeps it in the file.

Built-in rules:

idNameGroupOut of the boxUnder enforce, out of the box
unicode-tagsUnicode tag charactersinvisibleonstrip
bidi-controlsBidirectional controlsinvisibleonstrip
zero-widthZero-width charactersinvisibleoffstrip
private-usePrivate-use charactersinvisibleoffstrip
ignore-previous-instructionsIgnore previous instructionsinjectiononblock
ignore-all-previousIgnore all previousinjectiononblock
disregard-your-instructionsDisregard your instructionsinjectiononblock
jailbreakJailbreakinjectionoffblock
danDANinjectionoffblock
developer-modeDeveloper modeinjectionoffblock
you-are-nowPersona manipulationpersonaoffblock
new-personaNew personapersonaoffrecord
act-asAct aspersonaoffrecord
pretend-to-bePretend to bepersonaoffrecord
system-promptSystem prompt extractionpersonaoffrecord
reveal-your-instructionsReveal instructionspersonaoffrecord
what-are-your-rulesWhat are your rulespersonaoffrecord
base64-wallBase64 smugglingpersonaoffrecord
zh-ignore-previousIgnore previous instructions (Chinese)chineseoffblock
zh-forget-yourForget your instructions (Chinese)chineseoffblock
zh-do-not-followDo not follow (Chinese)chineseoffblock
zh-you-are-nowYou are now (Chinese)chineseoffblock
zh-role-playRole-play (Chinese)chineseoffrecord
zh-reveal-yourReveal your instructions (Chinese)chineseoffrecord
zh-system-promptSystem prompt (Chinese)chineseoffrecord
zh-jailbreakJailbreak (Chinese)chineseoffblock

The invisible group matches characters people cannot see and models can read. Unicode tag characters (U+E0000–U+E007F) and bidirectional controls (U+202A–U+202E, U+2066–U+2069) are on out of the box; zero-width characters (U+200B–U+200D, U+2060, U+FEFF) and private-use characters (U+E000–U+F8FF, U+F0000–U+FFFFD, U+100000–U+10FFFD) are off, since emoji, Persian and icon fonts use them too. All four delete what they match under enforce.

security:
  redact:
    mode: enforce
    enable: [internal-ip]
    custom:
      - name: employee-id
        pattern: 'EMP-\d{6}'
        label: EMPLOYEE
  inspect_tools:
    mode: enforce
  content:
    mode: enforce
    enable: [zero-width]
    custom:
      - name: project-x
        pattern: project-x
        action: strip
      - name: codename
        pattern: polaris
        scope: [request, response]
        action: replace
        replacement: "[codename]"

retention#

Two limits, because the two kinds of data differ in size by three orders of magnitude: request bodies are tens of kilobytes each, a request’s record a few hundred bytes. The byte limit covers bursts.

Each request and response body is kept up to 4 MiB; of a longer one, the beginning is kept. Bodies are written with credentials and personal numbers already taken out. A request stored under enforce carries the placeholders the upstream received; anything else the redaction rules (security.redact) recognize is masked, in every mode, off included.

FieldTypeDefaultDescription
body_daysinteger7Days to keep request and response bodies.
row_daysinteger90Days to keep the record of each request (time, model, usage, cost).
body_max_bytesinteger5368709120Upper bound on the bytes bodies may take; beyond it the oldest days go first. The default is 5 GiB.

failover#

An upstream that fails is set aside for a while, so that the next requests go straight to the next candidate. How long depends on the reason the upstream gives: an insufficient balance waits for a top-up, and a used-up quota waits until the moment the upstream says it resets. A request with a single candidate is never affected. On a Responses WebSocket connection, each response.create counts here like a request: one that fails because of the upstream before any content arrives counts as a failure, and so does a connection the upstream refuses or that cannot be made.

Before the first content of a streamed answer reaches the client, an error the upstream sends in the stream moves the request to the next candidate, the same as an error status would.

When every candidate has been tried and the last one failed temporarily before any content reached the client — 500, 502, 503, 504 or 529, an overload or server error reported in a streamed answer, unreachable, or its stream broke or ended before any content — the request is retried after a pause instead of passing the error on. The pause is 2, 4 and 8 seconds, then 16 seconds each time, varied by up to a fifth; when the upstream asks for a longer one (retry-after-ms, Retry-After), it is used instead. Each retry goes again to the candidates that failed temporarily, in order, skipping those set aside unless all of them are; a conversation that stays on an upstream retries only that one. Retrying stops when the next pause would end after wait_secs, the time a request may wait in all; the client then gets the last error. Rate limits, used-up quotas, an insufficient balance, rejected credentials, missing models, errors in the request itself, other errors reported in the stream, other 5xx statuses and idle timeouts are not retried; they still move the request to the next candidate as before. Each retry appears among the request’s attempts with its number and the pause before it. wait_secs: 0 turns retrying off.

An upstream can also go quiet: it accepts the request and then sends nothing, or stops partway through. After idle_timeout_secs (300 by default) without content, the gateway stops waiting for it. The time counts from the moment the request is sent to that upstream, so waiting for a free slot does not count, and starts again with every piece of content: text, reasoning and tool calls count; keep-alives (SSE comments, Anthropic’s ping, empty chunks, Responses’ response.in_progress) do not, so an upstream that only keeps the connection alive still runs out of time. A whole, non-streamed answer counts from sending to the complete answer.

  • When no content has reached the client yet, the upstream counts as failed (towards failures_to_pause, like a 5xx), the attempt appears with the outcome idle_timeout, and the request moves to the next candidate. Until then an upstream’s answer is held back from the client, so the next upstream starts it afresh. With no candidate left, the client gets a timeout error (504) in its own format. When the client receives a stream, the last candidate’s stream is held the same way while the request can still be retried within wait_secs; otherwise it is passed on as it arrives, and once its response has started, a timeout there ends it with an error event instead. A non-streamed answer from the last candidate is always passed on as it arrives.
  • A streamed answer is held for at most 15 seconds. If no content has arrived by then, the client receives 200 and the streaming headers, so that its own wait for headers does not run out, followed by an SSE comment (: keep-alive) every 15 seconds; Gemini clients get no comments. The upstream’s events stay held until its first content, and failover continues as before under the same 200: the next upstream’s stream starts cleanly, and when no candidate is left, the stream ends with an error event in the client’s format instead of a 504. The comments do not count as content.
  • When content has already reached the client, the request cannot move on without repeating it: the answer ends with an error event in the client’s format, and the request is recorded as failed.

The conversation then no longer stays on that upstream for the rest of its turn. Models that think for a long time before they write anything need a longer timeout.

failover:
  idle_timeout_secs: 600

An upstream that answers 429 (rate limited) is not set aside but paused: for as long as it says (retry-after-ms, Retry-After, or a delay in the error body; under one second, or under one millisecond for retry-after-ms, counts as not said), and otherwise for 5, 10, 20 and 40 seconds, then 60 seconds each time, by its consecutive 429s; any other answer starts the count again. The pause is at most rate_limit_max_pause_secs, and the circuit breaker and load-balance do not count it as a failure. The request that got the 429 moves to the next candidate, or waits for the pause to end; the client sees nothing of it. A streamed answer that reports a rate limit before any content counts as a 429 too; only on a request’s last candidate, once the request can no longer wait (wait_secs), does the client get that error (the upstream is paused all the same). When the pause ends, one request goes first, and the others follow once the upstream has accepted it (for a streamed answer, once its content starts), or at once if that request gets no answer, times out or finds the upstream’s quota used up. A pause lasts until twcore restarts at most.

A request waits for upstreams that cannot take it yet for at most wait_secs in all: an upstream at its max_concurrent, one paused after a 429, and one at its minute or hour usage limit. The same time also covers waiting for a key’s minute or hour limit and the pauses before retries, so a request never waits longer than wait_secs for all of them together. An ongoing conversation waits for the upstream it stays on and, if it does not become available in time, moves on to the next one, where its cache starts over. A new conversation skips such an upstream at once. An upstream that has used up a day, week or month usage limit is skipped. When no candidate is left, the request waits for whichever becomes available first; streamed answers keep the client’s connection open with the 200 and keep-alives described above. If none becomes available, the client gets a 429 with Retry-After and retry-after-ms set to when the first one will; when every candidate has used up its period, Retry-After is the earliest reset and x-should-retry is false. If an upstream did receive the request and failed, the client gets that failure instead.

FieldTypeDefaultDescription
failures_to_pauseinteger3Consecutive failures without a stated reason (5xx, connection errors) before the upstream is set aside. From 1 to 100.
pause_secsinteger60Seconds the first such pause lasts. Each further pause doubles it, up to max_pause_secs; one success resets it.
max_pause_secsinteger600Upper bound on the doubled pause, in seconds; not less than pause_secs.
no_balance_pause_secsinteger1800Seconds to set aside an upstream that reports an insufficient balance.
quota_pause_secsinteger3600Seconds to set aside an upstream whose quota is used up when it does not say when the quota resets. When it does, the upstream is set aside until then.
rate_limit_max_pause_secsinteger3600Longest pause, in seconds, after an upstream answers 429 (rate limited). The pause lasts as long as the upstream says (retry-after-ms, Retry-After, or a delay in the error body); when it says nothing, it backs off by the upstream’s consecutive 429s: 5, 10, 20 and 40 seconds, then 60 seconds each time. A pause is not a failure: requests wait for the upstream within wait_secs, and the circuit breaker does not count it.
idle_timeout_secsinteger300Seconds an upstream may go without sending content before the gateway stops waiting for it. Counted from the moment the request is sent and started again by every piece of content: text, reasoning and tool calls count, keep-alives do not. A whole (non-streamed) answer counts from sending to the complete answer. Before any content has reached the client, the upstream counts as failed and the request moves to the next one; with none left, the client gets a timeout error. After content has reached the client, the answer ends with an error. From 30 to 3600.
wait_secsinteger120Seconds a request waits in all, counted once the key’s own max_concurrent lets it in: for a key’s minute or hour limit to free up, for an upstream that cannot take it yet — at its max_concurrent, paused after answering 429, or at one of its minute or hour usage limits — and before retrying upstreams that failed temporarily before any content (overloaded, a server error, unreachable). A key limit that does not free up in time refuses the request. An upstream that cannot take it is skipped for the next one, and when none is left the request waits for the first to become available; a conversation that stays on an upstream waits for it. When nothing became available in time, the client gets 429 with Retry-After; when retries did not succeed in time, it gets the last upstream error. 0: never wait or retry. From 0 to 600.

aliases#

A model alias is another name for the same model. Clients request it like any model; each upstream receives the name it uses for that model.

aliases:
  deepseek-v4.1: DeepSeek-v4.1-flash
  claude-sonnet-5:
    - claude-sonnet-5
    - us.anthropic.claude-sonnet-5-v1:0
    - anthropic/claude-sonnet-5

To hide the mapped original names from client model lists, use the object form:

aliases:
  coding:
    models: [provider-model-2026]
    hide_originals: true  # optional; defaults to false

models accepts the same string or list forms as a simple alias. The option only changes discovery (/v1/models, Gemini lists and /codex/models); original-name requests and single-model lookups keep their existing permissions and routing. Filtering runs after protocol, availability and key permissions: only a usable, visible alias can hide its exact targets. Alias identities stay visible, and shared targets remain hidden until the last effective hiding alias is disabled or removed. Changes reload live; clients may cache their picker and need a model-list refresh. Admin inventories and permission editors stay complete.

  • An upstream serves an alias when it offers one of the listed names within its models_only, and receives the first such name in the list. Every upstream that serves an alias takes part in failover for it.
  • GET /v1/models lists aliases next to the upstream models, and the original names stay available unless hidden from the list as above.
  • An alias takes precedence over an upstream model of the same name: a request for claude-sonnet-5 above goes only to upstreams offering one of the three names. List an upstream’s own name when it should keep serving it.
  • A key’s allow and a rule’s when.model written for an upstream model name also cover the aliases that list it; written for an alias, they cover only the alias.
  • When the upstream answers with the model it was sent, the answer carries the name the client asked for. An answer naming a different model is passed on as it is.
  • Prices and the upstream check-up use the name sent upstream. The request log keeps the name the client asked for, and each attempt the name it sent.
  • To send one key’s requests to a particular upstream model, use a rule with pinned models (routes[].rules[].to) rather than an alias: an alias changes what the name means for every client.

groups#

A group puts several upstreams behind one name. Rules send requests to a group with to.

FieldTypeDefaultDescription
namestringrequiredName of the group; unique, and not the name of an upstream.
typefallback | select | load-balance | url-test | cheapestfallbackfallback: the first healthy member, in order. select: the member named in selected. load-balance: requests are shared out in proportion to the members’ weights; a conversation in progress stays where it is. url-test: the fastest by measured time from sending a request to the first content of the answer. cheapest: the lowest input price.
providerslist of strings or groups[].providers[]requiredMember upstreams, by name; not groups. Each upstream appears once in a group. In a load-balance group, a member can be written as {name, weight}.
selectedstring—For select: the chosen member.
balance_byweights | latency | health | latency-healthweightsFor load-balance: what the members’ weights are multiplied by. weights: nothing; the weights alone. latency: faster upstreams get more. health: upstreams that fail less get more. latency-health: both. Other group types take only weights.

fallback is the default because a single user’s machine has no load to spread.

In a load-balance group, a member can carry a weight, from 1 to 100; a member written as just its name has weight 1. Weights set how the group’s requests are shared out: with { name: anthropic, weight: 7 } and relay, the official API serves seven requests in ten. Conversations in progress stay on the upstream that answers them (see below) and count toward its share, so the balance is kept by where new conversations start. An upstream that is cooling down after failures, is at its max_concurrent, or cannot serve a request, sits that request out, and the others share it by their weights. A new WebSocket connection is placed the same way and counts as one request; everything sent on it then goes to the upstream it connected to. Other group types take no weights.

FieldTypeDefaultDescription
namestringrequiredThe upstream, by name. A member written as just its name has weight 1.
weightinteger1The member’s share of a load-balance group’s requests, in proportion to the other members’ weights. From 1 to 100. Other group types take no weight other than 1.
groups:
  - name: pool
    type: load-balance
    providers:
      - { name: anthropic, weight: 7 }
      - relay

Whatever the type, a conversation stays on the upstream that last answered it, so that what the upstream holds of it in its prompt cache is read again rather than paid for in full elsewhere. Within a turn (while the client sends tool results back) it always stays; across turns it stays while the previous answer read or wrote at least 1024 tokens of prompt cache and came less than five minutes ago. An upstream that is cooling down after failures releases the conversation, and whichever upstream answered after a failover is the one it stays on. The rule a turn matched at its start also holds for the rest of that turn: rules keyed on input size or images do not move a turn halfway, unless its input no longer fits the context window of a model the rule sends it to. A load-balance weight is therefore the long-run share of requests: conversations in progress stay where they are and count toward that upstream’s share.

A conversation carrying content an upstream encrypted is a different case. OpenAI Responses hands clients encrypted state to send back each turn (encrypted_content: reasoning, compacted history, what Codex sub-agents report to each other), and only the account that encrypted it can read it. Such a conversation goes back to the upstream that last answered it whatever the rule now picks, even after the configuration changed, the cache went cold or a day has passed (it is remembered for 30 days). When it carries more than encrypted reasoning, that upstream is the only candidate: there is no failover, and if the upstream has been removed the request is refused with an explanation. Encrypted reasoning alone can be left out, so other upstreams remain as failover. After twcore restarts the gateway no longer knows which upstream encrypted a conversation; if another upstream receives it and cannot read it, the failure says so, and the conversation has to go back to the original upstream or start over.

balance_by lets a load-balance group also look at how each upstream has been doing lately. Each member’s weight is multiplied by a factor, and the group shares out requests by the result in the same way as above.

  • weights (the default): the weights alone.
  • latency: faster upstreams get a larger share. Speed is the typical time from sending a request to the first content of the answer, the same measurement url-test uses. An upstream twice as fast as the middle of the group has its weight multiplied by four, by at most ten and at least a tenth.
  • health: upstreams that fail less get a larger share. It looks at the last 50 requests within the past 30 minutes. Server errors, used-up quota or balance, rejected credentials, timeouts (including failover.idle_timeout_secs before any content) and connection errors count as failures; errors caused by the request itself do not, and neither does a rate limit (429), a client that cancels, a request aborted by hand, or an upstream skipped because it is at its max_concurrent or at a usage limit. An upstream that keeps failing keeps a twentieth of its weight, so it still gets the occasional request and its recovery is noticed; one that fails outright is set aside by failover as before.
  • latency-health: both factors, multiplied.

Speed is measured on streamed answers only, from the moment the request is sent to that upstream, so waiting and upstreams that failed before it do not count. An upstream given up on because it sent no content within failover.idle_timeout_secs counts as taking the whole wait. On a Responses WebSocket connection, each response.create counts as one request for both speed and failures, its speed measured from the moment the upstream starts answering it.

An upstream without enough measurements yet counts as average. As with weights alone, conversations in progress stay where they are, and new conversations make up the difference.

groups:
  - name: pool
    type: load-balance
    balance_by: latency-health
    providers:
      - { name: official, weight: 3 }
      - relay

routes#

A route is a list of rules evaluated top to bottom. Each key takes the route named in its route, otherwise default_route, otherwise the route named default; without any, requests fail over across all upstreams in the order they are declared.

FieldTypeDefaultDescription
namestringrequiredName of the route; unique. default is the one keys use unless told otherwise.
ruleslist of routes[].rules[][]Evaluated top to bottom; the first rule with to or deny that matches decides where the request goes.

routes[].rules#

FieldTypeDefaultDescription
namestringrequiredName shown in logs and in the traffic view.
whenobject, routes[].rules[].when—Conditions, all of which have to hold. Unset: matches every request.
tostring, or list of routes[].rules[].to[]—An upstream or a group, by name; __all__ is every upstream in declared order. Or pinned models: a list of upstreams with the model to send to each, tried in order. Not allowed together with when.provider_would_be.
setobject, routes[].rules[].set—Parameters to rewrite. Collected from every matching rule, not only the first.
denystring—Refuse the request with this reason.

routes[].rules[].to#

to names an upstream or a group, or pins models: a list of upstreams, each with the model sent to it, tried in order. A pinned model is sent as written, without aliases or set.model, so a rule can send a key’s requests to one upstream’s model even when an alias of the same name points elsewhere.

routes:
  - name: default
    rules:
      - name: Opus on Bedrock
        when: { model: claude-opus-5 }
        to:
          - { provider: bedrock, model: us.anthropic.claude-opus-5-v1:0 }
          - { provider: anthropic, model: claude-opus-5 }

FieldTypeDefaultDescription
providerstringrequiredAn upstream, by name; not a group. Each upstream appears once in the list.
modelstringrequiredThe model name sent to that upstream, as written: aliases do not apply, and no set.model changes it. Not allowed together with set.model in the same rule.

routes[].rules[].when#

FieldTypeDefaultDescription
modelstring—Requested model, glob (claude-opus-*). An upstream model name also matches requests for the aliases that list it; an alias matches only requests for the alias.
clientstring—Name of the gateway key the request used, exactly.
dialectstring—API format the client spoke: anthropic, openai-chat, openai-responses, gemini.
input_tokenscomparison (>200k, <=4k, ==3)—Estimated input tokens.
max_tokenscomparison (>200k, <=4k, ==3)—The request’s max_tokens. A request without one never matches.
tool_countcomparison (>200k, <=4k, ==3)—Number of tools offered.
cachebool—Whether the request uses the prompt cache.
toolsbool—Whether the request offers tools.
imagebool—Whether the request contains an image.
thinkingbool—Whether extended thinking is on.
streambool—Whether the response is streamed.
intentstring or list of strings—A client helper request: assistant_internal for any of them, or one class (titling). Classes set to intercept in client_probes are answered locally and never reach routing.
timestring or list of strings—Local time of the machine core runs on, as a window [days ]HH:MM-HH:MM: mon-fri 09:00-18:00, sat,sun 00:00-24:00, 22:00-06:00. Several windows: any of them. See below.
provider_would_bestring or list of strings—The upstream routing chose. Such a rule is evaluated after routing, may only set or deny, and cannot have to.

A comparison starts with >, >=, <, <= or ==, and the number may end in k or m: ">200k", "<=4k". Without an operator it is an error, not an equality: "200k" alone is refused.

A time window is [<days> ]<HH:MM>-<HH:MM>, evaluated against the local wall-clock time of the machine core runs on, at the moment the request is routed. <days> is a comma-separated list of mon, tue, wed, thu, fri, sat, sun and ranges such as mon-fri or fri-mon (which wraps past Sunday); leave it out for every day. Case does not matter. The hours are 24-hour: the start is included, the end is not, and the end may be 24:00. An end earlier than the start is an overnight window, and its days are the days it starts on: fri 22:00-06:00 runs from Friday 22:00 to Saturday 05:59. A value that does not fit this shape is refused when the configuration is read, naming the rule and the value.

routes[].rules[].set#

set.model may name an alias; each upstream then receives its own name for it. Answers carry the name the client asked for, as with aliases.

set.reasoning writes the effort where the client’s format keeps it (output_config.effort, reasoning_effort, reasoning.effort, Gemini’s thinkingLevel, or the thinking budget when the client gave a budget). A request converted for an upstream of another format carries it the way that format writes it: off sent to an OpenAI Chat or Responses upstream is none there, and sent to Gemini 2.5 Flash it is a thinking budget of 0.

When an upstream answers 400 because of the reasoning effort, whether the client or a rule chose it, the request is sent once more to the same upstream with a level the upstream named (the nearest one, or the ceiling it gave), or without an effort when it named none. A refused none is replaced by the lowest level the upstream named, so reasoning stays as low as it allows. The resend is a new attempt in the request details and does not count as a retry. If it succeeds, the level is remembered for that upstream and model for 7 days in effort-memory.json in the data directory, so later requests send the working level directly. The entries of an upstream are forgotten when its address, key or protocol changes, or when it is removed. Requests that only give a thinking budget are not resent. On WebSocket connections (Codex), a turn refused for its effort is not resent and nothing is remembered: the refusal reaches the client as it is.

FieldTypeDefaultDescription
modelstring—Send a different model. The prompt cache of the session is lost.
max_tokensinteger—Replace max_tokens.
reasoningoff | low | medium | high | xhigh | max—Reasoning. off turns it off: the reasoning settings and the effort are removed, and in the OpenAI formats the effort is set to none. A level changes the effort. Only requests that already have reasoning on are changed; requests without reasoning (such as title generation) are left as they are.
only_at_session_startboolfalseApply only when a session starts. Recorded and shown; not in effect yet.
groups:
  - name: fast
    type: url-test
    providers: [anthropic, relay]

routes:
  - name: default
    rules:
      - name: long context goes to the official API
        when: { input_tokens: ">200k" }
        to: anthropic
      - name: titles go to the cheap model
        when: { intent: titling }
        set: { model: claude-haiku-4-5 }
      - name: everything else
        to: fast
default_route: default

plugins#

Script plugins change requests before they reach an upstream and answers before they reach the client. They run in a sandbox inside core, without access to files, the network or the real values of secrets. The app installs them: each plugin’s code goes to plugins/<id>.js next to this file, a copy of the approved code to plugins/.approved/<id>.js, and the code’s SHA-256 to sha256.

A plugin’s file holds its settings too. The manifest at the top of the file says what to do when the plugin fails (on_error), which requests it handles (match) and the value of each setting (settings.<name>.value). When the app changes one of these, it rewrites only the manifest in the file and updates sha256 along with it. This list keeps just the plugin, its approved hash and whether it is on. Entries written by core 0.58 also have on_error, scope and settings: they are ignored, and removed the next time the app changes a plugin.

A plugin runs only while its file has exactly the approved hash. When the file changes on disk or disappears, the plugin stops within seconds and the app shows the change for review. Until the change is approved, the requests the plugin covers are refused (on_error: "reject", the default) or pass without it (on_error: "skip"), as the approved file says. A plugin that does not load is handled the same way. Neither keeps the rest of the configuration from taking effect.

Plugins run in the order of this list.

In match, clients, models and upstreams are lists of names or patterns with * anywhere in them, matched regardless of case; a list that is empty or left out matches everything. clients names the client app (claude-code, codex, …), and a request whose app is not recognised matches only an empty list. models matches the model sent to the upstream: when a routing rule renames the model, the new name is the one that matches. upstreams applies to requests and answers alike.

A plugin changes a request after routing, each time the request is sent to an upstream. A request that fails over to another upstream starts again from what the client sent, and the plugin sees which upstream and which model name the request goes to. Routing, model checks and session grouping use what the client sent. A plugin that changes the model name only renames what is sent to that upstream: the request is not routed again, and the new name must still be one of the models the key may use (clients[].allow), or the request is not sent.

A plugin handles the kinds of request its code declares: conversations (Anthropic Messages, OpenAI Chat Completions and Responses, and Gemini, including their token counts and compaction), embeddings (/v1/embeddings, Gemini :embedContent and :batchEmbedContents) and legacy completions (/v1/completions). A plugin that declares none handles conversations only. Requests of a kind a plugin does not handle pass without it, whatever its on_error. Other endpoints, such as images and audio, pass without any plugin.

FieldTypeDefaultDescription
idstringrequiredLowercase letters, digits and hyphens, 1 to 40 characters; unique.
filestringrequiredThe plugin’s code, relative to this file’s directory. It is always plugins/<id>.js; the app writes it.
sha256stringrequiredSHA-256 of the approved code, 64 lowercase hexadecimal characters. When the file no longer has this hash, the plugin stops running until the change is approved in the app. The approved code is kept in plugins/.approved/<id>.js.
enabledbooltrueRun the plugin. false keeps it installed and out of every request.
plugins:
  - id: add-date
    file: plugins/add-date.js
    sha256: 9f2b6c0e4a1d8f3b7c5e2a9d6f1b4c8e3a7d0f5b2c9e6a1d4f8b3c7e0a5d2f9b
    enabled: true

The start of plugins/add-date.js:

export const manifest = {
  name: "Add date",
  api: 1,
  permissions: ["system"],
  match: { clients: ["claude-code"], models: ["claude-*"], upstreams: [] },
  on_error: "reject",
  settings: {
    note: { type: "string", label: "Note", value: "Answer in English." },
  },
};

sync#

Several devices (this Mac, a laptop, a server running a remote core) can keep one configuration in step through a sync location you choose: a folder that a cloud drive carries between devices, a WebDAV server, or S3-compatible storage. Each change is merged object by object with what the other devices did since the last sync; only when two devices change the same part of the same object to different values is there a conflict, which the app shows on every device until one of them resolves it. Turn sync on, join a location and resolve conflicts in the app (Settings, Sync and backup) or with twcore sync enable on a server; this section records the result.

What syncs is the configuration a configuration pack carries: upstreams, proxies, gateway keys, aliases, groups, routes, price sheets, security rules, plugins and the general settings, keys included. listen, retention and sync itself stay on this device, and so do a ChatGPT account’s sign-in and OAuth tokens. An upstream signed in on another device stays turned off on this one until it is signed in here; that does not turn it off anywhere else.

A device that joins with a configuration of its own merges it item by item; items that differ are chosen one by one, except the default gateway key: every installation makes one, so the joining device takes the sync data’s value and the clients set up on it are updated.

With encryption on, versions that are not encrypted are never read. A keyring that went missing, or an older one put back, does not turn encryption off: the app asks whether to put back the copy this device keeps. A location that reads as empty is checked a few more times before the app says so, and turning sync off is still up to you.

FieldTypeDefaultDescription
device_idstringrequiredGenerated when sync is turned on. File names at the sync location show its first 8 characters.
device_namestringrequiredShown in the device list of every device that syncs here.
encryptbooltrueEncrypt what goes to the sync location with the sync passphrase. Every device at one location uses the same setting; change it in the app, which re-uploads the latest version.
passphrasestring—The sync passphrase; required when encrypt is on, and kept after encryption is turned off: the keyring that says it is off is checked with it. Kept here in the clear, like the other keys in this file.
excludelist of strings[]Upstreams, by name, that are not synced: they are not uploaded and another device’s upstream of the same name does not replace them. Routes, aliases and groups that refer to them are not synced either.
targetobject, sync.targetrequiredThe sync location. The keyring, versions/ and devices/ live directly in it.

FieldTypeDefaultDescription
typefolder | webdav | s3requiredWhich kind of location; the other fields depend on it, see the three tables below.

FieldTypeDefaultDescription
typestringrequiredfolder
pathstringrequiredAn absolute path on the machine core runs on: a folder that iCloud Drive, Dropbox, OneDrive, Syncthing or a NAS share carries between devices.
proxystringdirectNot used for a folder.

FieldTypeDefaultDescription
typestringrequiredwebdav
urlstringrequiredThe WebDAV address of the folder. https://, with the certificate checked; http:// only for this machine and the local network.
userstring''User name.
passstring''Password, or the app password the service issues for WebDAV.
proxystringdirectWhere requests to the sync location go out: direct, system (the system proxy), or the name of a proxy under proxies. An http:// address on this machine or the local network has to be direct.

FieldTypeDefaultDescription
typestringrequireds3
endpointstringrequiredThe S3 endpoint. Hosts ending in amazonaws.com, aliyuncs.com or myqcloud.com put the bucket in the host name; every other endpoint (Cloudflare R2, MinIO, an address) puts it in the path. http:// only for this machine and the local network.
regionstringrequiredThe signing region; auto for Cloudflare R2.
bucketstringrequiredThe bucket. Keep it private.
prefixstring''A folder inside the bucket. No .. parts, and not starting with /.
access_key_idstringrequiredAccess key ID.
secret_access_keystringrequiredSecret access key.
proxystringdirectWhere requests to the sync location go out: direct, system (the system proxy), or the name of a proxy under proxies. An http:// address on this machine or the local network has to be direct.
sync:
  device_id: 3f9a2c1e-0b1d-4e2f-9a3b-5c6d7e8f9a0b
  device_name: Mac mini
  encrypt: true
  passphrase: "correct horse battery staple"
  exclude: [local ollama]
  target:
    type: webdav
    url: https://dav.example.com/dav/ThinkWatch
    user: name@example.com
    pass: "app password"

Environment variables#

VariableEffect
THINKWATCH_HOMEData directory, instead of ~/.thinkwatch (%APPDATA%\ThinkWatch on Windows).
TWCORE_LOGLog filter, in tracing syntax (info, debug, tw_gateway=debug).
HTTPS_PROXY, HTTP_PROXY, ALL_PROXY, NO_PROXYUsed by upstreams with proxy: system.
Any otherRead where the configuration writes ${NAME}.

Web tools#

Configure search, page reading and find-in-page under Routing → Auxiliary in Lite. core handles standalone Claude Code search requests without a helper model call, and runs hosted web tools in Anthropic Messages and OpenAI Responses requests through a bounded model/tool loop. Main-model calls are still metered. Client-side webpage fetching is unchanged. Web-tool responses are buffered and emitted as native JSON or SSE; keep-alives continue while waiting. Public HTTP/HTTPS text pages are supported; location-specific and cache-only native search modes require upstream execution.

Search connects directly to Google Docs web research, DuckDuckGo and Bing without API keys or a browser runtime. Automatic order is Google (Docs), DuckDuckGo, then Bing. Google (Docs) searches public web pages through a legacy endpoint; it depends on eligible public document contexts and may stop working if Google changes the service. It does not read or edit the context documents, does not support regional preferences, and enforces at least six seconds between requests. Document access failures can select another context; throttling and server errors cool down the engine and fall back to the next source. Challenges and unrecognized pages are failures, with per-engine pacing and cooldowns; they are never reported as empty search results. HTTP, HTTPS and SOCKS proxies tunnel to the public IPs checked by Core, keeping the original hostname for TLS and HTTP. Responses WebSocket connections use the same protected execution path. Only standalone helper searches can fall back upstream, when explicitly enabled.

FieldTypeDefaultDescription
modeupstream | coreupstreamupstream forwards hosted tools; core executes supported web operations.
engineslist of strings[]Empty tries google_docs, duckduckgo, then bing. Otherwise list these engine IDs in priority order.
proxystringsystemsystem, direct, or an existing proxy name.
languagestring''BCP 47 language tag. Empty follows the system language on the Core host (including a remote Core); if unavailable, the engine decides. Lite provides a language picker.
regionstring''Two-letter country/region code. Empty follows the region in the Core host’s system locale; if unavailable, the engine decides. Search engines apply supported regional preferences on a best-effort basis.
allowed_domainslist of strings[]Only these domains and their subdomains; also applies to page reads. Empty allows all public domains.
blocked_domainslist of strings[]Excluded domains and their subdomains; exclusions take precedence.
max_resultsinteger8Search results, 1–20.
read_pagesinteger2Read up to 0–5 pages when search snippets are insufficient. Explicit page tools remain available at zero.
max_page_charsinteger12000Returned page characters, 1000–50000.
timeout_secsinteger30Whole operation timeout including queueing, 1–120 seconds.
max_concurrentinteger4Concurrent web operations, 1–16.
interval_msinteger1000Minimum interval per search engine, 250–60000 milliseconds. A source may enforce a higher minimum (google_docs: 6000).
cache_secsinteger600Successful result lifetime, 0–86400 seconds. Zero disables caching.
max_tool_callsinteger8Maximum web operations per model request, 1–32.
fallback_to_upstreamboolfalseAllow standalone searches to fall back to the upstream before returning results. This may incur upstream fees.

This page is published from docs/config.md in the ThinkWatch Core repository, at v0.99.0.

Last updated 2026-10-25 Edit on GitHub