An AI bastion host for organizations
ThinkWatch secures, audits, and governs every AI API call and MCP tool invocation across an organization from a single control plane.
- Self-hosted
- Docker Compose or Kubernetes
- v3.2.0
- BSL 1.1
Drop-in compatible with
- Claude Code
- Cursor
- Continue
- Cline
- OpenAI SDK
- Anthropic SDK
AI adoption without governance
As AI agents are adopted across engineering teams, organizations face increasing governance requirements.
Unmanaged API keys
Keys are hardcoded in .env files, shared over chat, and rarely rotated.
Limited visibility
No record of which user called which model, how many tokens were consumed, or at what cost.
No access control
Every developer has direct access to every model and MCP tool.
Compliance gaps
AI-assisted code generation and data access leave no audit trail.
Unattributed costs
Monthly AI spend cannot be explained or attributed to users or teams.
ThinkWatch addresses these issues with a single deployment.
All AI requests pass through a single gateway
A drop-in replacement for the OpenAI and Anthropic SDKs. Agents keep their existing code and change only the base URL to point at ThinkWatch.
Gateway, MCP proxy, RBAC, and analytics in one binary
The gateway, the MCP proxy, and the admin API run as a single Rust binary backed by PostgreSQL, Redis, and ClickHouse, without separate microservices or glue code.
A single port for all supported models, with drop-in compatibility.
Multi-format proxy
OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Gemini APIs on a single port. Cursor, Continue, Cline, Claude Code, and the official SDKs connect by changing only the base URL.
Per-model routing: automatic or manual
Each model has its own routing setting. Automatic modes weight upstreams by latency, by success rate, or by both combined (the default); manual mode splits traffic by fixed weights set on a drag bar. A circuit breaker takes failing upstreams out of rotation, a request that fails with a retryable error is sent to another upstream, and the console shows live health for every route.
Multiple providers and a model-level kill switch
OpenAI, Anthropic, Google Gemini, Azure OpenAI, AWS Bedrock, or any OpenAI-compatible endpoint, with automatic format conversion. A single model can be paused without disabling the whole provider, and every request log records the upstream model that actually served the request.
Virtual API keys
Each tw- key belongs to one user and can be limited to the AI gateway, the MCP gateway, the admin API, or any combination, and tagged with a cost center. Keys support expiry, inactivity timeouts, and rotation with a grace period in which the old key keeps working; revoked and disabled keys are listed on a separate tab. The plaintext is shown once, and only a hash is stored.
Rate limits & budgets
Sliding windows from one minute to one week limit requests or tokens, and daily, weekly, or monthly budgets cap token use, weighted per model. Both attach to users, API keys, or roles and apply to MCP tool calls as well as model requests. Request counts are checked before a request; tokens and budgets are counted after the response, and once a budget is used up further requests are refused. If Redis is unavailable, limits are skipped by default; a setting makes the gateway refuse requests instead.
Cost reporting
Each request is priced from a platform-wide per-token price and per-model weights. Spend is reported by model, user, provider, and cost center and can be filtered by team, with CSV chargeback reports and a month-end forecast. Budget alerts at 50%, 80%, 95%, and 100% are written to the audit log once per period.
Management API & OpenAPI
API keys, users, providers, and settings can be managed through the admin API, using a tw- key that is allowed to call it. An OpenAPI specification describes the endpoints, so scripts and CI pipelines can manage key lifecycles without the console.
Built for organizations, not single users
Most MCP gateways are designed for a single user with one shared service account. ThinkWatch is designed for organizations in which each developer authenticates upstream under their own identity, each tool call is audited back to a specific person, and the entire system runs inside the organization's network.
| Capability | ThinkWatch | SaaS gateways | DIY mcp-proxy |
|---|---|---|---|
| Per-user upstream OAuth (no shared service account) | ✓ | shared account | — |
| Onboarding from a server URL via Dynamic Client Registration | ✓ | partial | — |
| Tool-level RBAC and per-user tool catalogs | ✓ | limited | — |
| Full audit trail in ClickHouse (queryable, forwardable) | ✓ | hosted-only | — |
| Response cache kept separate per user and account | ✓ | n/a | — |
| Self-hosted, single Rust binary, distroless | ✓ | SaaS-only | varies |
| Bilingual UI out of the box (English + 中文) | ✓ | English-only | — |
| BSL 1.1: free for non-production and small production use | ✓ | varies by vendor | OSS |
SaaS gateways = hosted MCP gateways, where traffic leaves your network and upstream access is typically shared. DIY = mcp-proxy or homegrown shims. Individual products vary; check the one you are evaluating.
Per-user upstream identity
With per-user OAuth, GitHub issues are created by Alice and Linear tickets are assigned to Bob, rather than to a shared mcp-bot service account. Audit trails extend end to end, from the IDE through ThinkWatch to the upstream system.
Onboarding through Dynamic Client Registration
For upstreams that support Dynamic Client Registration, the OAuth handshake runs automatically. The connection is established once the server URL is pasted and access is approved on the consent page, with no app registration or manual copying of client_id and secret.
Access control for tools
Tool-level RBAC and per-user tool catalogs ensure that each user sees only the tools permitted by their role and upstream account. A role can be limited to specific tools, such as read-only ones.
Fully self-hosted
ThinkWatch runs on the organization's own infrastructure as a single Rust binary backed by Postgres, Redis, and ClickHouse. There is no SaaS lock-in, and MCP traffic goes from ThinkWatch directly to the upstream servers, without a third-party gateway service in between.
Visibility into every token, key, and call
Real-time observability for AI requests across an organization. The panels below are React components rendered with mock data, not screenshots, and illustrate the console interface.
- OpenAIus-eastHealthyCB:Closed412 ms99.8%
- Anthropicus-westHealthyCB:Closed538 ms99.6%
- Google Geminius-centralHealthyCB:Closed297 ms99.9%
- Azure OpenAIeastus2DegradedCB:HalfOpen1240 ms96.2%
- AWS Bedrockus-east-1HealthyCB:Closed624 ms99.4%
Run it from source in four steps.
These are the development steps from the ThinkWatch README. For production, the deployment guide covers Docker Compose, Kubernetes with Helm, SSL, and production hardening.
Deployment guide- Start infrastructure
Brings up PostgreSQL, Redis, ClickHouse, and the other development services with Docker Compose.
$ make infra - Generate dev secrets and start the backend
Writes .env from .env.example with random secrets, then starts the gateway (:3000) and the console (:3001).
$ make dev-secrets$ make dev-backend - Start the web console
Installs dependencies and starts the Vite dev server.
$ cd web && pnpm install && pnpm dev - Finish the setup wizard
Create the Super Admin account and set the site name; the wizard then issues a first API key.
http://localhost:5173/setup
curl https://gateway.your-org.com/v1/chat/completions \
-H "Authorization: Bearer tw-your-api-key-here" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Hello"}],
"stream": true
}' Drop-in replacement: change the base URL and use a virtual tw- key.
Free production tier
BSL 1.1. Free for non-production use, and free in production up to 10,000,000 billable tokens and 10,000 MCP tool calls per month.