Caching

​

How It Works

1. The gateway generates a cache key from the request (model, messages, parameters)
2. If a cached response exists and hasn’t expired, it’s returned immediately
3. If not, the request goes to the provider and the response is cached

Request --> Cache Hit?
            |-- Yes --> Return cached response (fast, free)
            |-- No  --> Forward to provider --> Cache response --> Return

​

Enabling Caching

1. Go to your project Settings
2. Navigate to Gateway → Caching
3. Toggle caching on
4. Set your preferred TTL (time-to-live)

​

Cache TTL

​

Cache Keys

• Model name
• Messages/prompt content
• Temperature (if set)
• Other generation parameters

​

Cache Headers

X-Muxx-Cache: HIT    # Response from cache
X-Muxx-Cache: MISS   # Response from provider (now cached)

​

Bypassing Cache

client = OpenAI(
    base_url="https://gateway.muxx.dev/v1",
    default_headers={
        "X-Muxx-Api-Key": "muxx_sk_live_xxxxxxxxxxxx",
        "X-Muxx-Cache-Control": "no-cache"
    }
)

​

Cost Savings

• 1000 requests, 60% cache hit rate
• Only 400 requests billed to the provider
• 60% cost reduction

​

Viewing Cache Stats

• Cache hit rate over time
• Cost savings from caching
• Most frequently cached requests