提示缓存

最大化缓存命中率

设置 x-grok-conv-id（聊天补全 API）

x-grok-conv-id HTTP 头将具有相同对话 ID 的请求路由到同一台服务器。由于缓存条目是按服务器存储的，这可以最大化您的缓存命中率。

curl https://api.x.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "x-grok-conv-id: conv_abc123" \
  -d '{
    "model": "grok-4.5",
    "messages": [
      {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},
      {"role": "user", "content": "What is prompt caching?"}
    ]
  }'

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_XAI_API_KEY",
    base_url="https://api.x.ai/v1",
)

response = client.chat.completions.create(
    model="grok-4.5",
    messages=[
        {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},
        {"role": "user", "content": "What is prompt caching?"},
    ],
    extra_headers={
        "x-grok-conv-id": "conv_abc123",
    },
)

print(response.choices[0].message.content)
print(f"Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}")

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'YOUR_XAI_API_KEY',
  baseURL: 'https://api.x.ai/v1',
});

const response = await client.chat.completions.create(
  {
    model: 'grok-4.5',
    messages: [
      {
        role: 'system',
        content:
          'You are Grok, a helpful and truthful AI assistant built by xAI.',
      },
      { role: 'user', content: 'What is prompt caching?' },
    ],
  },
  {
    headers: {
      'x-grok-conv-id': 'conv_abc123',
    },
  },
);

console.log(response.choices[0].message.content);
console.log(
  `Cached tokens: ${response.usage.prompt_tokens_details.cached_tokens}`,
);

设置 prompt_cache_key（响应 API）

对于响应 API，直接在请求正文中使用 prompt_cache_key 字段。其功能与设置 x-grok-conv-id 相同——它将请求路由到同一台服务器以实现缓存重用。

curl https://api.x.ai/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -d '{
    "model": "grok-4.5",
    "input": "What is prompt caching?",
    "prompt_cache_key": "b79ad29b-b3f9-463c-bca6-041d5058d366"
  }'

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_XAI_API_KEY",
    base_url="https://api.x.ai/v1",
)

response = client.responses.create(
    model="grok-4.5",
    input="What is prompt caching?",
    extra_body={
        "prompt_cache_key": "b79ad29b-b3f9-463c-bca6-041d5058d366",
    },
)

print(response.output_text)
print(f"Cached tokens: {response.usage.input_tokens_details.cached_tokens}")

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'YOUR_XAI_API_KEY',
  baseURL: 'https://api.x.ai/v1',
});

const response = await client.responses.create({
  model: 'grok-4.5',
  input: 'What is prompt caching?',
  // @ts-expect-error -- xAI-specific field
  prompt_cache_key: 'b79ad29b-b3f9-463c-bca6-041d5058d366',
});

console.log(response.output_text);
console.log(
  `Cached tokens: ${response.usage.input_tokens_details.cached_tokens}`,
);

import { xai } from '@ai-sdk/xai';
import { generateText } from 'ai';

const { text, usage } = await generateText({
  model: xai.responses('grok-4.5'),
  prompt: 'What is prompt caching?',
  providerOptions: {
    xai: {
      promptCacheKey: 'b79ad29b-b3f9-463c-bca6-041d5058d366',
    },
  },
});

console.log(text);
console.log(`Total tokens: ${usage.totalTokens}`);

设置 x-grok-conv-id 元数据（gRPC API）

对于使用 xAI SDK 的 gRPC API，将 x-grok-conv-id 作为 gRPC 元数据传递，以启用缓存重用的粘性路由。

from xai_sdk import Client
from xai_sdk.chat import system, user

client = Client(
    api_key="YOUR_API_KEY",
    metadata=(("x-grok-conv-id", "conv_abc123"),),
)

chat = client.chat.create(model="grok-4.5")
chat.append(system("You are Grok, a helpful and truthful AI assistant built by xAI."))
chat.append(user("What is prompt caching?"))

response = chat.sample()
print(f"Response: {response.content}")
print(f"Cached tokens: {response.usage.cached_prompt_text_tokens}")

下一步

• 什么会破坏缓存