模型能力

图像理解

WARNING

发送图像时，建议不要在服务器上存储请求/响应历史记录。否则请求可能会失败。 查看 。

某些模型允许输入图像。模型在生成响应时会考虑图像上下文。

构建消息体 - 与纯文本提示的区别

图像理解的请求消息类似于纯文本提示。主要区别在于，我们不是发送文本输入：

[
  {
    "role": "user",
    "content": "What is in this image?"
  }
]

而是将 content 作为对象列表发送：

[
  {
    "role": "user",
    "content": [
      {
        "type": "input_image",
        "image_url": "data:image/jpeg;base64,<base64_image_string>"
      },
      {
        "type": "input_text",
        "text": "What is in this image?"
      }
    ]
  }
]

image_url 值也可以是互联网上的公共 URL，而不是 base64 数据 URL。

图像理解示例

import os
from xai_sdk import Client
from xai_sdk.chat import user, image

client = Client(
    api_key=os.getenv("XAI_API_KEY"),
    management_api_key=os.getenv("XAI_MANAGEMENT_API_KEY"),
    timeout=3600,
)

image_url = "https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png"
chat = client.chat.create(model="grok-4.5")
chat.append(
    user(
        "What's in this image?",
        image(image_url=image_url, detail="high"),
    )
)

response = chat.sample()
print(response)

# The response ID that can be used to continue the conversation later

print(response.id)

import os
import httpx
from openai import OpenAI

client = OpenAI(
    api_key="<YOUR_XAI_API_KEY_HERE>",
    base_url="https://api.x.ai/v1",
    timeout=httpx.Timeout(3600.0), # Override default timeout with longer timeout for reasoning models
)
image_url = (
    "https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png"
)

response = client.responses.create(
    model="grok-4.5",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": image_url,
                    "detail": "high",
                },
                {
                    "type": "input_text",
                    "text": "What's in this image?",
                },
            ],
        },
    ],
)

print(response)

# The response ID that can be used to continue the conversation later

print(response.id)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: "<api key>",
    baseURL: "https://api.x.ai/v1",
    timeout: 360000, // Override default timeout with longer timeout for reasoning models
});

const image_url =
    "https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png";

const response = await client.responses.create({
    model: "grok-4.5",
    input: [
        {
            role: "user",
            content: [
                {
                    type: "input_image",
                    image_url: image_url,
                    detail: "high",
                },
                {
                    type: "input_text",
                    text: "What's in this image?",
                },
            ],
        },
    ],
});

console.log(response);

// The response ID that can be used to recall the conversation later
console.log(response.id);

import { xai } from '@ai-sdk/xai';
import { generateText } from 'ai';

const { text, response } = await generateText({
    model: xai.responses('grok-4.5'),
    messages: [
        {
            role: 'user',
            content: [
                {
                    type: 'image',
                    image: new URL('https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png'),
                },
                {
                    type: 'text',
                    text: "What's in this image?",
                },
            ],
        },
    ]
});

console.log(text);

// The response ID can be used to continue the conversation
console.log(response.id);

curl https://api.x.ai/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -m 3600 \
  -d '{
    "model": "grok-4.5",
    "input": [
      {
        "role": "user",
        "content": [
          {
            "type": "input_image",
            "image_url": "https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png",
            "detail": "high"
          },
          {
            "type": "input_text",
            "text": "What'\''s in this image?"
          }
        ]
      }
    ]
  }'

图像输入一般限制

• 最大图像大小：20MiB
• 最大图像数量：无限制
• 支持的图像文件类型：jpg/jpeg 或 png
• 任何图像/文本输入顺序均可接受（例如，文本提示可以在图像提示之前）