模型能力

语音概览

xAI Voice API 提供了一系列强大的语音功能，所有功能均由 Grok 驱动，具备企业级可靠性和亚秒级延迟。

语音转语音

通过 WebSockets 构建实时语音转语音的语音代理，具有低延迟的轮流发言和工具使用功能。对于客户端应用，使用临时令牌安全连接，无需暴露您的 API 密钥。

python

import asyncio
import json
import os
import websockets

async def voice_agent():
    async with websockets.connect(
        "wss://api.x.ai/v1/realtime?model=grok-voice-latest",
        additional_headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"}
    ) as ws:
        # Configure voice and enable tools
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "voice": "eve",
                "instructions": "You are a helpful customer support agent.",
                "turn_detection": {"type": "server_vad"},
                "tools": [{"type": "web_search"}]
            }
        }))
        
        # Stream audio and receive responses
        async for message in ws:
            event = json.loads(message)
            if event["type"] == "response.output_audio.delta":
                # Play audio: base64.b64decode(event["delta"])
                pass

asyncio.run(voice_agent())

javascript

import WebSocket from "ws";

const ws = new WebSocket("wss://api.x.ai/v1/realtime?model=grok-voice-latest", {
  headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },
});

ws.on("open", () => {
  // Configure voice and enable tools
  ws.send(JSON.stringify({
    type: "session.update",
    session: {
      voice: "eve",
      instructions: "You are a helpful customer support agent.",
      turn_detection: { type: "server_vad" },
      tools: [{ type: "web_search" }]
    }
  }));
});

ws.on("message", (data) => {
  const event = JSON.parse(data);
  if (event.type === "response.output_audio.delta") {
    // Play audio: Buffer.from(event.delta, "base64")
  }
});

演示应用： Web 代理 · Twilio 电话代理 · WebRTC 代理 · iOS 测试应用

文本转语音

使用大量富有表现力的语音将文本转换为口语音频。内联语音标签（笑声、耳语、停顿）和高保真 MP3 到电话 μ-law 等输出格式。支持单次请求或 WebSocket 流式传输。

bash

curl -X POST https://api.x.ai/v1/tts \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Welcome to xAI. How can I help you today?",
    "voice_id": "eve",
    "language": "en"
  }' \
  --output welcome.mp3

python

import os
import requests

response = requests.post(
    "https://api.x.ai/v1/tts",
    headers={
        "Authorization": f"Bearer {os.environ['XAI_API_KEY']}",
        "Content-Type": "application/json",
    },
    json={
        "text": "Welcome to xAI. How can I help you today?",
        "voice_id": "eve",
        "language": "en",
    },
)

with open("welcome.mp3", "wb") as f:
    f.write(response.content)

javascript

import fs from "fs";

const response = await fetch("https://api.x.ai/v1/tts", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.XAI_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    text: "Welcome to xAI. How can I help you today?",
    voice_id: "eve",
    language: "en",
  }),
});

const buffer = Buffer.from(await response.arrayBuffer());
fs.writeFileSync("welcome.mp3", buffer);

实际应用示例： LiveKit · Pipecat

语音转文本

通过一次调用转录音频文件，或通过 WebSocket 进行流式处理。支持 12 种音频格式、词级时间戳、多声道、说话人分离、智能轮流发言结束检测以及 25 种语言。

bash

curl -X POST https://api.x.ai/v1/stt \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -F file=@recording.mp3

python

import os
import requests

response = requests.post(
    "https://api.x.ai/v1/stt",
    headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
    files={"file": ("recording.mp3", open("recording.mp3", "rb"), "audio/mpeg")},
)

print(response.json()["text"])

javascript

import fs from "fs";

const formData = new FormData();
formData.append("file", new Blob([fs.readFileSync("recording.mp3")]), "recording.mp3");

const response = await fetch("https://api.x.ai/v1/stt", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },
  body: formData,
});

const result = await response.json();
console.log(result.text);

实际应用示例： Voximplant

快速入门：自定义语音

从简短的参考片段克隆语音，然后在任何内置语音可用的地方使用生成的 voice_id：

bash

# 1. Create a custom voice from a reference audio clip (max 120s).
curl -X POST https://api.x.ai/v1/custom-voices \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -F "name=Friendly Narrator" \
  -F "language=en" \
  -F "file=@reference.wav;type=audio/wav"

# Response: { "voice_id": "nlbqfwie", ... }

# 2. Use the custom voice for TTS.
curl -X POST https://api.x.ai/v1/tts \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hello! This is my custom voice.",
    "voice_id": "nlbqfwie",
    "language": "en"
  }' \
  --output custom.mp3

python

import os
import requests

# 1. Create a custom voice from a reference audio clip (max 120s).
with open("reference.wav", "rb") as f:
    create = requests.post(
        "https://api.x.ai/v1/custom-voices",
        headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
        files={"file": ("reference.wav", f, "audio/wav")},
        data={"name": "Friendly Narrator", "language": "en"},
    )
voice_id = create.json()["voice_id"]

# 2. Use the custom voice for TTS.
speech = requests.post(
    "https://api.x.ai/v1/tts",
    headers={
        "Authorization": f"Bearer {os.environ['XAI_API_KEY']}",
        "Content-Type": "application/json",
    },
    json={
        "text": "Hello! This is my custom voice.",
        "voice_id": voice_id,
        "language": "en",
    },
)
with open("custom.mp3", "wb") as f:
    f.write(speech.content)

javascript

import fs from "fs";

// 1. Create a custom voice from a reference audio clip (max 120s).
const form = new FormData();
form.append("file", new Blob([fs.readFileSync("reference.wav")]), "reference.wav");
form.append("name", "Friendly Narrator");
form.append("language", "en");

const create = await fetch("https://api.x.ai/v1/custom-voices", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },
  body: form,
});
const { voice_id } = await create.json();

// 2. Use the custom voice for TTS.
const speech = await fetch("https://api.x.ai/v1/tts", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.XAI_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    text: "Hello! This is my custom voice.",
    voice_id,
    language: "en",
  }),
});
fs.writeFileSync("custom.mp3", Buffer.from(await speech.arrayBuffer()));

自定义的 voice_id 也适用于流式 TTS WebSocket 和语音转语音实时 API。完整 API 请参见自定义语音指南。

语音

使用语音转语音 API 或文本转语音时，您可以从完整的内置语音集合中选择。每个语音都有自己的个性和语调，因此请选择最适合您应用的语音（eve 是默认选项）：

企业合规与安全

xAI Voice API 针对具有严格安全和合规要求的生产工作负载而构建。所有音频数据均实时处理，从不存储或用于训练。

SOC 2 Type II — 安全、可用性和保密性的审计控制
符合 HIPAA — 可处理 PHI 的医疗健康应用可签署 BAA
符合 GDPR — 数据处理协议和欧盟数据驻留选项
数据驻留 — 符合合规要求的区域性处理
高可用性 — 企业工作负载的多区域基础设施和自定义 SLA
SSO 与 RBAC — SAML SSO、基于角色的访问和审计日志

模型能力 ​

语音概览 ​

语音转语音 ​

文本转语音 ​

语音转文本 ​

快速入门：自定义语音 ​

语音 ​

企业合规与安全 ​

模型能力

语音概览

语音转语音

文本转语音

语音转文本

快速入门：自定义语音

语音

企业合规与安全