> ## Documentation Index
> Fetch the complete documentation index at: https://spatialwalk-6179c111.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Integration Modes

> Choose the right integration approach for your use case

## Choose Your Integration Mode

SpatialReal offers four distinct integration modes to suit different architectural needs, latency requirements, and development preferences.

> **[SDK Mode](#sdk-mode)**
>
> Client-centric integration with minimal server-side changes

> **[RTC Mode](#rtc-mode)**
>
> Ultra-low latency via LiveKit or Agora

> **[Framework Plugin](#framework-plugin)**
>
> Seamless integration with LiveKit Agents or TEN Framework

> **[Host Mode](#host-mode)**
>
> Full control with custom transport layer

***

## SDK Mode

In this mode, the client-side application manages the audio input. The developer passes the audio to the SpatialReal Client SDK, which handles the server interaction to retrieve animation data and render the avatar.

**1. Pass Audio**

Developer passes audio to the SpatialReal SDK on the client.

**2. Inference**

SDK calls the inference service.

**3. Render**

SDK receives drive parameters and plays the avatar.

**Best Suited For:**

* **Client-Centric Logic:** Scenarios where the voice agent logic resides primarily on the device.
* **Moderate Latency:** Projects where ultra-low latency is not the absolute priority.
* **Simplified Architecture:** Minimal server-side development required (only for authentication), allowing most logic to remain on the client.

```mermaid actions={false} theme={null}
---
config:
  "look": "handDrawn"
  "theme": "base"
  "themeVariables":
    "background": "#ffffff"
    "textColor": "#111827"
    "lineColor": "#64748b"
    "primaryColor": "#e8f4fd"
    "primaryTextColor": "#111827"
    "primaryBorderColor": "#2196F3"
    "secondaryColor": "#f3e5f5"
    "secondaryTextColor": "#111827"
    "secondaryBorderColor": "#9C27B0"
    "tertiaryColor": "#fff3e0"
    "tertiaryTextColor": "#111827"
    "tertiaryBorderColor": "#FF9800"
    "clusterBkg": "#f8fafc"
    "clusterBorder": "#cbd5e1"
    "edgeLabelBackground": "#ffffff"
    "actorBkg": "#e8f4fd"
    "actorBorder": "#2196F3"
    "actorTextColor": "#111827"
    "noteBkgColor": "#fff7ed"
    "noteTextColor": "#111827"
    "signalColor": "#64748b"
    "signalTextColor": "#111827"
---
flowchart LR
    subgraph Client ["🖥️ Client"]
        App["Client App (Voice Logic)"]
        SDK["SpatialReal SDK"]
    end

    Cloud["☁️ SpatialReal Cloud"]

    App -- "Audio Data" --> SDK
    SDK -- "Send Audio" --> Cloud
    Cloud -- "Drive Params" --> SDK
    SDK -- "Render & Play" --> App

    style Client fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#000
    style Cloud fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#000
```

> **[View SDK Mode Guide](https://spatialwalk-6179c111.mintlify.site/guide/sdk-mode.md)**

***

## RTC Mode

This mode leverages Real-Time Communication (RTC) infrastructure (currently supporting LiveKit and Agora). The developer streams audio to SpatialReal, but instead of returning data to the developer, SpatialReal pushes the drive parameters directly into an RTC room/channel.

**1. Stream Audio**

Developer streams audio via Server SDK.

**2. Push to RTC**

SpatialReal Service pushes audio and avatar drive data to an RTC Room.

**3. Subscribe & Play**

Client joins the room using the SpatialReal RTC Client to subscribe, parse, and play.

> [!NOTE]
>
> The stream contains **binary drive parameters and audio**, not a pre-rendered video feed. Therefore, it cannot be played by standard video players; it requires the SpatialReal Client to render.

**Best Suited For:**

* **Existing RTC Users:** Teams already using LiveKit or Agora for voice agents but *not* using their specific agent frameworks (e.g., LiveKit Agents or TEN Framework).
* **Server-Managed State:** Scenarios requiring server-side management of conversation state (e.g., handling interruptions).
* **Ultra-Low Latency:** Leveraging established RTC networks for minimal delay.

```mermaid actions={false} theme={null}
---
config:
  "look": "handDrawn"
  "theme": "base"
  "themeVariables":
    "background": "#ffffff"
    "textColor": "#111827"
    "lineColor": "#64748b"
    "primaryColor": "#e8f4fd"
    "primaryTextColor": "#111827"
    "primaryBorderColor": "#2196F3"
    "secondaryColor": "#f3e5f5"
    "secondaryTextColor": "#111827"
    "secondaryBorderColor": "#9C27B0"
    "tertiaryColor": "#fff3e0"
    "tertiaryTextColor": "#111827"
    "tertiaryBorderColor": "#FF9800"
    "clusterBkg": "#f8fafc"
    "clusterBorder": "#cbd5e1"
    "edgeLabelBackground": "#ffffff"
    "actorBkg": "#e8f4fd"
    "actorBorder": "#2196F3"
    "actorTextColor": "#111827"
    "noteBkgColor": "#fff7ed"
    "noteTextColor": "#111827"
    "signalColor": "#64748b"
    "signalTextColor": "#111827"
---
flowchart BT
    subgraph Server ["🔧 Developer Backend"]
        DevServer["Server Logic"]
        ServerSDK["SpatialReal Server SDK"]
        DevServer --> ServerSDK
    end

    subgraph Cloud ["☁️ SpatialReal Cloud"]
        Service["Avatar Service"]
        RTCPub["RTC Egress"]
        Service --> RTCPub
    end

    subgraph RTC ["📡 RTC Provider"]
        Room["LiveKit Room / Agora Channel"]
    end

    subgraph Client ["🖥️ Client"]
        RTCClient["SpatialReal RTC Client"]
    end

    ServerSDK -->|Audio Stream| Service
    RTCPub -->|Drive Params + Audio| Room
    Room -->|Subscribe| RTCClient

    style Client fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#000
    style RTC fill:#f3e5f5,stroke:#9C27B0,stroke-width:2px,color:#000
    style Cloud fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#000
    style Server fill:#e8eaf6,stroke:#3F51B5,stroke-width:2px,color:#000
```

> **[View RTC Mode Guide](https://spatialwalk-6179c111.mintlify.site/guide/rtc-mode.md)**

***

## Framework Plugin

This is the most streamlined approach for modern voice agent frameworks. Developers use a provided plugin that sits inside the voice agent pipeline (e.g., LiveKit Agents, TEN Framework).

**1. Intercept Audio**

The Plugin intercepts audio from the agent pipeline.

**2. Send to SpatialReal**

It sends audio to SpatialReal and configures the RTC push parameters.

**3. Push & Render**

SpatialReal pushes data to the RTC room (Client side is identical to RTC Mode).

**Best Suited For:**

* **Framework Users:** Teams already using frameworks like LiveKit Agents or TEN Framework.
* **Rapid Integration:** Low implementation cost; the plugin automatically handles signal processing and conversation state (interruption logic).
* **Migration:** Easy to switch if you are currently using other avatar services within these frameworks.

```mermaid actions={false} theme={null}
---
config:
  "look": "handDrawn"
  "theme": "base"
  "themeVariables":
    "background": "#ffffff"
    "textColor": "#111827"
    "lineColor": "#64748b"
    "primaryColor": "#e8f4fd"
    "primaryTextColor": "#111827"
    "primaryBorderColor": "#2196F3"
    "secondaryColor": "#f3e5f5"
    "secondaryTextColor": "#111827"
    "secondaryBorderColor": "#9C27B0"
    "tertiaryColor": "#fff3e0"
    "tertiaryTextColor": "#111827"
    "tertiaryBorderColor": "#FF9800"
    "clusterBkg": "#f8fafc"
    "clusterBorder": "#cbd5e1"
    "edgeLabelBackground": "#ffffff"
    "actorBkg": "#e8f4fd"
    "actorBorder": "#2196F3"
    "actorTextColor": "#111827"
    "noteBkgColor": "#fff7ed"
    "noteTextColor": "#111827"
    "signalColor": "#64748b"
    "signalTextColor": "#111827"
---
flowchart BT
    subgraph Agent ["🤖 Voice Agent"]
        Pipeline["Agent Pipeline (LiveKit / TEN)"]
        Plugin["SpatialReal Plugin"]
    end

    subgraph Cloud ["☁️ SpatialReal Cloud"]
        Service["Inference Service"]
        RTCPub["RTC Egress"]
    end

    subgraph RTC ["📡 RTC Provider"]
        Room["LiveKit Room / Agora Channel"]
    end

    subgraph Client ["🖥️ Client"]
        RTCClient["SpatialReal RTC Client"]
    end

    Pipeline -. Audio .-> Plugin
    Plugin -- Audio & Config --> Service
    Service --> RTCPub
    RTCPub -- RTC Push --> Room
    Room -- Subscribe --> RTCClient

    style Client fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#000
    style RTC fill:#f3e5f5,stroke:#9C27B0,stroke-width:2px,color:#000
    style Cloud fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#000
    style Agent fill:#e8eaf6,stroke:#3F51B5,stroke-width:2px,color:#000
```

> **[View Framework Plugin Guide](https://spatialwalk-6179c111.mintlify.site/guide/rtc-mode.md)**

***

## Host Mode

In Host Mode, the developer acts as the bridge. You use the SpatialReal Server SDK to stream audio to the service and receive streaming drive parameters back. It is then your responsibility to transport this data to the client.

**1. Stream Audio**

Developer sends streaming audio via Server SDK.

**2. Receive Parameters**

SpatialReal returns streaming drive parameters to the Developer's Server.

**3. Transport**

Developer transmits audio and parameters to the Client SDK via a custom transport layer.

**4. Render**

Client SDK renders the avatar.

> [!WARNING]
>
> The custom transport layer must ensure data is delivered **without duplication, loss, or disorder**.

**Best Suited For:**

* **Custom Transport:** Teams that already have a reliable, controllable transport layer.
* **Deep Integration:** Developers willing to handle server-side adaptation for maximum control.
* **High Low-Latency Demands:** When you need to optimize the network path manually.

```mermaid actions={false} theme={null}
---
config:
  "look": "handDrawn"
  "theme": "base"
  "themeVariables":
    "background": "#ffffff"
    "textColor": "#111827"
    "lineColor": "#64748b"
    "primaryColor": "#e8f4fd"
    "primaryTextColor": "#111827"
    "primaryBorderColor": "#2196F3"
    "secondaryColor": "#f3e5f5"
    "secondaryTextColor": "#111827"
    "secondaryBorderColor": "#9C27B0"
    "tertiaryColor": "#fff3e0"
    "tertiaryTextColor": "#111827"
    "tertiaryBorderColor": "#FF9800"
    "clusterBkg": "#f8fafc"
    "clusterBorder": "#cbd5e1"
    "edgeLabelBackground": "#ffffff"
    "actorBkg": "#e8f4fd"
    "actorBorder": "#2196F3"
    "actorTextColor": "#111827"
    "noteBkgColor": "#fff7ed"
    "noteTextColor": "#111827"
    "signalColor": "#64748b"
    "signalTextColor": "#111827"
---
flowchart LR
    subgraph Client ["🖥️ Client"]
        ClientSDK["SpatialReal Client SDK"]
    end

    subgraph Server ["🔧 Developer Backend"]
        DevServer["Server Logic"]
        ServerSDK["SpatialReal Server SDK"]
    end

    Cloud["☁️ SpatialReal Cloud"]

    DevServer -- "Audio Stream" --> ServerSDK
    ServerSDK -- "Send Audio" --> Cloud
    Cloud -- "Drive Params" --> ServerSDK
    ServerSDK -- "Animation Stream" --> DevServer
    DevServer == "Custom Transport: Audio + Animation" ==> ClientSDK

    style Client fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#000
    style Server fill:#e8eaf6,stroke:#3F51B5,stroke-width:2px,color:#000
    style Cloud fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#000
```

> **[View Host Mode Guide](https://spatialwalk-6179c111.mintlify.site/guide/host-mode.md)**

***

## Still Not Sure?

Check integration overview for a side-by-side comparison and runnable example projects.

> **[Open Integrations Overview](https://spatialwalk-6179c111.mintlify.site/overview/integration_overview.md)**

***

## Want a Quick Taste?

Check this quick start guide to get a simple demo up and running in minutes.

> **[Open Quickstart Guide](https://spatialwalk-6179c111.mintlify.site/overview/speech-to-avatar.md)**

***

## Next Steps

> **[SDK Mode Guide](https://spatialwalk-6179c111.mintlify.site/guide/sdk-mode.md)**
>
> Get started with the simplest integration approach

> **[Host Mode Guide](https://spatialwalk-6179c111.mintlify.site/guide/host-mode.md)**
>
> Learn how to implement custom transport

> **[RTC Mode & Framework Plugin Guide](https://spatialwalk-6179c111.mintlify.site/guide/rtc-mode.md)**
>
> Set up real-time communication with LiveKit or Agora

> **[Demo Projects](https://spatialwalk-6179c111.mintlify.site/overview/demo-projects.md)**
>
> Explore sample implementations


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.