LiveKit Client

Real-time voice communication with avatars using LiveKit

Web Only: RTC Mode is currently available for Web applications only.

Do not call initializeAudioContext() — it is not needed in RTC Mode. Avatar audio is delivered as a native WebRTC audio track by the LiveKit client SDK, not through the AvatarKit SDK’s internal audio player. The AvatarPlayer adapter only feeds animation data to the SDK for rendering; audio playback is handled entirely by LiveKit’s WebRTC stack.

Installation

Critical: We are only compatable with livekit-client under version 2.17, since 2.17 introduced single PC by default. We are working on adding support for 2.17+ in a future release, but for now please ensure you install version 2.16.1 to avoid compatibility issues.

pnpm

pnpm add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1

npm

npm install @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1

yarn

yarn add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1

You also need to configure your build tool for WASM files — see Build Tool Configuration.

Authentication

Credential	How to Obtain	Notes
App ID	SpatialReal Studio → Create App	For SDK initialization
Session Token	Your backend → AvatarKit Server	For avatar loading (max 24 hours)
LiveKit Token	Your backend → LiveKit Server	For RTC room connection

Quick Start

For a complete client + server implementation, see AvatarKit Voice Agent Demo.

1. Initialize SDK in Host Mode

   import { AvatarSDK, AvatarManager, AvatarView, DrivingServiceMode, Environment } from '@spatialwalk/avatarkit'
   
   await AvatarSDK.initialize('your-app-id', {
     environment: Environment.intl,
     drivingServiceMode: DrivingServiceMode.host,  // MUST be host for RTC
   })
   
   AvatarSDK.setSessionToken('your-session-token')
   
2. Load Avatar & Create View

   const avatar = await AvatarManager.shared.load('avatar-id')
   const container = document.getElementById('avatar-container')!
   const avatarView = new AvatarView(avatar, container)
   
3. Create Player with LiveKit Provider

   import { AvatarPlayer, LiveKitProvider } from '@spatialwalk/avatarkit-rtc'
   
   const provider = new LiveKitProvider()
   const player = new AvatarPlayer(provider, avatarView, {
     logLevel: 'warning',
   })
   
4. Connect to LiveKit Server

   await player.connect({
     url: 'wss://your-livekit-server.com',
     token: 'your-livekit-token',
     roomName: 'room-name',
   })
   
5. Start Voice Interaction

   // Start microphone publishing
   await player.startPublishing()
   
   // Stop microphone
   await player.stopPublishing()
   
   // Disconnect when done
   await player.disconnect()
   

AvatarPlayer API

Constructor

new AvatarPlayer(provider: LiveKitProvider, avatarView: AvatarView, options?: AvatarPlayerOptions)

AvatarPlayerOptions

interface AvatarPlayerOptions {
  /** Start speaking transition frames, default 5 (~200ms at 25fps) */
  transitionStartFrameCount?: number

  /** End speaking transition frames, default 40 (~1600ms at 25fps) */
  transitionEndFrameCount?: number

  /** Log level: 'info' | 'warning' | 'error' | 'none', default 'warning' */
  logLevel?: LogLevel
}

Connection

// Connect to LiveKit server
await player.connect(config: LiveKitConnectionConfig)

// Disconnect and clean up
await player.disconnect()

// Reconnect using last config (useful after stalls)
await player.reconnect()

// Check connection status
player.isConnected  // boolean
player.getConnectionState()  // ConnectionState

LiveKitConnectionConfig

interface LiveKitConnectionConfig {
  url: string       // LiveKit server URL (wss://...)
  token: string     // Auth token from your backend
  roomName: string  // Room name
}

Microphone Control

// Start microphone (requests permission automatically)
await player.startPublishing()

// Stop microphone
await player.stopPublishing()

Custom Audio Publishing

For non-microphone audio sources like audio elements or Web Audio API.

// Publish a custom audio track
await player.publishAudio(track: MediaStreamTrack)

// Stop custom audio
await player.unpublishAudio()

Audio Source	How to Obtain Track
<audio> element	audioElement.captureStream().getAudioTracks()[0]
Screen share audio	getDisplayMedia({ audio: true })
Web Audio API	audioContext.createMediaStreamDestination().stream.getAudioTracks()[0]

unpublishAudio() does not stop the track. You are responsible for calling track.stop() to release the resource.

Native Client Access

Access the underlying LiveKit Room instance for advanced features.

import { LiveKitRoom } from '@spatialwalk/avatarkit-rtc'

// Via provider (recommended — full type safety)
const room = provider.getNativeClient()  // LiveKitRoom | null

// Via player (requires type assertion)
const room = player.getNativeClient() as LiveKitRoom | null

console.log('Remote participants:', room?.remoteParticipants.size)

ConnectionState

type ConnectionState = 'disconnected' | 'connecting' | 'connected' | 'reconnecting' | 'failed'

Events

player.on(event: string, handler: Function)
player.off(event: string, handler: Function)

Event	Callback	Description
'connected'	()	Connected to RTC server
'disconnected'	()	Disconnected from RTC server
'error'	(error: Error)	An error occurred
'connection-state-changed'	(state: ConnectionState)	Connection state changed
'stalled'	()	Data stream stalled (no frames for 3s)

When a stalled event fires, the avatar automatically transitions to idle animation. Consider calling player.reconnect() to recover.

Stall recovery example:

player.on('stalled', async () => {
  console.log('Stream stalled, reconnecting...')
  try {
    await player.reconnect()
  } catch (error) {
    console.error('Reconnection failed:', error)
  }
})

Complete Example

import { AvatarPlayer, LiveKitProvider } from '@spatialwalk/avatarkit-rtc'
import { AvatarSDK, AvatarView, AvatarManager, DrivingServiceMode, Environment } from '@spatialwalk/avatarkit'

async function init() {
  // Initialize SDK in host mode
  await AvatarSDK.initialize('your-app-id', {
    environment: Environment.intl,
    drivingServiceMode: DrivingServiceMode.host,
  })
  AvatarSDK.setSessionToken('your-session-token')

  // Load avatar and create view
  const avatar = await AvatarManager.shared.load('character-id')
  const container = document.getElementById('avatar-container')!
  const avatarView = new AvatarView(avatar, container)

  // Create player
  const provider = new LiveKitProvider()
  const player = new AvatarPlayer(provider, avatarView, {
    logLevel: 'info',
    transitionStartFrameCount: 5,
    transitionEndFrameCount: 40,
  })

  // Listen to events
  player.on('connected', () => console.log('Connected!'))
  player.on('disconnected', () => console.log('Disconnected!'))
  player.on('error', (err) => console.error('Error:', err))
  player.on('stalled', async () => {
    console.log('Stream stalled, reconnecting...')
    await player.reconnect()
  })

  // Connect to LiveKit server
  await player.connect({
    url: 'wss://your-livekit-server.com',
    token: 'your-livekit-token',
    roomName: 'my-room',
  })

  // Start microphone
  await player.startPublishing()
}

Browser Compatibility

Browser	Minimum Version	Notes
Chrome	94+	VP8 + RTCRtpScriptTransform
Firefox	117+	RTCRtpScriptTransform required
Safari	15.4+	RTCRtpScriptTransform supported
Edge	94+	Same as Chrome