Class RemotePlayWorld

HTTP client for the Agent Play web UI: session, snapshot RPC, mutating RPC with sid, and optional SSE subscription.

Authenticates like the CLI: x-node-id (derived node id) and x-node-passw (hashed passphrase material) on every request.

Register automation agents with RemotePlayWorld.addAgent (nodeId is the agent node id; the server stores it as agentId). Call RemotePlayWorld.initAudio before addAgent to have the SDK mint per-agent OpenAI Realtime client secrets and forward them on registration.

Incremental updates: RemotePlayWorld.subscribeWorldState listens for playerChainNotify in SSE data, then fetches each changed leaf via RemotePlayWorld.getPlayerChainNode and merges with mergeSnapshotWithPlayerChainNode.

Human→agent intercom (Assist/Chat from the watch UI) is delivered as SSE world:intercom payloads with status forwarded. Call RemotePlayWorld.subscribeIntercomCommands with playerId or playerIds (one SSE stream; routes forwarded commands by toPlayerId) so your process runs tools and posts intercomResponse via RemotePlayWorld.sendIntercomResponse (the subscription does this when executeTool resolves).

Set logging: "on" to trace forwarded commands for subscribed ids and sendIntercomResponse HTTP results.

Index

Constructors

constructor

Methods

addAgent addPlayer close connect getPlayerChainNode getPreviewUrl getSessionId getWorldSnapshot hold initAudio onClose recordInteraction recordJourney registerMcp sendIntercomResponse subscribeIntercomCommands subscribeWorldState

Constructors

constructor

new RemotePlayWorld(options?): RemotePlayWorld
Parameters
- options: RemotePlayWorldOptions = {}
Returns RemotePlayWorld
- Defined in packages/sdk/src/lib/remote-play-world.ts:380

Methods

addAgent

addAgent(input): Promise<RegisteredPlayer>
Registers an automation agent using agent node id (nodeId), sent to the server as agentId.

If initAudio was called and enableP2a is "on", this call mints a per-agent OpenAI Realtime client secret and forwards it as realtimeWebrtc.
Parameters
- input: AddAgentInput
Returns Promise<RegisteredPlayer>
- Defined in packages/sdk/src/lib/remote-play-world.ts:766

addPlayer

addPlayer(input): Promise<RegisteredPlayer>
Parameters
- input: AddPlayerInput
Returns Promise<RegisteredPlayer>
Deprecated
Use addAgent with nodeId (agent node id) for integrations and automation.
- Defined in packages/sdk/src/lib/remote-play-world.ts:910

close

close(): Promise<void>
Returns Promise<void>
- Defined in packages/sdk/src/lib/remote-play-world.ts:553

connect

connect(options?): Promise<void>
Establishes the HTTP session via GET /api/agent-play/session. With RemotePlayWorldConnectOptions.mainNodeId, validates node identity with POST /api/nodes/validate first.
Parameters
- Optionaloptions: RemotePlayWorldConnectOptions
Returns Promise<void>
- Defined in packages/sdk/src/lib/remote-play-world.ts:521

getPlayerChainNode

getPlayerChainNode(stableKey): Promise<PlayerChainNodeResponse>
Fetches one player-chain node (genesis, header, occupant row, or removal) for stableKey, same snapshot scope as RemotePlayWorld.getWorldSnapshot.
Parameters
- stableKey: string
Returns Promise<PlayerChainNodeResponse>
- Defined in packages/sdk/src/lib/remote-play-world.ts:631

getPreviewUrl

getPreviewUrl(): string
Returns string
- Defined in packages/sdk/src/lib/remote-play-world.ts:589

getSessionId

getSessionId(): string
Returns string
- Defined in packages/sdk/src/lib/remote-play-world.ts:582

getWorldSnapshot

getWorldSnapshot(): Promise<AgentPlaySnapshot>
Returns Promise<AgentPlaySnapshot>
- Defined in packages/sdk/src/lib/remote-play-world.ts:606

hold

hold(): RemotePlayWorldHold
Returns RemotePlayWorldHold
- Defined in packages/sdk/src/lib/remote-play-world.ts:441

initAudio

initAudio(options): this
Enables SDK-managed OpenAI Realtime client secret minting for subsequent addAgent calls.

When set and addAgent({ enableP2a: "on" }) is used, the SDK mints a short-lived client secret and includes it in the players registration payload as realtimeWebrtc.
Parameters
- options: RemotePlayWorldInitAudioOptions
Returns this
- Defined in packages/sdk/src/lib/remote-play-world.ts:601

onClose

onClose(handler): (() => void)
Parameters
- handler: (() => void)
  - - (): void
    - Returns void
Returns (() => void)
- - (): void
  - Returns void
- Defined in packages/sdk/src/lib/remote-play-world.ts:434

recordInteraction

recordInteraction(input): Promise<void>
Parameters
- input: RecordInteractionInput
Returns Promise<void>
- Defined in packages/sdk/src/lib/remote-play-world.ts:935

recordJourney

recordJourney(playerId, journey): Promise<void>
Parameters
- playerId: string
- journey: Journey
Returns Promise<void>
- Defined in packages/sdk/src/lib/remote-play-world.ts:943

registerMcp

registerMcp(options): Promise<string>
Parameters
- options: {
  name: string;
  url?: string;
  }
  - name: string
  - Optionalurl?: string
Returns Promise<string>
- Defined in packages/sdk/src/lib/remote-play-world.ts:1187

sendIntercomResponse

sendIntercomResponse(payload): Promise<void>
Parameters
- payload: {
      error?: null | string;
      fromPlayerId: string;
      intercomAddress?: string;
      kind: "assist" | "chat" | "realtime";
      mainNodeId: string;
      message?: string;
      requestId: string;
      result?: Record<string, unknown>;
      status: "stream" | "completed" | "failed";
      toolName?: string;
      toPlayerId: string;
      ts: string;
  }
  - Optionalerror?: null | string
  - fromPlayerId: string
  - OptionalintercomAddress?: string
  - kind: "assist" | "chat" | "realtime"
  - mainNodeId: string
  - Optionalmessage?: string
  - requestId: string
  - Optionalresult?: Record<string, unknown>
  - status: "stream" | "completed" | "failed"
  - OptionaltoolName?: string
  - toPlayerId: string
  - ts: string
Returns Promise<void>
- Defined in packages/sdk/src/lib/remote-play-world.ts:947

subscribeIntercomCommands

subscribeIntercomCommands(options): {
close: (() => void);
}
Subscribes to the session SSE stream and handles forwarded intercom commands whose toPlayerId is in playerId or playerIds, invoking assist/chat execution and posting intercomResponse (completed / failed). Realtime commands mint ephemeral WebRTC credentials in the agent process and return them through intercomResponse. Uses a single SSE connection when playerIds lists multiple automation agents (recommended for several agents in one process). Not invoked automatically by RemotePlayWorld.addAgent.
Parameters
- options: SubscribeIntercomCommandsOptions
Returns {
close: (() => void);
}
- close: (() => void)
  - - (): void
    - Returns void
- Defined in packages/sdk/src/lib/remote-play-world.ts:986

subscribeWorldState

subscribeWorldState(callbacks): {
close: (() => void);
}
Opens the session SSE stream, emits an initial snapshot from RemotePlayWorld.getWorldSnapshot, then on each playerChainNotify merges nodes in deterministic order via RemotePlayWorld.getPlayerChainNode.
Parameters
- callbacks: {
      onError?: ((err: Error) => void);
      onIntercomEvent?: ((payload: {
          channelKey?: string;
          command?: {
              args?: Record<string, unknown>;
              fromPlayerId: string;
              intercomAddress?: string;
              kind: "assist" | "chat" | "realtime";
              mainNodeId: string;
              requestId: string;
              text?: string;
              toolName?: string;
              toPlayerId: string;
          };
          error?: null | string;
          fromPlayerId: string;
          intercomAddress?: string;
          kind: "assist" | "chat" | "realtime";
          mainNodeId: string;
          message?: string;
          requestId: string;
          result?: Record<string, unknown>;
          status:
              | "started"
              | "stream"
              | "completed"
              | "failed"
              | "forwarded";
          toolName?: string;
          toPlayerId: string;
          ts: string;
      }) => void);
      onSnapshot: ((snapshot: AgentPlaySnapshot) => void);
  }
  - OptionalonError?: ((err: Error) => void)
    - (err): void
      Parameters
      err: Error
      Returns void
  - OptionalonIntercomEvent?: ((payload: {
        channelKey?: string;
        command?: {
            args?: Record<string, unknown>;
            fromPlayerId: string;
            intercomAddress?: string;
            kind: "assist" | "chat" | "realtime";
            mainNodeId: string;
            requestId: string;
            text?: string;
            toolName?: string;
            toPlayerId: string;
        };
        error?: null | string;
        fromPlayerId: string;
        intercomAddress?: string;
        kind: "assist" | "chat" | "realtime";
        mainNodeId: string;
        message?: string;
        requestId: string;
        result?: Record<string, unknown>;
        status:
            | "started"
            | "stream"
            | "completed"
            | "failed"
            | "forwarded";
        toolName?: string;
        toPlayerId: string;
        ts: string;
    }) => void)
    - (payload): void
      Parameters
      payload: {
          channelKey?: string;
          command?: {
              args?: Record<string, unknown>;
              fromPlayerId: string;
              intercomAddress?: string;
              kind: "assist" | "chat" | "realtime";
              mainNodeId: string;
              requestId: string;
              text?: string;
              toolName?: string;
              toPlayerId: string;
          };
          error?: null | string;
          fromPlayerId: string;
          intercomAddress?: string;
          kind: "assist" | "chat" | "realtime";
          mainNodeId: string;
          message?: string;
          requestId: string;
          result?: Record<string, unknown>;
          status:
              | "started"
              | "stream"
              | "completed"
              | "failed"
              | "forwarded";
          toolName?: string;
          toPlayerId: string;
          ts: string;
      }
      OptionalchannelKey?: string
      Optionalcommand?: {
          args?: Record<string, unknown>;
          fromPlayerId: string;
          intercomAddress?: string;
          kind: "assist" | "chat" | "realtime";
          mainNodeId: string;
          requestId: string;
          text?: string;
          toolName?: string;
          toPlayerId: string;
      }
      Optionalargs?: Record<string, unknown>
      fromPlayerId: string
      OptionalintercomAddress?: string
      kind: "assist" | "chat" | "realtime"
      mainNodeId: string
      requestId: string
      Optionaltext?: string
      OptionaltoolName?: string
      toPlayerId: string
      Optionalerror?: null | string
      fromPlayerId: string
      OptionalintercomAddress?: string
      kind: "assist" | "chat" | "realtime"
      mainNodeId: string
      Optionalmessage?: string
      requestId: string
      Optionalresult?: Record<string, unknown>
      status:
          | "started"
          | "stream"
          | "completed"
          | "failed"
          | "forwarded"
      OptionaltoolName?: string
      toPlayerId: string
      ts: string
      Returns void
  - onSnapshot: ((snapshot: AgentPlaySnapshot) => void)
    - (snapshot): void
      Parameters
      snapshot: AgentPlaySnapshot
      Returns void
Returns {
close: (() => void);
}
- close: (() => void)
  - - (): void
    - Returns void
- Defined in packages/sdk/src/lib/remote-play-world.ts:660

Class RemotePlayWorld

Index

Constructors

Methods

Constructors

constructor

Parameters

Returns RemotePlayWorld

Methods

addAgent

Parameters

Returns Promise<RegisteredPlayer>

addPlayer

Parameters

Returns Promise<RegisteredPlayer>

Deprecated

close

Returns Promise<void>

connect

Parameters

Returns Promise<void>

getPlayerChainNode

Parameters

Returns Promise<PlayerChainNodeResponse>

getPreviewUrl

Returns string

getSessionId

Returns string

getWorldSnapshot

Returns Promise<AgentPlaySnapshot>

hold

Returns RemotePlayWorldHold

initAudio

Parameters

Returns this

onClose

Parameters

Returns void

Returns (() => void)

Returns void

recordInteraction

Parameters

Returns Promise<void>

recordJourney

Parameters

Returns Promise<void>

registerMcp

Parameters

name: string

Optionalurl?: string

Returns Promise<string>

sendIntercomResponse

Parameters

Optionalerror?: null | string

fromPlayerId: string

OptionalintercomAddress?: string

kind: "assist" | "chat" | "realtime"

mainNodeId: string

Optionalmessage?: string

requestId: string

Optionalresult?: Record<string, unknown>

status: "stream" | "completed" | "failed"

OptionaltoolName?: string

toPlayerId: string

ts: string

Returns Promise<void>

subscribeIntercomCommands

Parameters

Returns { close: (() => void); }

close: (() => void)

Returns void

subscribeWorldState

Parameters

OptionalonError?: ((err: Error) => void)

Parameters

Returns void

Parameters

OptionalchannelKey?: string

Optionalcommand?: { args?: Record<string, unknown>; fromPlayerId: string; intercomAddress?: string; kind: "assist" | "chat" | "realtime"; mainNodeId: string; requestId: string; text?: string; toolName?: string; toPlayerId: string; }

Optionalargs?: Record<string, unknown>

`Optional`url?: string

`Optional`error?: null | string

`Optional`intercomAddress?: string

`Optional`message?: string

`Optional`result?: Record<string, unknown>

`Optional`toolName?: string

Returns {
close: (() => void);
}

`Optional`onError?: ((err: Error) => void)

`Optional`channelKey?: string

`Optional`command?: {
args?: Record<string, unknown>;
fromPlayerId: string;
intercomAddress?: string;
kind: "assist" | "chat" | "realtime";
mainNodeId: string;
requestId: string;
text?: string;
toolName?: string;
toPlayerId: string;
}

`Optional`args?: Record<string, unknown>

`Optional`intercomAddress?: string

`Optional`text?: string

`Optional`toolName?: string

`Optional`error?: null | string

`Optional`intercomAddress?: string

`Optional`message?: string

`Optional`result?: Record<string, unknown>

status:
| "started"
| "stream"
| "completed"
| "failed"
| "forwarded"

`Optional`toolName?: string

Returns {
close: (() => void);
}