Skip to main content

Message Types

This page documents the WebSocket message interfaces used for real-time communication in the MentraOS SDK.

BaseMessage

The fundamental structure for all messages exchanged within the MentraOS system.

App to Cloud Messages

AppConnectionInit

Message sent by App to initiate connection with cloud.
Note: This message is automatically sent by the SDK when appSession.connect() is called.

AppSubscriptionUpdate

Message sent by App to update its active event subscriptions.
Note: This message is automatically sent by the SDK when appSession.subscribe() is called or when subscription settings change.

DisplayRequest

Message sent from a App to request displaying a layout. Covered in detail in the Layout Types section.
Note: This message is automatically sent by the SDK when using appSession.layouts methods.

DashboardContentUpdate

Message sent from a App to update dashboard content.
Note: This message is automatically sent by the SDK when using session.dashboard.content methods.

Cloud to App Messages

AppConnectionAck

Message sent by cloud to App confirming successful connection and providing initial settings/config.
When this message is received, the SDK fires the onConnected event handler with the settings.

AppConnectionError

Message sent by cloud to App indicating a connection failure.

AppStopped

Message sent by cloud to App indicating the session has been stopped.
When this message is received, the SDK triggers the disconnect process and fires the onDisconnected event handler.

SettingsUpdate

Message sent by cloud to App when the user updates the App’s settings.
When this message is received, the SDK updates its internal settings and fires the onSettingsUpdate event handler.

DataStream

Wrapper message sent by cloud to App carrying data for a subscribed stream.
The SDK unwraps this message and dispatches it to the appropriate event handlers based on the streamType.

DashboardModeChanged

Message sent by cloud to App when the dashboard mode changes.
When this message is received, the SDK fires any registered onModeChange callbacks with the new mode.

DashboardAlwaysOnChanged

Message sent by cloud to App when the always-on dashboard state changes.
When this message is received, the SDK fires any registered always-on change callbacks.

Stream Data Messages

Stream data can either be sent wrapped in a DataStream message or directly as its own message type.

TranscriptionData

Data for real-time speech transcription. See Event Types for details.

TranslationData

Data for real-time speech translation. See Event Types for details.

AudioChunk

Raw audio data chunk. See Event Types for details.

WebSocketError

Structure for reporting WebSocket-specific errors.
When a WebSocket error occurs, the SDK fires the onError event handler with this object.

Message Type Enums

Four enums are used to identify the types of messages exchanged between different components:

AppToCloudMessageType

Message types sent FROM App TO cloud.

CloudToAppMessageType

Message types sent FROM cloud TO App.

GlassesToCloudMessageType

Message types sent FROM glasses TO cloud.

CloudToGlassesMessageType

Message types sent FROM cloud TO glasses.

Type Guards

The SDK provides type guard functions to identify message types:

WebSocket Connection Flow

  1. Initialization:
  2. Authentication:
  3. Subscribing to Streams:
  4. Receiving Data:
    • The cloud sends data for subscribed streams either directly or wrapped in a DataStream message
    • The SDK dispatches this data to the appropriate event handlers
  5. Session Termination:
    • When a session is stopped, the cloud sends an AppStopped message
    • The SDK handles cleanup and fires the onDisconnected event handler