Camera Module Reference

The Camera Module provides comprehensive camera functionality, including photo capture and both managed and unmanaged RTMP streaming capabilities. It allows apps to request photos from connected smart glasses and stream live camera feeds using two different approaches.

Overview

The Camera Module is part of the App Session and provides three main capabilities:

Photo Capture: Request individual photos from smart glasses
Managed Streaming: Zero-infrastructure streaming with automatic HLS/DASH URLs
Unmanaged Streaming: Full-control streaming to custom RTMP endpoints

Access the camera module through your app session:

const photo = await session.camera.requestPhoto();
const stream = await session.camera.startManagedStream();
await session.camera.startStream({ rtmpUrl: 'rtmp://example.com/live/key' });

Photo Functionality

requestPhoto()

Request a photo from the connected smart glasses.

async requestPhoto(options?: PhotoRequestOptions): Promise<PhotoData>

Parameters

Parameter	Type	Description
`options`	`PhotoRequestOptions`	Optional configuration for the photo request

PhotoRequestOptions

interface PhotoRequestOptions {
  /** Whether to save the photo to the device gallery */
  saveToGallery?: boolean;
  /** Custom webhook URL to override the TPA's default webhookUrl */
  customWebhookUrl?: string;
  /** Authentication token for custom webhook authentication */
  authToken?: string;
  /** Desired photo size: "small" | "medium" | "large" */
  size?: "small" | "medium" | "large";
  /** Image compression level for upload optimization */
  compress?: "none" | "medium" | "heavy";
}

Compression Options

The compress option controls image compression during webhook upload to optimize network transfer speed. Defaults to "none" (no compression).

Level	Dimensions	JPEG Quality	Typical Size Reduction	Use Case
`"none"`	Original	Original	0% (no compression)	High-quality photos, fast WiFi
`"medium"`	75% scale	80% quality	~50-60% smaller	Balanced quality/speed
`"heavy"`	50% scale	60% quality	~75-85% smaller	Slow connections, large photos

Example: A 1440×1080 pixel photo captured at medium size (typical: ~280KB):

"none": 1440×1080, full quality → ~280KB
"medium": 1080×810, 80% quality → ~140KB (50% smaller)
"heavy": 720×540, 60% quality → ~70KB (75% smaller)

For large size photos (3200×2400, typical: ~800KB):

"none": 3200×2400, full quality → ~800KB
"medium": 2400×1800, 80% quality → ~400KB
"heavy": 1600×1200, 60% quality → ~200KB

Important Notes:

Compression only applies to webhook uploads (WiFi/cellular)
BLE transfers use a different ultra-aggressive compression (AVIF format, 10-15KB) regardless of this setting, controlled by the size parameter
Aspect ratio is always preserved
Uses JPEG format with bilinear scaling for high-quality results

Understanding Webhook vs BLE Compression

MentraOS uses two different compression algorithms depending on the transfer method: Webhook Compression (controlled by compress option):

Format: JPEG
Algorithm: Percentage-based dimension scaling + JPEG quality control
Use case: WiFi/cellular uploads where bandwidth is measured in Mbps
Result: 75-300KB files with good visual quality

BLE Transfer Compression (controlled by size option):

Format: AVIF (with JPEG fallback)
Algorithm: Fit-within-box dimension constraint + ultra-low quality AVIF encoding
Use case: Bluetooth fallback where bandwidth is ~50-100 KB/s
Result: 10-15KB files with acceptable quality for thumbnails

The system automatically selects the appropriate compression:

First attempts webhook upload (uses compress setting)
Falls back to BLE if webhook fails (uses aggressive AVIF compression based on size)

This dual approach ensures photos are delivered either with high quality over WiFi or as compressed thumbnails over BLE, depending on available connectivity.

Returns

Returns a Promise<PhotoData> that resolves with the captured photo data.

PhotoData Interface

interface PhotoData {
  /** The actual photo file as a Buffer */
  buffer: Buffer;
  /** MIME type of the photo (e.g., 'image/jpeg') */
  mimeType: string;
  /** Original filename from the camera */
  filename: string;
  /** Unique request ID that correlates to the original request */
  requestId: string;
  /** Size of the photo in bytes */
  size: number;
  /** Timestamp when the photo was captured */
  timestamp: Date;
}

Example

// Basic photo request (no compression)
const photo = await session.camera.requestPhoto();
console.log(`Photo taken at timestamp: ${photo.timestamp}`);
console.log(`MIME type: ${photo.mimeType}, size: ${photo.size} bytes`);

// Access raw photo data
const photoBuffer = photo.buffer;
const base64Photo = photo.buffer.toString('base64');

// Save to gallery
const photoWithSave = await session.camera.requestPhoto({
  saveToGallery: true
});

// With medium compression (recommended for most use cases)
const compressedPhoto = await session.camera.requestPhoto({
  size: 'large',
  compress: 'medium'  // 75% scale, 80% quality - good balance
});

// With heavy compression (slow connections or very large photos)
const heavilyCompressed = await session.camera.requestPhoto({
  size: 'large',
  compress: 'heavy'  // 50% scale, 60% quality - maximum size reduction
});

// Custom webhook with authentication and compression
const photoWithWebhook = await session.camera.requestPhoto({
  customWebhookUrl: 'https://my-api.example.com/photo-upload',
  authToken: 'bearer-token-or-api-key',
  size: 'large',
  compress: 'medium'  // Compress before uploading to custom endpoint
});

Managed Streaming Functionality (Recommended)

Managed streaming provides zero-infrastructure streaming where the cloud handles all RTMP endpoints and returns HLS/DASH URLs for viewing. Multiple apps can access the same managed stream simultaneously.

startManagedStream()

Start a managed stream with automatic URL generation.

async startManagedStream(options?: ManagedStreamOptions): Promise<ManagedStreamResult>

Parameters

Parameter	Type	Description
`options`	`ManagedStreamOptions`	Optional configuration for the managed stream

ManagedStreamOptions

interface ManagedStreamOptions {
  /** Stream quality preset */
  quality?: '720p' | '1080p';
  /** Enable WebRTC for ultra-low latency viewing */
  enableWebRTC?: boolean;
  /** Optional video configuration settings */
  video?: VideoConfig;
  /** Optional audio configuration settings */
  audio?: AudioConfig;
  /** Optional stream configuration settings */
  stream?: StreamConfig;
  /** Optional RTMP destinations to re-stream to (e.g., YouTube, Twitch) */
  restreamDestinations?: { url: string; name?: string }[];
}

ManagedStreamResult

interface ManagedStreamResult {
  /** HLS URL for viewing the stream */
  hlsUrl: string;
  /** DASH URL for viewing the stream */
  dashUrl: string;
  /** WebRTC URL if enabled */
  webrtcUrl?: string;
  /** Internal stream ID */
  streamId: string;
  /** Cloud-hosted preview/player URL (for quick embedding and testing) */
  previewUrl?: string;
  /** Thumbnail image URL for previews */
  thumbnailUrl?: string;
}

Example

// Basic managed streaming
const result = await session.camera.startManagedStream();
console.log('HLS URL:', result.hlsUrl);
console.log('DASH URL:', result.dashUrl);

// Advanced configuration with WebRTC
const result = await session.camera.startManagedStream({
  quality: '1080p',
  enableWebRTC: true,
  video: { frameRate: 30 },
  audio: { sampleRate: 48000 }
});
console.log('WebRTC URL:', result.webrtcUrl); // Low latency option

stopManagedStream()

Stop the current managed stream.

async stopManagedStream(): Promise<void>

Example

await session.camera.stopManagedStream();

checkExistingStream()

Check if there’s an existing active stream (managed or unmanaged) for the current user.

async checkExistingStream(): Promise<StreamCheckResult>

Returns

Returns a Promise<StreamCheckResult> with information about any active stream.

StreamCheckResult Interface

interface StreamCheckResult {
  /** Whether any stream is currently active */
  hasActiveStream: boolean;
  /** Details about the active stream (if any) */
  streamInfo?: {
    /** Type of stream - either 'managed' or 'unmanaged' */
    type: 'managed' | 'unmanaged';
    /** Unique identifier for the stream */
    streamId: string;
    /** Current status of the stream */
    status: string;
    /** When the stream was created */
    createdAt: Date;
    // For managed streams only:
    /** HLS playback URL */
    hlsUrl?: string;
    /** DASH playback URL */
    dashUrl?: string;
    /** WebRTC playback URL (if enabled) */
    webrtcUrl?: string;
    /** Preview/embed URL */
    previewUrl?: string;
    /** Thumbnail image URL */
    thumbnailUrl?: string;
    /** Number of active viewers */
    activeViewers?: number;
    // For unmanaged streams only:
    /** RTMP destination URL */
    rtmpUrl?: string;
    /** ID of the app that requested the stream */
    requestingAppId?: string;
  };
}

Example

// Check for existing streams before starting a new one
const streamCheck = await session.camera.checkExistingStream();

if (streamCheck.hasActiveStream) {
  console.log(`Found existing ${streamCheck.streamInfo?.type} stream`);

  if (streamCheck.streamInfo?.type === 'managed') {
    // Managed stream is active - can display URLs or join it
    console.log('HLS URL:', streamCheck.streamInfo.hlsUrl);
    console.log('Active viewers:', streamCheck.streamInfo.activeViewers);

    // Option 1: Display the existing stream info
    session.layouts.showTextWall(`Stream active.\nHLS: ${streamCheck.streamInfo.hlsUrl}`);

    // Option 2: Join the existing managed stream
    // (Managed streams support multiple apps)
    const result = await session.camera.startManagedStream();
  } else {
    // Unmanaged stream is active - cannot start another
    console.log('RTMP URL:', streamCheck.streamInfo.rtmpUrl);
    console.log('Requested by:', streamCheck.streamInfo.requestingAppId);

    session.layouts.showTextWall('⚠️ Another app is currently streaming');
  }
} else {
  // No active stream, safe to start a new one
  console.log('No existing stream, starting new...');
  const result = await session.camera.startManagedStream();
}

Use Cases

Reconnection: Apps can check for and reconnect to existing streams after restart
Multi-app coordination: Detect if another app is already streaming
Resource management: Avoid creating duplicate streams
User experience: Inform users about existing streams before starting new ones

Managed Stream Status Monitoring

onManagedStreamStatus()

Subscribe to managed stream status updates.

onManagedStreamStatus(handler: (status: ManagedStreamStatus) => void): () => void

ManagedStreamStatus

interface ManagedStreamStatus {
  type: string;
  status: 'initializing' | 'preparing' | 'active' | 'stopping' | 'stopped' | 'error';
  hlsUrl?: string;
  dashUrl?: string;
  webrtcUrl?: string;
  message?: string;
  streamId?: string;
  timestamp: Date;
}

Example

// Monitor managed stream status
const unsubscribe = session.camera.onManagedStreamStatus((status) => {
  console.log(`Managed stream status: ${status.status}`);

  if (status.status === 'active') {
    console.log('Stream is live!');
    console.log(`HLS URL: ${status.hlsUrl}`);
    console.log(`DASH URL: ${status.dashUrl}`);
  } else if (status.status === 'error') {
    console.error(`Stream error: ${status.message}`);
  }
});

// Later, unsubscribe
unsubscribe();

Unmanaged RTMP Streaming Functionality

Unmanaged streaming provides full control over RTMP endpoints but requires your own streaming infrastructure. Only one unmanaged stream can be active at a time, and it blocks other apps from using the camera.

startStream()

Start an unmanaged RTMP stream to a specific URL.

async startStream(options: RtmpStreamOptions): Promise<void>

Parameters

Parameter	Type	Description
`options`	`RtmpStreamOptions`	Configuration options for the stream

RtmpStreamOptions

interface RtmpStreamOptions {
  /** The RTMP URL to stream to (e.g., rtmp://server.example.com/live/stream-key) */
  rtmpUrl: string;
  /** Optional video configuration settings */
  video?: VideoConfig;
  /** Optional audio configuration settings */
  audio?: AudioConfig;
  /** Optional stream configuration settings */
  stream?: StreamConfig;
}

VideoConfig

interface VideoConfig {
  /** Optional width in pixels (e.g., 1280) */
  width?: number;
  /** Optional height in pixels (e.g., 720) */
  height?: number;
  /** Optional bitrate in bits per second (e.g., 2000000 for 2 Mbps) */
  bitrate?: number;
  /** Optional frame rate in frames per second (e.g., 30) */
  frameRate?: number;
}

AudioConfig

interface AudioConfig {
  /** Optional audio bitrate in bits per second (e.g., 128000 for 128 kbps) */
  bitrate?: number;
  /** Optional audio sample rate in Hz (e.g., 44100) */
  sampleRate?: number;
  /** Optional flag to enable echo cancellation */
  echoCancellation?: boolean;
  /** Optional flag to enable noise suppression */
  noiseSuppression?: boolean;
}

StreamConfig

interface StreamConfig {
  /** Optional maximum duration in seconds (e.g., 1800 for 30 minutes) */
  durationLimit?: number;
}

Example

// Basic streaming
await session.camera.startStream({
  rtmpUrl: 'rtmp://live.example.com/stream/key'
});

// Advanced configuration
await session.camera.startStream({
  rtmpUrl: 'rtmp://live.example.com/stream/key',
  video: {
    width: 1920,
    height: 1080,
    bitrate: 5000000,
    frameRate: 30
  },
  audio: {
    bitrate: 128000,
    sampleRate: 44100,
    echoCancellation: true,
    noiseSuppression: true
  },
  stream: {
    durationLimit: 1800 // 30 minutes
  }
});

stopStream()

Stop the current unmanaged RTMP stream.

async stopStream(): Promise<void>

Example

await session.camera.stopStream();

Unmanaged Stream Status Monitoring

onStreamStatus()

Subscribe to unmanaged stream status updates.

onStreamStatus(handler: StreamStatusHandler): () => void

StreamStatusHandler

type StreamStatusHandler = (status: RtmpStreamStatus) => void;

RtmpStreamStatus

interface RtmpStreamStatus {
  type: string;
  streamId?: string;
  status: 'initializing' | 'connecting' | 'reconnecting' | 'streaming' | 'error' | 'stopped' | 'active' | 'stopping' | 'disconnected' | 'timeout';
  errorDetails?: string;
  appId?: string;
  stats?: {
    bitrate: number;
    fps: number;
    droppedFrames: number;
    duration: number;
  };
  timestamp: Date;
}

Example

// Monitor unmanaged stream status
const unsubscribe = session.camera.onStreamStatus((status) => {
  console.log(`Stream status: ${status.status}`);

  if (status.status === 'active') {
    console.log('Stream is live!');
    if (status.stats) {
      console.log(`Bitrate: ${status.stats.bitrate} bps`);
      console.log(`FPS: ${status.stats.fps}`);
      console.log(`Duration: ${status.stats.duration}s`);
      console.log(`Dropped frames: ${status.stats.droppedFrames}`);
    }
  } else if (status.status === 'error') {
    console.error(`Stream error: ${status.errorDetails}`);
  }
});

// Later, unsubscribe
unsubscribe();

Unmanaged Stream Utility Methods

isCurrentlyStreaming()

Check if currently streaming (unmanaged).

isCurrentlyStreaming(): boolean

getCurrentStreamUrl()

Get the URL of the current unmanaged stream.

getCurrentStreamUrl(): string | undefined

getStreamStatus()

Get the current unmanaged stream status.

getStreamStatus(): RtmpStreamStatus | undefined

Streaming Comparison

Feature	Managed Streaming	Unmanaged Streaming
Infrastructure Required	None	RTMP Server
Multiple Apps Can Stream	✅ Yes	❌ No (Exclusive)
Blocks Other Apps	❌ No	✅ Yes
Viewer URLs Provided	✅ HLS/DASH/WebRTC	❌ You manage
Best For	Social media, prototypes, multi-app scenarios	Custom servers, exclusive access
Requires Internet	Yes	No

Error Handling

Photo Errors

Timeout: Photo requests timeout after 30 seconds
Cancellation: Requests can be cancelled manually or during session cleanup
Device Errors: Camera unavailable or hardware issues

Unmanaged Stream Errors

Already Streaming: Cannot start while any stream (managed or unmanaged) is active
Invalid URL: RTMP URL validation failures
Network Issues: Connection problems to RTMP endpoint
Device Limitations: Hardware doesn’t support requested configuration

Getting Started

Core Concepts

SDK Reference

​Camera Module Reference

​Overview

​Photo Functionality

​requestPhoto()

​Parameters

​PhotoRequestOptions

Compression Options

Understanding Webhook vs BLE Compression

​Returns

​PhotoData Interface

​Example

​Managed Streaming Functionality (Recommended)

​startManagedStream()

​Parameters

​ManagedStreamOptions

​ManagedStreamResult

​Example

​stopManagedStream()

​Example

​checkExistingStream()

​Returns

​StreamCheckResult Interface

​Example

​Use Cases

​Managed Stream Status Monitoring

​onManagedStreamStatus()

​ManagedStreamStatus

​Example

​Unmanaged RTMP Streaming Functionality

​startStream()

​Parameters

​RtmpStreamOptions

​VideoConfig

​AudioConfig

​StreamConfig

​Example

​stopStream()

​Example

​Unmanaged Stream Status Monitoring

​onStreamStatus()

​StreamStatusHandler

​RtmpStreamStatus

​Example

​Unmanaged Stream Utility Methods

​isCurrentlyStreaming()

​getCurrentStreamUrl()

​getStreamStatus()

​Streaming Comparison

​Error Handling

​Photo Errors

​Unmanaged Stream Errors

Camera Module Reference

Overview

Photo Functionality

requestPhoto()

Parameters

PhotoRequestOptions

Returns

PhotoData Interface

Example

Managed Streaming Functionality (Recommended)

startManagedStream()

Parameters

ManagedStreamOptions

ManagedStreamResult

Example

stopManagedStream()

Example

checkExistingStream()

Returns

StreamCheckResult Interface

Example

Use Cases

Managed Stream Status Monitoring

onManagedStreamStatus()

ManagedStreamStatus

Example

Unmanaged RTMP Streaming Functionality

startStream()

Parameters

RtmpStreamOptions

VideoConfig

AudioConfig

StreamConfig

Example

stopStream()

Example

Unmanaged Stream Status Monitoring

onStreamStatus()

StreamStatusHandler

RtmpStreamStatus

Example

Unmanaged Stream Utility Methods

isCurrentlyStreaming()

getCurrentStreamUrl()

getStreamStatus()

Streaming Comparison

Error Handling

Photo Errors

Unmanaged Stream Errors