Skip to main content
Camera and streaming features depend on Mentra Live being connected, ready, and permitted to capture. Gate UI by SDK status before exposing photo, video, or stream controls.

​
Photo Upload

Photo upload sends a JPEG from supported glasses to a webhook URL. In the starter apps, the default Camera flow starts a receiver on the phone and sends the photo from the glasses to that phone URL. Turn on Use cloud server only when you want to send the upload to an external endpoint, such as your production API, a staging API, or the optional starter-kit helper. 
const photo = await BluetoothSdk.requestPhoto({
  requestId: `photo-${Date.now()}`,
  appId: 'com.example.app',
  size: 'medium',
  webhookUrl: 'https://api.example.com/mentra/photo',
  authToken: 'optional-token',
  compress: 'medium',
  sound: true,
  exposureTimeNs: null,
  iso: null,
});
console.log('photo delivered', photo.photoUrl ?? photo.uploadUrl);
requestPhoto(...) resolves when the full photo action reaches terminal success: capture completed and the JPEG was delivered to the webhook, either directly from the glasses over Wi-Fi or through the phone’s Bluetooth fallback relay. It rejects or throws when the glasses report a terminal photo_response error, when phone-side fallback upload fails, when the SDK cannot send the command, or when no terminal response arrives before the command timeout. Use photo_status for intermediate progress such as accepted, configuring, capturing, captured, uploading, ble_fallback_compression, ready_for_transfer, and transferring; photo_response is the final success/error result. The returned success includes uploadUrl, and may include webhook-returned metadata such as photoUrl, statusUrl, contentType, or fileSizeBytes. The webhook should accept multipart form data with a photo file and requestId. If authToken is provided, the uploader adds Authorization: Bearer <token>. The camera light is always enabled for photo capture. For one-shot manual capture tuning, pass exposureTimeNs and iso together. exposureTimeNs is sensor exposure time in nanoseconds; iso is sensor ISO. If exposureTimeNs is omitted, null / nil, invalid, or unsupported by the connected glasses, the camera uses auto exposure and ignores iso. Listen for photo_status to render progress and inspect the effective settings the glasses used: 
import {useBluetoothEvent} from '@mentra/bluetooth-sdk/react';

useBluetoothEvent('photo_status', (event) => {
  if (event.status === 'configuring') {
    console.log('effective JPEG config', event.resolvedConfig);
  }
  if (event.status === 'capturing') {
    console.log('requested still config', event.requestedCaptureConfig);
    console.log('latest metered preview', event.meteredPreview);
  }
  if (event.status === 'captured') {
    console.log('actual still capture', event.captureMetadata);
  }
});
photo_status metadata is stage-specific:
StatusOptional metadataWhen to use it
configuringresolvedConfigShow the effective JPEG dimensions, quality, requested size, source, transfer method, compression, and manual exposure fields when present.
capturingrequestedCaptureConfig, meteredPreviewCompare the Camera2 still request with the latest auto-exposure preview estimate before the capture is fired.
capturedcaptureMetadataRead the actual HAL-applied still capture values, including exposure time, ISO, frame duration, AE state/name, noise reduction mode, edge mode, sensor timestamp, and MFNR hint when available.
Upload and BLE transfer statuses such as uploading, compressing, ble_fallback_compression, ready_for_transfer, and transferring are transport progress only. They do not carry capture metadata, so use the captured event when you need the actual exposure/ISO/frame data for debugging or UI. ble_fallback_compression means the direct Wi-Fi/webhook upload failed and the glasses are compressing the already-captured photo for Bluetooth fallback delivery. Mentra Live has gallery mode for the right action button. When gallery mode is enabled, a short press takes a photo, a long press starts video recording, and a short press stops the active video recording. Button and touch events are still reported to your app. 
await BluetoothSdk.setGalleryModeEnabled(true);
await BluetoothSdk.setButtonPhotoSettings('medium');
await BluetoothSdk.setButtonVideoRecordingSettings(1280, 720, 30);
await BluetoothSdk.setButtonMaxRecordingTime(3);
const cameraFov = await BluetoothSdk.setCameraFov({fov: 102, roiPosition: 'center'});
console.log(`Camera ready at ${cameraFov.fov}°`);
Use setGalleryModeEnabled(false) when you want the action button to report events without triggering local gallery capture while the glasses are connected. Action-button photos emitted by the glasses use the same photo_status metadata shape when the phone SDK is connected. These local captures report resolvedConfig.source: "button" and resolvedConfig.transferMethod: "local", then expose requestedCaptureConfig / meteredPreview on capturing and captureMetadata on captured. setCameraFov accepts FOV degrees from 62 to 118 and ROI position "center", "bottom", or "top" on React Native, CameraRoiPosition on Android, and CameraRoiPosition on iOS. React Native also accepts {preset: "narrow" | "standard" | "wide"}; presets map to 82, 102, and 118 degrees with center ROI. On Mentra Live, applying FOV/ROI restarts the camera; the returned CameraFovResult resolves from the ASG client only when the setting has been applied to camera hardware after the restart cooldown, and the promise rejects on glasses-side error, persist-only/no hardware-application ack, or timeout. Treat FOV as a framing/ROI control; output resolution and effective detail can vary by capture path, firmware, and camera mode.

​
Streaming

Streaming sends camera video from supported glasses to an ingest URL. In the starter apps, the default Stream flow uses the phone as the receiver where the platform example supports it. Turn on Use cloud server when you want the glasses to stream to an external RTMP, SRT, or WHIP/WebRTC ingest endpoint. Use your own server in production; the local helper below is only a demo convenience for testing external endpoints from the starter apps. 
const streamId = `stream-${Date.now()}`;

await BluetoothSdk.startStream({
  type: 'start_stream',
  streamUrl: 'http://192.168.1.42:8889/mentra-live/whip',
  streamId,
});

await BluetoothSdk.stopStream();
Use rtmp:// or rtmps:// for RTMP, srt:// for SRT, and http:// or https:// for WHIP/WebRTC ingest. startStream() resolves once glasses report status: "streaming"; stopStream() resolves once glasses report status: "stopped" and also resolves with a normalized stopped event when glasses report that no stream was active. The SDK sends stream keep-alives automatically while streaming and reports keep-alive failures through stream_status. The camera light is always enabled while streaming.

​
Optional Local Demo Helper

The starter kit includes a local helper server so developers can try the Use cloud server path without first deploying their own webhook or streaming service: 
git clone https://github.com/Mentra-Community/Mentra-Bluetooth-SDK-Starter-Kit.git
cd Mentra-Bluetooth-SDK-Starter-Kit
python3 examples/local-demo-cloud/server.py
In the starter apps, leave Use cloud server off for the default glasses-to-phone flow. Turn it on when you want to test an external endpoint, then paste the printed LAN /upload URL into the Camera screen or the printed RTMP, SRT, or WHIP publish URL into the Stream screen. You can skip this helper entirely when you already have reachable upload and streaming endpoints.
Do not use localhost from the mobile app. The glasses, phone, and computer must be on a network where the glasses can reach the printed LAN address.

​
Direct Phone Capture And Streaming

The starter kit examples also include direct phone photo and WebRTC receive flows:
  • Android can receive photos directly on the phone and host a GStreamer WHIP receiver.
  • iOS can receive photos directly on the phone and host a GStreamer WHIP receiver.
  • React Native includes local native companion modules for Android and iOS direct phone photo and WebRTC demos.
Use real hardware for these flows. Keep the glasses Wi-Fi active and make sure the phone and glasses are on a reachable local network. For direct phone photo upload in React Native, start the phone receiver first and pass its uploadUrl as the photo webhook: 
import BluetoothSdk from '@mentra/bluetooth-sdk';
import MentraPhotoReceiver from '@mentra/bluetooth-sdk/photo-receiver';

const receiver = await MentraPhotoReceiver.startPhotoReceiver();

const uploadSubscription = MentraPhotoReceiver.addListener('photoUpload', (event) => {
  console.log(event.requestId, event.fileUri, event.byteCount);
});

try {
  const photo = await BluetoothSdk.requestPhoto({
    requestId: `photo-${Date.now()}`,
    appId: 'com.example.app',
    size: 'medium',
    webhookUrl: receiver.uploadUrl,
    authToken: null,
    compress: 'medium',
    sound: true,
  });
  console.log('photo delivered', photo.photoUrl ?? photo.uploadUrl);
} catch (error) {
  uploadSubscription.remove();
  await MentraPhotoReceiver.stopPhotoReceiver();
  throw error;
}

// Later:
uploadSubscription.remove();
await MentraPhotoReceiver.stopPhotoReceiver();

​
Stream Status

Subscribe to stream status before starting a stream. Treat stream start as accepted, then drive UI from status events: 
import {useBluetoothEvent} from '@mentra/bluetooth-sdk/react';

export function StreamStatusLogger() {
  useBluetoothEvent('stream_status', (event) => {
    console.log(event);
    console.log(event.resolvedConfig?.video?.fps);
  });

  return null;
}
Status values include initializing, streaming, stopped, reconnecting, and error states. When a status event includes resolvedConfig, it contains the effective settings used by the glasses after defaults, clamps, and camera-mode selection. In resolvedConfig.video, width and height are the encoded output dimensions, captureWidth and captureHeight are the native camera buffer dimensions before crop or downscale, bitrate is the encoded video bitrate in bits per second, and fps is the resolved capture/encode frame rate. resolvedConfig.audio reports the audio bitrate, sample rate, echo cancellation, and noise suppression settings when audio is active.