Transcription

Mentra Miniapp SDK betaThe SDK is in beta, so its APIs may change before general availability.There is currently no way to distribute a miniapp built with the Miniapp SDK. In a future release, developers will upload miniapps through the Mentra Developer Console, and users will download them from the Mentra Miniapp Store. Neither service supports Miniapp SDK distribution in MentraOS 3.0.Only use the Miniapp SDK if you are comfortable with these limitations.Developing on Mentra Live? We recommend using the Mentra Bluetooth SDK.To enable miniapp development, open Settings in the Mentra App and tap the version number at the bottom ten times. Then open Debug Settings, turn on Miniapp Developer Settings, return to Settings, and open Miniapp Developer Settings.Share feedback with an in-app bug report, on Discord, or by email at help@mentra.glass.

session.transcription turns what the user says into text. Register a handler and you start getting results; the SDK handles the subscription for you.

session.transcription.on((data) => {
  if (data.isFinal) {
    session.display.render([{type: "text", id: "msg", box: {x: 0, y: 0, w: 576, h: 288}, text: data.text}]);
  }
});

Transcription needs the MICROPHONE permission in your manifest. Without it, the phone rejects the subscription with PERMISSION_NOT_DECLARED and no events arrive.

Results

Each event is a TranscriptionData:

Field	Type	Notes
`text`	`string`	The transcribed text so far.
`isFinal`	`boolean`	`false` for interim results that keep updating, `true` once the utterance is settled.
`language`	`string?`	The detected language (when auto-detecting).

Interim results stream as the user speaks and get replaced; wait for isFinal if you only want settled text.

Targeting languages

on() auto-detects the language. To listen for specific ones, use forLanguage() with BCP-47 tags:

// One language.
session.transcription.forLanguage("en-US", (data) => { /* … */ });

// Several at once.
session.transcription.forLanguage(["en-US", "es-ES"], (data) => { /* … */ });

Each call is independent, and you can have several active at the same time.

Requiring on-device transcription

Pass {forceLocal: true} as the final argument to route that subscription through on-device STT and suppress cloud transcript delivery:

session.transcription.on(handler, {forceLocal: true});
session.transcription.forLanguage("en-US", handler, {forceLocal: true});

This requires a downloaded local STT model. Cloud transcripts are suppressed for the local-only subscription, even while other miniapps continue using cloud transcription. Routing is per listener, so a default listener on the same stream can continue receiving cloud transcription.

Configuring

configure() tunes the recognizer. It’s fire-and-forget and applies to subsequent results:

session.transcription.configure({
  languageHints: ["en", "ja"],     // ISO 639-1 hints to improve detection
  vocabulary: ["MentraOS", "HIPAA"], // boost domain terms
  diarization: true,                // tag speakers when the provider supports it
});

Cleaning up

Every subscription returns an unsubscribe function. stop() tears down every subscription this module owns at once:

const off = session.transcription.on(handler);
off();                       // drop this one

session.transcription.stop(); // drop all of them

session.transcription.hasPermission tells you whether MICROPHONE is declared in your manifest. It does not tell you whether the user granted the OS prompt: if they denied it, your handler simply never fires. See Permissions.

Transcription is for text. For raw audio frames or voice-activity detection, use session.mic. For translated text, use session.translation. Subscribing to one does not subscribe you to the others.

Getting Started

Core Concepts

UI Layer

Display

Audio & Speech

Sensors & Input

Camera

Phone & System

Data

Reference

Results

Targeting languages

Requiring on-device transcription

Configuring

Cleaning up

​Results

​Targeting languages

​Requiring on-device transcription

​Configuring

​Cleaning up

Results

Targeting languages

Requiring on-device transcription

Configuring

Cleaning up