Playing Audio Files

Play audio files from URLs using session.audio.playAudio(). Stream sound effects, music, or pre-recorded messages to the user’s device.

Basic Usage

const result = await session.audio.playAudio({
  audioUrl: 'https://example.com/sound.mp3'
});

if (result.success) {
  session.logger.info('Audio played successfully');
}

How It Works

Provide a URL to an audio file
MentraOS downloads and streams the file
Audio plays through glasses speakers or phone
Promise resolves when playback completes

Supported Formats

MP3 - Most common, recommended
WAV - Uncompressed, larger files
OGG - Open format, good compression
M4A - Apple format

MP3 format is recommended for best compatibility and file size.

Audio Options

Volume Control

await session.audio.playAudio({
  audioUrl: 'https://example.com/notification.mp3',
  volume: 0.5  // 0.0 - 1.0
});

Stop Other Audio

await session.audio.playAudio({
  audioUrl: 'https://example.com/alert.mp3',
  stopOtherAudio: true  // Default: true
});

Options Reference

Option	Type	Default	Description
`audioUrl`	string	required	URL to audio file
`volume`	number	1.0	Volume level (0.0-1.0)
`stopOtherAudio`	boolean	true	Stop currently playing audio

Common Patterns

Sound Effects

async function playClickSound(session: AppSession) {
  await session.audio.playAudio({
    audioUrl: 'https://cdn.example.com/sounds/click.mp3',
    volume: 0.8
  });
}

// Use on button press
session.events.onButtonPress(async (data) => {
  await playClickSound(session);
  // Handle button action
});

Notification Sounds

const SOUNDS = {
  success: 'https://cdn.example.com/success.mp3',
  error: 'https://cdn.example.com/error.mp3',
  warning: 'https://cdn.example.com/warning.mp3'
};

async function notify(session: AppSession, type: keyof typeof SOUNDS) {
  await session.audio.playAudio({
    audioUrl: SOUNDS[type],
    volume: 0.7
  });
}

// Usage
await notify(session, 'success');

Background Music

protected async onSession(session: AppSession, sessionId: string, userId: string) {
  // Play ambient background music
  await session.audio.playAudio({
    audioUrl: 'https://cdn.example.com/ambient.mp3',
    volume: 0.3
  });
}

Error Handling

async function playAudioSafe(session: AppSession, url: string) {
  try {
    const result = await session.audio.playAudio({
      audioUrl: url
    });

    if (!result.success) {
      session.logger.error('Audio failed:', result.error);
      // Fallback to TTS
      await session.audio.speak('Audio notification');
    }
  } catch (error) {
    session.logger.error('Audio error:', error);
  }
}

Sequential Audio

async function playSequence(session: AppSession) {
  // Play sounds in sequence
  await session.audio.playAudio({
    audioUrl: 'https://cdn.example.com/beep1.mp3'
  });

  await session.audio.playAudio({
    audioUrl: 'https://cdn.example.com/beep2.mp3'
  });

  await session.audio.playAudio({
    audioUrl: 'https://cdn.example.com/beep3.mp3'
  });
}

Conditional Audio

session.events.onTranscription(async (data) => {
  if (!data.isFinal) return;

  const text = data.text.toLowerCase();

  if (text.includes('success')) {
    await session.audio.playAudio({
      audioUrl: 'https://cdn.example.com/success.mp3'
    });
  } else if (text.includes('error')) {
    await session.audio.playAudio({
      audioUrl: 'https://cdn.example.com/error.mp3'
    });
  }
});

Stopping Audio

Stop all currently playing audio:

session.audio.stopAudio();

Stop before playing new audio:

// Automatically stops other audio (default behavior)
await session.audio.playAudio({
  audioUrl: 'https://example.com/new-sound.mp3',
  stopOtherAudio: true
});

// Allow overlap
await session.audio.playAudio({
  audioUrl: 'https://example.com/background.mp3',
  stopOtherAudio: false
});

Best Practices

Use CDN for Audio Files

Host audio on a CDN for fast, reliable access:

// ✅ Good - CDN URL
audioUrl: 'https://cdn.example.com/sounds/click.mp3'

// ❌ Avoid - slow origin server
audioUrl: 'https://slow-server.com/large-file.wav'

Keep Files Small

Smaller files = faster loading = better UX:

// ✅ Good - short sound effect
// click.mp3 - 50KB, 0.5 seconds

// ❌ Avoid - huge file for simple sound
// click.wav - 5MB, 0.5 seconds

Handle Failures Gracefully

Always check the result:

const result = await session.audio.playAudio({ audioUrl: url });

if (!result.success) {
  // Fallback: TTS or visual feedback
  await session.audio.speak('Notification');
  // or
  session.layouts.showTextWall('🔔 Alert');
}

Provide Visual Feedback

Combine audio with visual cues:

session.layouts.showTextWall('✓ Success');
await session.audio.playAudio({
  audioUrl: 'https://cdn.example.com/success.mp3'
});

Audio Routing

Audio automatically routes to the best available output:

Glasses with speakers → Plays on glasses
Glasses without speakers → Plays on phone
Phone Bluetooth → Respects phone audio routing

Users can connect glasses to their phone via Bluetooth like regular headphones to route all audio through glasses speakers.

Performance Tips

Pre-load Common Sounds

Cache frequently used audio:

// Store audio URLs as constants
const AUDIO_CACHE = {
  click: 'https://cdn.example.com/click.mp3',
  success: 'https://cdn.example.com/success.mp3',
  error: 'https://cdn.example.com/error.mp3'
};

// Reuse throughout your app
await session.audio.playAudio({ audioUrl: AUDIO_CACHE.click });

Optimize File Size

Use appropriate bitrate and format:

Sound effects: 128kbps MP3
Voice: 64-96kbps MP3
Music: 192-256kbps MP3

Use Short Durations

Keep audio brief for better UX:

// ✅ Good - 0.2-1 second for UI sounds
// ✅ Good - 2-5 seconds for notifications
// ❌ Avoid - 30+ seconds blocks interaction

Return Value

interface AudioPlayResult {
  success: boolean;
  error?: string;
  duration?: number;  // Audio duration in seconds
}

Example:

const result = await session.audio.playAudio({
  audioUrl: 'https://example.com/sound.mp3'
});

console.log('Success:', result.success);
console.log('Duration:', result.duration);

if (!result.success) {
  console.log('Error:', result.error);
}

Troubleshooting

Audio Not Playing

Check URL accessibility:

// Verify URL is publicly accessible
// Test in browser first

// Ensure HTTPS (not HTTP)
audioUrl: 'https://example.com/sound.mp3'  // ✅
audioUrl: 'http://example.com/sound.mp3'   // ❌ May fail

Playback Errors

Common issues:

CORS errors: Ensure CDN allows cross-origin requests
File format: Use MP3 for best compatibility
File size: Large files may timeout (60s limit)
URL access: Must be publicly accessible

Volume Too Low

Adjust volume:

await session.audio.playAudio({
  audioUrl: url,
  volume: 1.0  // Maximum volume
});

Example: Sound Library

class SoundLibrary {
  private baseUrl = 'https://cdn.example.com/sounds';

  private sounds = {
    click: 'click.mp3',
    success: 'success.mp3',
    error: 'error.mp3',
    notification: 'notification.mp3',
    confirm: 'confirm.mp3'
  };

  async play(session: AppSession, sound: keyof typeof this.sounds, volume = 0.8) {
    const url = `${this.baseUrl}/${this.sounds[sound]}`;

    try {
      const result = await session.audio.playAudio({
        audioUrl: url,
        volume
      });

      if (!result.success) {
        session.logger.warn(`Sound ${sound} failed:`, result.error);
      }

      return result.success;
    } catch (error) {
      session.logger.error(`Error playing ${sound}:`, error);
      return false;
    }
  }
}

// Usage
const sounds = new SoundLibrary();

session.events.onButtonPress(async (data) => {
  await sounds.play(session, 'click');
  // Handle button action
});

Next Steps

Text-to-Speech

Generate speech from text

Speech-to-Text

Capture user voice input

Audio Manager

Complete audio API reference

Device Control

All device control features

Getting Started

Core Concepts

SDK Reference

Playing Audio Files

Basic Usage

How It Works

Supported Formats

Audio Options

Volume Control

Stop Other Audio

Options Reference

Common Patterns

Sound Effects

Notification Sounds

Background Music

Error Handling

Sequential Audio

Conditional Audio

Stopping Audio

Best Practices

Audio Routing

Performance Tips

Return Value

Troubleshooting

Example: Sound Library

Next Steps

Text-to-Speech

Speech-to-Text

Audio Manager

Device Control

​Basic Usage

​How It Works

​Supported Formats

​Audio Options

​Volume Control

​Stop Other Audio

​Options Reference

​Common Patterns

​Sound Effects

​Notification Sounds

​Background Music

​Error Handling

​Sequential Audio

​Conditional Audio

​Stopping Audio

​Best Practices

​Audio Routing

​Performance Tips

​Return Value

​Troubleshooting

​Example: Sound Library

​Next Steps

Text-to-Speech

Speech-to-Text

Audio Manager

Device Control

Basic Usage

How It Works

Supported Formats

Audio Options

Volume Control

Stop Other Audio

Options Reference

Common Patterns

Sound Effects

Notification Sounds

Background Music

Error Handling

Sequential Audio

Conditional Audio

Stopping Audio

Best Practices

Audio Routing

Performance Tips

Return Value

Troubleshooting

Example: Sound Library

Next Steps