Skip to main content

​
What is Mira?

Mira is MentraOS’s AI assistant that lets users control your app using natural language. Instead of building UI buttons and menus, you expose functions that Mira automatically calls when users ask for something.
The magic: User says “remind me to buy milk” → Mira calls your add_todo function → Your app creates the todo → Mira confirms back to the user.
Your app doesn’t even need to be running! Mira can trigger your functions anytime and relay results back to users.

​
Quick Start

1

Define your tool in the Developer Console

Go to console.mentra.glass/apps and add your tool with an ID, description, and parameters.
2

Implement the onToolCall method

Override onToolCall in your AppServer to handle the tool logic.
3

Test with Mira

Talk naturally to Mira and watch your tool get called automatically!

​
Example Code

Important: What you return from onToolCall() is context for Mira’s AI, not what the user sees. Mira uses your response to formulate a natural language reply. If you want to control the display yourself, return GIVE_APP_CONTROL_OF_TOOL_RESPONSE instead. See “Understanding Tool Responses” below for details.
Here’s how to handle a Mira tool call in your app: 
import {AppServer, ToolCall} from "@mentra/sdk"

export class TodoAppServer extends AppServer {
  protected async onToolCall(toolCall: ToolCall): Promise<string | undefined> {
    // Check which tool Mira is calling
    if (toolCall.toolId === "add_todo") {
      // Get parameters Mira extracted from the user's message
      const todoItem = toolCall.toolParameters.todo_item as string;
      const dueDate = toolCall.toolParameters.due_date as string | undefined;

      // Do your app logic
      await this.todoService.addTodo(toolCall.userId, todoItem, dueDate);

      // Return a message for Mira to tell the user
      return `Added: "${todoItem}"${dueDate ? ` due ${dueDate}` : ''}`;
    }

    return undefined; // You don't handle this tool
  }
}
  1. Override the onToolCall method in your AppServer class
  2. Check toolCall.toolId to identify which function to run
  3. Extract parameters from toolCall.toolParameters
  4. Execute your app logic
  5. Return a message (see “Understanding Tool Responses” below)

​
Understanding Tool Responses

Important: What you return from onToolCall() determines how Mira responds to the user.

​
Default: Return Context for Mira

By default, your return string is context for Mira’s AI, not what the user sees directly: 
protected async onToolCall(toolCall: ToolCall): Promise<string | undefined> {
  if (toolCall.toolId === "add_todo") {
    const item = toolCall.toolParameters.todo_item as string;

    // Save the todo
    await db.todos.create({ userId: toolCall.userId, item });
    
// Return context for Mira (not displayed verbatim)
return `Todo item "${item}" was successfully saved to database with ID 123`;
  }
}
What happens:
  1. Your app returns context about what happened
  2. Mira’s AI uses this to formulate a natural response
  3. User hears something like: “Got it, I’ve added that to your list” or “Done, that’s been saved”
Why this is good:
  • ✅ Natural, conversational responses
  • ✅ Mira adapts the language to context
  • ✅ Feels like talking to an assistant

​
Manual Control: Display Your Own UI

If you want precise control over what displays, return GIVE_APP_CONTROL_OF_TOOL_RESPONSE: 
import { AppServer, ToolCall, GIVE_APP_CONTROL_OF_TOOL_RESPONSE } from '@mentra/sdk';

protected async onToolCall(toolCall: ToolCall): Promise<string | undefined> {
  if (toolCall.toolId === "show_shopping_list") {
    const session = this.getSessionByUserId(toolCall.userId);
    const items = await db.todos.getAll(toolCall.userId);

    // You control exactly what displays
    session.layouts.showReferenceCard(
      "Shopping List",
      items.map(item => `• ${item.name}`).join('\n')
    );

    // Tell Mira: "I've handled the display, don't say anything"
    return GIVE_APP_CONTROL_OF_TOOL_RESPONSE;
  }
}
What happens:
  1. Your app displays custom UI (layouts, dashboard, etc.)
  2. Mira stays silent - doesn’t add any response
  3. User sees exactly what you designed

​
Return undefined

If you don’t handle a tool, return undefined: 
protected async onToolCall(toolCall: ToolCall): Promise<string | undefined> {
  if (toolCall.toolId === "my_tool") {
    // Handle it
    return "Context for Mira";
  }

  // Don't handle this tool
  return undefined;
}

​
Quick Reference

Return ValueBehaviorUse When
String (e.g., "Task completed")Mira uses as context to formulate natural responseDefault - natural conversation
GIVE_APP_CONTROL_OF_TOOL_RESPONSEYou display UI, Mira stays silentNeed custom formatting or precise control
undefinedYou don’t handle this toolTool not recognized
Best practice: Let Mira handle responses (return strings) unless you need specific formatting. Users expect conversational AI behavior from Mira.

​
Examples: Context vs Control

// Simple confirmation
return "Alarm set for 7:00 AM";
// Mira: "Alright, I've set your alarm for 7 AM"

// Data retrieval
return "User has 3 pending tasks: Buy milk, Call mom, Send email";
// Mira: "You have 3 pending tasks: Buy milk, Call mom, and Send email"

// Status update
return "Weather is 72 degrees F and sunny in San Francisco";
// Mira: "It's 72 degrees and sunny in San Francisco"

​
Take Manual Control

// Custom formatted display
const session = this.getSessionByUserId(toolCall.userId);

session.layouts.showTextWall(
  "Today's Weather\n" +
  "72°F Sunny\nHumidity: 45%\nWind: 5mph"
);

return GIVE_APP_CONTROL_OF_TOOL_RESPONSE;

// Complex list that needs formatting
const tasks = await db.tasks.getAll(userId);

session.layouts.showReferenceCard(
  "Your Tasks",
  tasks.map((t, i) => `${i+1}. ${t.title} ${t.done ? '[DONE]' : '[TODO]'}`).join('\n')
);

return GIVE_APP_CONTROL_OF_TOOL_RESPONSE;

// Silent action (no response needed)
await db.settings.update(userId, { theme: 'dark' });

session.layouts.showTextWall("Theme updated");

return GIVE_APP_CONTROL_OF_TOOL_RESPONSE;

​
Developer Console Setup

Define Your Tools

Configure your Mira tools in the Developer Console:
  1. Select your app
  2. Navigate to “AI Tools” section
  3. Add tools with IDs, descriptions, and parameters
  4. Mira will automatically start recognizing them
The Developer Console UI is being updated. Screenshots may look different, but the core functionality remains the same.

​
How Mira Works Behind the Scenes

1

User speaks to Mira

Example: “Hey Mira, remind me to buy milk tomorrow”
2

Mira analyzes the request

Mira’s AI identifies your add_todo tool as the best match
3

Mira extracts parameters

  • todo_item: "buy milk"
  • due_date: "2025-11-04"
4

Mira calls your app

Sends a POST request to your /tool endpoint (SDK handles this automatically)
5

Your code executes

Your onToolCall method runs the business logic
6

You respond

Return a string like “Added: buy milk”
7

Mira responds to user

Mira formats your response into natural language and delivers it
Key insight: Tool calls work even when your app isn’t running. Mira can also chain multiple tools together to fulfill complex requests.

​
Tool Configuration

Tool Properties

  • id: Unique identifier (e.g., "add_todo")
  • description: What it does (helps Mira know when to call it)
  • activationPhrases: Optional trigger phrases
  • parameters: What data you need

Parameter Properties

  • type: "string", "number", or "boolean"
  • description: What this parameter is for
  • required: Is it mandatory?
  • enum: Limit to specific values (optional)
See ToolSchema for the complete type reference.

​
Next Steps

Dashboard Note Example

Complete working example with Mira Tool Calls

Tool Types Reference

Complete API documentation

Simple Storage

Persist data in your tools

Display

Learn about layouts and dashboard