Prompt API Documentation

A system prompt provides context for the entire session. It must be the very first message in the sequence. You can provide it in three ways:

1. Via the initialPrompts option in LanguageModel.create().
2. By calling session.append() with a system role message immediately after creation.
3. By including the system role as the first object in the array passed to session.prompt().

Note: Placing a { role: "system" } prompt anywhere other than the 0th position in the first message sequence will result in a TypeError.

// Option 1: Create a new session with a system prompt as the first message.
const session1 = await LanguageModel.create({
  initialPrompts: [{ role: "system", content: "Pretend to be an eloquent hamster." }]
});
console.log(await session1.prompt("What is your favorite food?"));

// Option 2: Create a new session and append a system prompt as the first message.
const session2 = await LanguageModel.create();
await session2.append([{ role: "system", content: "Pretend to be an eloquent hamster." }]);
console.log(await session2.prompt("What is your favorite food?"));

// Option 3: Create a new session and prompt with a system prompt as the first message.
const session3 = await LanguageModel.create();
console.log(await session3.prompt([
  { role: "system", content: "Pretend to be an eloquent hamster." },
  { role: "user", content: "What is your favorite food?" }
]));

Language model sessions have a maximum token capacity (the contextWindow).

Monitoring usage:

• Use session.contextUsage to see current tokens used.
• Use session.contextWindow to see total available tokens.
• Use session.measureContextUsage(input) to estimate token consumption of a potential prompt without sending it.

Handling overflow: If a prompt exceeds the remaining capacity, the session automatically removes the oldest conversation history (excluding initialPrompts) to make room. You can listen for the "contextoverflow" event to detect this.

If the prompt is so large that even after removing history it cannot fit, the call will fail with a QuotaExceededError.

// Check current usage
console.log(`${session.contextUsage} tokens used, out of ${session.contextWindow} tokens available.`);

// Estimate usage for a string
const stringUsage = await session.measureContextUsage(promptString);

// Estimate usage for multimodal input
const audioUsage = await session.measureContextUsage([{
  role: "user",
  content: [
    { type: "text", value: "My response:" },
    { type: "audio", value: audioBlob }
  ]
}]);

// Listen for overflow events
session.addEventListener("contextoverflow", () => {
  console.log("Context window exceeded; old messages dropped.");
});

To ensure programmatic processing of model responses, you can use the responseConstraint option in session.prompt(). This allows you to force the model to adhere to a specific structure or pattern.

JSON Schema

Pass a valid JSON schema object. The returned string can be parsed with JSON.parse(). If the model cannot produce a compliant response, a SyntaxError is thrown.

Regular Expressions

Pass a RegExp object. The returned string will match the pattern. If the model cannot produce a matching response, a SyntaxError is thrown.

Optimizing Context Usage

By default, the constraint is included in the prompt, consuming tokens. To avoid this, use omitResponseConstraintInput: true and include the instructions manually in your prompt string.

Note: If omitResponseConstraintInput is true but responseConstraint is not set, a TypeError occurs.

// Using JSON Schema
const schema = {
  type: "object",
  required: ["rating"],
  additionalProperties: false,
  properties: {
    rating: { type: "number", minimum: 0, maximum: 5 },
  },
};

const result = await session.prompt("Summarize this feedback into a rating between 0-5: The food was delicious.", {
  responseConstraint: schema
});
const { rating } = JSON.parse(result);

// Using RegExp
const emailRegExp = /^[a-zA-Z0-9.!#$%&'*+\/:=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9]{0,61}[a-zA-Z0-9])?$/;
const emailAddress = await session.prompt(
  `Create a fictional email address for ${characterName}.`,
  { responseConstraint: emailRegExp }
);

// Using omitResponseConstraintInput to save tokens
const result = await session.prompt(
  `Summarize this feedback into a rating between 0-5, only outputting a JSON object { rating }, with a single property whose value is a number: The food was delicious.`,
  { responseConstraint: schema, omitResponseConstraintInput: true }
);

The Prompt API supports tool use via the tools option in LanguageModel.create(). This allows the model to invoke external JavaScript functions.

Each tool object requires:

• name: A unique identifier.
• description: A description of what the tool does.
• inputSchema: A JSON schema defining the arguments.
• execute: An async function that performs the task and returns a result (e.g., a JSON string).

Concurrency Note: The model may call tools multiple times concurrently (e.g., if a user asks about multiple locations). The user agent handles these calls using internal Promise.all() logic before the model generates its final response.

const session = await LanguageModel.create({
  initialPrompts: [
    { role: "system", content: "You are a helpful assistant. You can use tools to help the user." },
  ],
  expectedInputs: [{ type: "text", languages: ["en"] }, { type: "tool-response" }],
  expectedOutputs: [{ type: "text", languages: ["en"] }, { type: "tool-call" }],
  tools: [
    {
      name: "getWeather",
      description: "Get the weather in a location.",
      inputSchema: {
        type: "object",
        properties: {
          location: {
            type: "string",
            description: "The city to check for the weather condition.",
          },
        },
        required: ["location"],
      },
      async execute({ location }) {
        const res = await fetch("https://weatherapi.example/?location=" + location);
        return JSON.stringify(await res.json());
      },
    },
  ],
});

const result = await session.prompt("What is the weather in Seattle?");

To use multimodal inputs, you must configure the session using the expectedInputs option during LanguageModel.create(). This ensures necessary downloads occur and validates model capability.

Format Changes for Multimodal Prompts: When using multimodal content, you cannot use the string shorthand. You must use the explicit array-of-objects format:

1. prompt() must receive an array of messages.
2. Each message must have a role property.
3. The content property must be an array of content objects (e.g., { type: 'text', value: '...' }, { type: 'image', value: ... }).

Supported Types:

• Images: Blob, ImageData, ImageBitmap, VideoFrame, OffscreenCanvas, HTMLImageElement, SVGImageElement, HTMLCanvasElement, HTMLVideoElement, or BufferSource.
• Audio: Blob, AudioBuffer, or BufferSource.

Security Note: Cross-origin data (like images from another domain) that lacks Access-Control-Allow-Origin headers will cause a SecurityError DOMException.

const session = await LanguageModel.create({
  expectedInputs: [
    { type: "audio" },
    { type: "image" }
  ]
});

const referenceImage = await (await fetch("/reference-image.jpeg")).blob();
const userDrawnImage = document.querySelector("canvas");

const response1 = await session.prompt([{
  role: "user",
  content: [
    { type: "text", value: "Give a helpful artistic critique of how well the second image matches the first:" },
    { type: "image", value: referenceImage },
    { type: "image", value: userDrawnImage }
  ]
}]);