Render visual components, attach files, and compose screens with the @agentapps/sdk.

UI interfaces

Agent Apps can produce visual output alongside plain text. Instead of returning a string and hoping the agent formats it nicely, your app can return structured data that the host renders natively — cards, tables, charts, forms, images, and more.

This page covers the three mechanisms for visual output:

UI resources — declarative components (tree, detail, form) the host renders on demand
Renderers — transform tool results into rendered output automatically when a bound tool returns
Attachments — return images, files, and videos in tool results

How visual output works

Your app  ──tool result──►  Host  ──renders──►  User sees a card / table / chart / image
              │
              ├── result (text or JSON — always sent to the model)
              ├── attachments (images, files — shown in the UI)
              └── rendered (renderer output — embedded in chat context)

The host always receives the raw result (for the LLM) and optionally attachments and rendered output (for the UI). This separation means the model gets the data it needs while the user sees a rich visual representation.

UI resources

UI resources are declarative components your app registers at build time. The host discovers them via list_ui_resources and fetches renderable data via get_ui_resource. Three resource types are supported:

Type	Purpose	Example
`tree`	Hierarchical node tree	File explorer, task list, category tree
`detail`	Key/value field panel	Node detail, settings summary, record view
`form`	Interactive form with typed fields	Create/edit dialog, configuration form

Register a UI resource

import { createApp } from '@agentapps/sdk';

const app = createApp({
  id: 'app.example.tasks',
  version: '1.0.0',
  name: 'Tasks',
  description: 'A task manager with a tree view',
  author: { name: 'Example', email: 'dev@example.com' },
  category: 'productivity',
  runtime: { transport: 'stdio', command: 'node', args: ['./dist/index.js'] },
});

// Register a tree resource
app.ui('tree', 'task-tree', {
  description: 'Task hierarchy',
  boundTools: ['create-task', 'complete-task'],
  handler: async (params, ctx) => ({
    type: 'tree' as const,
    title: 'Tasks',
    nodes: [
      {
        id: 'inbox',
        label: 'Inbox',
        icon: '📥',
        badge: 5,
        children: [
          { id: 't1', label: 'Fix login bug' },
          { id: 't2', label: 'Update docs' },
        ],
      },
      {
        id: 'done',
        label: 'Done',
        icon: '✅',
        badge: 12,
      },
    ],
  }),
});

Tree resource

A tree resource renders as a collapsible node hierarchy. Each TreeNode has:

id — unique identifier within the tree
label — display text
icon — optional emoji or icon name
badge — optional count or status indicator
children — nested nodes (recursive)
hasChildren — set to true for lazy-loaded nodes without inline children

app.ui('tree', 'file-tree', {
  description: 'Project file structure',
  handler: async () => ({
    type: 'tree' as const,
    title: 'Files',
    multiSelect: false,
    nodes: [
      {
        id: 'src',
        label: 'src/',
        icon: '📁',
        children: [
          { id: 'index', label: 'index.ts' },
          { id: 'utils', label: 'utils.ts' },
        ],
      },
      {
        id: 'readme',
        label: 'README.md',
        badge: '3 KB',
      },
    ],
  }),
});

Detail resource

A detail resource renders as a panel of sections containing key/value fields. Each DetailField has a label, value, and optional type hint that tells the host how to render the value:

Field type	Rendering
`text`	Plain text (default)
`code`	Monospace code block
`date`	Formatted date
`url`	Clickable link
`markdown`	Rendered markdown
`image`	Inline image

app.ui('detail', 'task-detail', {
  description: 'Detailed view of a task',
  parameters: {
    type: 'object',
    properties: { taskId: { type: 'string' } },
    required: ['taskId'],
  },
  handler: async (params) => {
    const task = await getTask(params.taskId);
    return {
      type: 'detail' as const,
      title: task.title,
      sections: [
        {
          title: 'Status',
          fields: [
            { label: 'State', value: task.status },
            { label: 'Priority', value: task.priority },
            { label: 'Created', value: task.createdAt, type: 'date' },
          ],
        },
        {
          title: 'Links',
          fields: [
            { label: 'URL', value: task.url, type: 'url' },
            { label: 'Preview', value: task.screenshotUrl, type: 'image' },
          ],
        },
      ],
    };
  },
});

Form resource

A form resource renders as an interactive form. The host collects user input and submits it back to your app via submit_ui_resource. Each FormField supports these input types:

text, textarea, number, select, checkbox, radio, date, time, datetime

app.ui('form', 'create-task', {
  description: 'Create a new task',
  handler: async () => ({
    type: 'form' as const,
    submitLabel: 'Create task',
    submitTool: 'create-task',
    fields: [
      {
        name: 'title',
        label: 'Title',
        type: 'text' as const,
        required: true,
        placeholder: 'Enter task title',
      },
      {
        name: 'priority',
        label: 'Priority',
        type: 'select' as const,
        options: [
          { label: 'Low', value: 'low' },
          { label: 'Medium', value: 'medium' },
          { label: 'High', value: 'high' },
        ],
        defaultValue: 'medium',
        helpText: 'How urgent is this task?',
      },
      {
        name: 'description',
        label: 'Description',
        type: 'textarea' as const,
        placeholder: 'Add details...',
      },
    ],
  }),
});

Binding resources to tools

Use boundTools to link a UI resource to specific tools. When those tools are called, the host automatically refreshes the resource:

app.ui('tree', 'task-tree', {
  description: 'Task hierarchy',
  boundTools: ['create-task', 'complete-task', 'delete-task'],
  refreshOnEvents: ['task.created', 'task.completed'],
  handler: async () => ({ /* ... */ }),
});

You can also use refreshOnEvents to refresh the resource when specific events are emitted.

Renderers

Renderers are bound to tools and automatically transform tool results into rendered output. When a bound tool returns, the SDK calls the renderer's render handler with the tool result and context, then attaches the rendered output to the tool result response.

Register a renderer

import { createApp, renderToOutput } from '@agentapps/sdk';
import type { UiResourceData } from '@agentapps/protocol';

const app = createApp({ /* ... */ });

app.renderer('task-card', {
  description: 'Render task results as a detail card',
  boundTools: ['get-task'],
  formats: ['markdown', 'text', 'html'],
  render: async (result, ctx) => {
    const task = result as { id: string; title: string; status: string };

    const data: UiResourceData = {
      type: 'detail',
      title: task.title,
      sections: [
        {
          title: 'Info',
          fields: [
            { label: 'ID', value: task.id, type: 'code' },
            { label: 'Status', value: task.status },
          ],
        },
      ],
    };

    return renderToOutput(data, 'task-card', 'markdown');
  },
});

renderToOutput()

renderToOutput(data, resourceName, format?) converts a UiResourceData object into a RenderedOutput — the full object with format, content, resource name, and resource type:

import { renderToOutput } from '@agentapps/sdk';

const output = renderToOutput(data, 'task-card', 'markdown');
// {
//   format: 'markdown',
//   content: '### Fix login bug\n\n**Info**\n- **ID:** `t1`\n- **Status:** open',
//   resource: 'task-card',
//   resourceType: 'detail',
// }

renderUiResourceData()

renderUiResourceData(data, format?) returns just the rendered string — useful when you only need the content without the wrapper object:

import { renderUiResourceData } from '@agentapps/sdk';

const markdown = renderUiResourceData(data, 'markdown');
const html = renderUiResourceData(data, 'html');
const text = renderUiResourceData(data, 'text');

Output formats

Three formats are supported. The host selects based on its presentation mode:

Format	Description	Use case
`markdown`	Markdown with headers, lists, links	Default — works in any chat that renders markdown
`text`	Plain text, no markup	Hosts without markdown support (CLI, SMS)
`html`	Self-contained HTML with inline styles	Hosts that render inline HTML

The renderer is framework-agnostic — it produces strings, not React components or DOM nodes. It does not depend on React Native, DOM APIs, or any host-specific rendering engine.

How renderers connect to tools

The dispatch logic in the SDK automatically finds renderers bound to a tool when it returns:

Tool handler executes and returns a result
SDK iterates registered renderers — first matching renderer wins
Renderer's render handler is called with the tool result
The RenderedOutput is attached to the tool result response
If the render handler throws, the tool result is still returned (best-effort)

This means you don't call renderers manually — just register them, and the SDK handles the rest.

Declarative UI components

Renderers can also produce declarative components — typed UI elements the host renders using its own theme. These are part of the RendererOutput schema and give you more control than the text/markdown/html fallback.

Supported component types:

Component	Description
`text`	Text block with optional style (`body`, `heading`, `subheading`, `caption`, `code`)
`table`	Tabular data with typed columns and rows
`card`	Card with optional title and key/value fields
`chart`	Bar, line, or pie chart
`form`	Interactive form (host collects input, returns to app)
`divider`	Visual separator

Example: return a card with data

app.renderer('summary-card', {
  description: 'Render a summary as a card',
  boundTools: ['get-summary'],
  render: async (result) => {
    const summary = result as { total: number; active: number; pending: number };

    return {
      components: [
        {
          type: 'card',
          title: 'Summary',
          fields: [
            { label: 'Total', value: summary.total },
            { label: 'Active', value: summary.active },
            { label: 'Pending', value: summary.pending },
          ],
        },
      ],
      variants: {
        markdown: `### Summary\n- **Total:** ${summary.total}\n- **Active:** ${summary.active}\n- **Pending:** ${summary.pending}`,
        text: `Summary\n  Total: ${summary.total}\n  Active: ${summary.active}\n  Pending: ${summary.pending}`,
      },
    };
  },
});

Example: return a table

return {
  components: [
    {
      type: 'table',
      columns: [
        { key: 'name', label: 'Name' },
        { key: 'status', label: 'Status' },
        { key: 'priority', label: 'Priority' },
      ],
      rows: [
        { name: 'Fix login', status: 'open', priority: 'high' },
        { name: 'Update docs', status: 'done', priority: 'low' },
      ],
    },
  ],
};

Example: return a chart

return {
  components: [
    {
      type: 'chart',
      chartType: 'bar',
      title: 'Tasks by status',
      data: [
        { label: 'Open', value: 5 },
        { label: 'In Progress', value: 3 },
        { label: 'Done', value: 12 },
      ],
    },
  ],
};

When declarative components are present, the host renders them natively. The variants object provides fallback text/markdown/html for hosts that can't render the declarative schema.

Attachments

Attachments let your tool return images, files, and videos alongside the text result. They appear in the chat UI as downloadable or viewable items.

Attachment shape

Each attachment has three fields:

Field	Required	Description
`url`	Yes	Public URL to the file
`mime`	Yes	MIME type (e.g. `image/png`, `application/pdf`)
`filename`	No	Display name for the file

Return an image from a tool

app.tool('take-screenshot', {
  description: 'Take a screenshot of a URL',
  parameters: {
    type: 'object',
    properties: { url: { type: 'string' } },
    required: ['url'],
  },
  handler: async (args, ctx) => {
    const screenshot = await captureScreenshot(args.url);

    // Write the file to the volume
    await ctx.volume.write(
      `screenshots/${screenshot.id}.png`,
      screenshot.bytes,
    );

    // Create a public URL for it
    const shared = await ctx.share.share(`screenshots/${screenshot.id}.png`, {
      expiresIn: 3600,
    });

    return {
      output: `Screenshot taken of ${args.url}`,
      attachments: [
        {
          url: shared.url,
          mime: 'image/png',
          filename: 'screenshot.png',
        },
      ],
    };
  },
});

Return a file

app.tool('export-report', {
  description: 'Export data as a PDF report',
  parameters: {
    type: 'object',
    properties: { format: { type: 'string', enum: ['pdf', 'csv'] } },
  },
  handler: async (args, ctx) => {
    const pdf = await generateReport(args.format);

    await ctx.volume.write(`reports/report.${args.format}`, pdf.bytes);
    const shared = await ctx.share.share(`reports/report.${args.format}`);

    return {
      output: `Report exported as ${args.format.toUpperCase()}`,
      attachments: [
        {
          url: shared.url,
          mime: args.format === 'pdf' ? 'application/pdf' : 'text/csv',
          filename: `report.${args.format}`,
        },
      ],
    };
  },
});

Multiple attachments

Return multiple files at once — useful for galleries or batch exports:

return {
  output: 'Generated 3 charts',
  attachments: [
    { url: chart1Url, mime: 'image/png', filename: 'revenue.png' },
    { url: chart2Url, mime: 'image/png', filename: 'users.png' },
    { url: chart3Url, mime: 'image/png', filename: 'engagement.png' },
  ],
};

Attachments + renderers

Attachments and renderers are independent — you can use both in the same tool result. The host shows the rendered visual output in the chat and displays attachments below or alongside it:

app.renderer('screenshot-view', {
  boundTools: ['take-screenshot'],
  render: async (result) => {
    const data = result as { url: string };
    return {
      components: [
        {
          type: 'card',
          title: 'Screenshot',
          fields: [{ label: 'Image', value: data.url }],
        },
      ],
    };
  },
});

Interfaces and screens

For full-page UIs beyond chat-embedded components, the SDK provides interfaces and screens:

Interface (app.uiInterface()) — a composable UI piece with a bundle URL (e.g. a React component bundle)
Screen (app.screen()) — a navigable window composed from one or more interfaces

Register an interface

app.uiInterface('task-board', {
  description: 'Kanban board component',
  bundleUrl: 'https://cdn.example.com/bundles/task-board.js',
  propsSchema: {
    type: 'object',
    properties: {
      projectId: { type: 'string' },
    },
  },
});

Register a screen

app.screen('tasks', {
  title: 'Tasks',
  description: 'Full task management view',
  icon: '📋',
  interfaces: ['task-board', 'task-detail'],
  launcher: true, // appears in the host's launcher / new-tab picker
});

Screens marked with launcher: true appear in the host's launcher so users can open them directly. Screens without launcher are only reachable via in-app navigation.

When to use visual output vs plain text

Situation	Use
Simple status or confirmation	Plain text result
Structured data (rows, fields)	UI resource (`detail`) or renderer with `card` / `table`
Hierarchical data	UI resource (`tree`)
User input needed	UI resource (`form`)
Tool produces an image or file	Attachments
Tool produces data that benefits from visual context	Renderer with declarative components
Full-page app experience	Interfaces + screens

Guidelines:

Always return a text result — the model needs it for reasoning. Visual output supplements text, it doesn't replace it.
Use renderers for automatic formatting — bind them to tools and let the SDK handle dispatch.
Use UI resources for on-demand views — the host fetches them when the user navigates to a panel.
Use attachments for binary content — images, PDFs, videos that can't be represented as text.
Keep renderers deterministic — the same tool result should always produce the same visual output. Neither the LLM nor the app should make runtime rendering decisions.

Next steps

Implement tools — register tool handlers that produce results
Events — emit events to refresh UI resources
Storage — persist data for your UI resources

UI interfaces

On this page