DOCSEARCHbyAlgolia
Submit Docs

sosumi.ai

repository·main·Indexed 19 days ago

https://github.com/nshipster/sosumi.ai

A service that converts JavaScript-rendered Apple Developer documentation, Human Interface Guidelines (HIG), and WWDC video transcripts into AI-readable Markdown. It provides an HTTP API, a CLI (@nshipster/sosumi), and an MCP server for AI agents to search and fetch Apple documentation and external Swift-DocC pages.

Tokens
10.9K
Snippets
47
Records
56
Agent score
66%

What's inside sosumi.ai

  2. Self-host Sosumi.ai

    main

    Sosumi is built with Hono, making it compatible with various runtimes (e.g., Cloudflare Workers).

    Prerequisites:

    • Node.js 20+
    • npm

    Quick Start Setup:

    1. Clone the repo: git clone https://github.com/nshipster/sosumi.ai.git && cd sosumi.ai
    2. Install dependencies: npm install
    3. Start development server: npm run dev

    Configuration for External Hosts: When self-hosting, you can control access to external Swift-DocC hosts using these environment variables:

    • EXTERNAL_DOC_HOST_ALLOWLIST: A newline-delimited list of permitted hosts.
    • EXTERNAL_DOC_HOST_BLOCKLIST: A newline-delimited list of denied hosts.
    git clone https://github.com/nshipster/sosumi.ai.git
cd sosumi.ai
npm install
npm run dev

  3. Fetch Apple documentation as Markdown via Sosumi

    main

    Sosumi allows you to fetch Apple documentation (API references, Human Interface Guidelines, WWDC transcripts, and external Swift-DocC) in Markdown format by replacing the developer.apple.com host with sosumi.ai in the URL.

    Core Workflow

    1. URL Transformation: If you have a developer.apple.com URL, replace the host with sosumi.ai and keep the same path.
    2. Search First: If the exact path is unknown, use the search functionality first to find the best match.
    3. Target Specificity: Prefer specific symbol pages over broad top-level pages for implementation questions.
    Original: https://developer.apple.com/documentation/swift/array
AI-readable: https://sosumi.ai/documentation/swift/array

  4. Use the Sosumi HTTP API to get AI-readable Markdown

    main

    Sosumi converts JavaScript-rendered Apple Developer documentation into Markdown by proxying requests. You can access any Apple Developer documentation, Human Interface Guidelines (HIG), or WWDC video transcripts by replacing developer.apple.com with sosumi.ai in the URL.

    For external Swift-DocC pages, use the /external/ prefix followed by the absolute HTTPS URL.

    Supported URL patterns:

    • API Reference & HIG: Replace https://developer.apple.com/ with https://sosumi.ai/.
    • WWDC Videos: Replace https://developer.apple.com/videos/play/ with https://sosumi.ai/videos/play/.
    • External Swift-DocC: Use https://sosumi.ai/external/<original_url>.
    // Original Apple Doc
https://developer.apple.com/documentation/swift/array

// Sosumi AI-readable Markdown
https://sosumi.ai/documentation/swift/array

// Original WWDC Video
https://developer.apple.com/videos/play/wwdc2021/10133/

// Sosumi AI-readable Video Transcript
https://sosumi.ai/videos/play/wwdc2021/10133

// External Swift-DocC
https://apple.github.io/swift-argument-parser/documentation/argumentparser

// Sosumi AI-readable External Doc
https://sosumi.ai/external/https://apple.github.io/swift-argument-parser/documentation/argumentparser

  5. Integrate Sosumi via MCP (Model Context Protocol)

    main

    Sosumi provides an MCP server that supports Streamable HTTP and Server-Sent Events (SSE) transport. If your client supports these, connect directly to https://sosumi.ai/mcp.

    For clients that require stdio transport, use mcp-remote to proxy the connection.

    Available MCP Tools:

    • searchAppleDocumentation(query: string): Searches Apple Developer documentation. Returns structured results (titles, URLs, descriptions, etc.).
    • fetchAppleDocumentation(path: string): Fetches Apple documentation or HIG by path (e.g., /documentation/swift). Returns Markdown.
    • fetchAppleVideoTranscript(path: string): Fetches WWDC video transcripts by path (e.g., /videos/play/wwdc2021/10133). Returns Markdown.
    • fetchExternalDocumentation(url: string): Fetches external Swift-DocC documentation by absolute HTTPS URL. Returns Markdown.
    {
  "mcpServers": {
    "sosumi": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://sosumi.ai/mcp"]
    }
  }
}

  6. Troubleshoot Sosumi errors

    main

    404 or sparse output

    • Cause: The path may be incorrect or too broad.
    • Solution: Run a search query first, then fetch the specific result path.

    External page cannot be fetched

    • Cause: The host may block access via robots.txt or X-Robots-Tag directives.
    • Solution: Try another canonical page URL for the same symbol.

  7. Use Sosumi MCP tools

    main

    When Sosumi is configured as an MCP server (at https://sosumi.ai/mcp), you can use the following tools:

    ToolParametersUse
    searchAppleDocumentationquery: stringSearch Apple documentation and return structured results
    fetchAppleDocumentationpath: stringFetch Apple docs or HIG content by path as Markdown
    fetchAppleVideoTranscriptpath: stringFetch Apple video transcript by /videos/play/... path
    fetchExternalDocumentationurl: stringFetch external Swift-DocC page by absolute HTTPS URL

  8. Configure External Documentation Access

    main

    The /external/* endpoint allows the service to fetch and convert DocC-based documentation from external hosts into Markdown.

    Security & Controls: Access to external hosts is governed by the following environment variables:

    • EXTERNAL_DOC_HOST_ALLOWLIST: A list of permitted hostnames.
    • EXTERNAL_DOC_HOST_BLOCKLIST: A list of forbidden hostnames.

    If an external host is not allowed, the service returns an ExternalAccessError (403/404).

  9. Sosumi URL patterns for different content types

    main

    Sosumi uses specific URL patterns to access different types of Apple-related content as Markdown:

    Apple API Reference

    Pattern: https://sosumi.ai/documentation/{framework}/{symbol}

    • https://sosumi.ai/documentation/swift/array
    • https://sosumi.ai/documentation/swiftui/view

    Human Interface Guidelines (HIG)

    Pattern: https://sosumi.ai/design/human-interface-guidelines/{topic}

    • https://sosumi.ai/design/human-interface-guidelines
    • https://sosumi.ai/design/human-interface-guidelines/foundations/color

    Apple Video Transcripts

    Pattern: https://sosumi.ai/videos/play/{collection}/{id}

    • https://sosumi.ai/videos/play/wwdc2021/10133
    • https://sosumi.ai/videos/play/meet-with-apple/208

    External Swift-DocC

    Pattern: https://sosumi.ai/external/{full-https-url}

    • https://sosumi.ai/external/https://apple.github.io/swift-argument-parser/documentation/argumentparser/
    • https://sosumi.ai/external/https://swiftpackageindex.com/pointfreeco/swift-composable-architecture/1.23.1/documentation/composablearchitecture

  10. Verify Sosumi CLI installation

    main

    If the Sosumi CLI is installed on your PATH, you can use the following commands for preflight checks:

    • sosumi --version: Confirms the CLI is installed.
    • sosumi --help: Shows top-level usage.
    • sosumi <command> --help: Shows the manual for a specific command.
    sosumi --version
sosumi --help
sosumi <command> --help

  11. Use the Sosumi CLI

    main

    The Sosumi CLI allows you to fetch documentation or search Apple docs directly from your terminal.

    Installation:

    npm i -g @nshipster/sosumi

    Common Commands:

    • sosumi fetch <path_or_url>: Fetches content as Markdown. Supports paths (e.g., /documentation/swift/array) or full URLs.
    • sosumi search <query>: Searches Apple documentation.
    • sosumi serve: Starts a local server.
    • sosumi serve --port <port>: Starts a local server on a specific port.

    Output Formats: By default, output is plain text/Markdown. Use the --json flag for machine-readable output in scripts.

    # Fetch via npx
npx @nshipster/sosumi fetch https://developer.apple.com/documentation/swift/array

# Fetch via installed CLI
sosumi fetch /documentation/swift/array

# Search with JSON output
sosumi search "SwiftData" --json

  12. Fetch raw video page HTML with fetchVideoTranscriptHtml()

    main

    Use fetchVideoTranscriptHtml to retrieve the raw HTML content of an Apple Developer video page. This function handles setting appropriate headers (including a random User-Agent) and checking response status.

    Parameters:

    • sourceUrl: The URL of the video page to fetch.

    Returns: A Promise<string> containing the raw HTML text.

    Errors:

    • Throws TranscriptNotFoundError if the response status is 404.
    • Throws a generic Error for other non-OK HTTP status codes.
    import { fetchVideoTranscriptHtml, TranscriptNotFoundError } from './src/lib/video/index';

try {
  const html = await fetchVideoTranscriptHtml('https://developer.apple.com/videos/play/wwdc24/sessions/12345/');
  console.log(html);
} catch (error) {
  if (error instanceof TranscriptNotFoundError) {
    // Handle 404
  }
}