Arkitect SDK Documentation

The AI Video/Audio to Document Assistant (media2doc) is a Web-based tool powered by Large Language Models (LLMs). It allows users to convert videos and audio files into various document styles (such as Xiaohongshu posts, WeChat Official Account articles, knowledge notes, mind maps, or content summaries) with a single click.

Key features include:

• Privacy: No login or registration required; task records are stored locally.
• Frontend Processing: Uses ffmpeg wasm technology, eliminating the need for local ffmpeg installation.
• AI Interaction: Supports secondary AI Q&A based on the video content.
• Customization: Supports custom prompts via the frontend and one-click subtitle export.
• Security: Supports setting an access password on the backend to restrict frontend usage.

Mobile Use is an AI Agent solution that automates mobile tasks using natural language instructions. It leverages Volcano Engine Cloud Phone for a secure, isolated execution environment and the Doubao Visual Large Model for vision-based reasoning and interaction.

Key features include:

• AI-Powered Automation: Uses visual models to understand and interact with mobile apps.
• Cloud Phone Integration: Executes tasks in a stable, low-latency cloud environment.
• MCP Protocol Support: Implements the Model Context Protocol via a Go-based MCP server.
• Real-time Streaming: Uses SSE (Server-Sent Events) for real-time feedback and monitoring.
• Web Interface: A Next.js frontend for task management and visualization.

The Computer Use Agent (CUA) allows users to execute tasks on a remote computer via simple natural language instructions (e.g., video editing, PPT creation, social media maintenance). It operates by interacting directly with Graphical User Interfaces (GUI) without requiring specific application APIs.

CUA follows a three-step cognitive loop:

1. Perception: Captures computer screen images to contextualize the digital environment.
2. Reasoning: Uses Chain-of-Thought (CoT) reasoning to evaluate observations and track progress by analyzing current and past screenshots.
3. Action: Executes tasks using virtual mouse and keyboard inputs (typing, clicking, scrolling).

AI App Lab is a toolkit provided by Volcengine Ark designed to help developers bridge the gap between large language models (LLMs) and real-world, scenario-based applications. It addresses technical challenges such as model API orchestration, plugin coordination, multi-modal fusion (image, text, voice), and tool calling.

The project consists of two primary components:

1. Arkitect: A high-code Python SDK for professional developers to build complex AI applications using a structured set of tools and workflows.
2. Demohouse: A collection of open-source prototype applications across various vertical domains (e.g., education, mobile automation, deep reasoning) that developers can "one-click copy" and customize with their own business logic.

The Real-time Conversational AI solution integrates LLMs, ASR, and TTS to enable low-latency, natural human-like voice interactions.

Key Capabilities:

• Intelligent Interruption: Supports full-duplex communication and audio-frame level Voice Activity Detection (VAD) for natural interruptions.
• On-device Noise Reduction: Uses RTC SDK to reduce background noise and music interference, improving interruption accuracy.
• Ultra-low Latency: A full-link streaming architecture (RTC+ASR+LLM+TTS) reduces total latency to under 1 second.
• Weak Network Resilience: Optimized via intelligent access and RTC cloud coordination to ensure reliable transmission in complex network environments.
• Multi-platform Support: Compatible with iOS, Android, Windows, Linux, macOS, Web, Flutter, Unity, Electron, and WeChat Mini Programs.

The veFaaS MCP Server is a Model Context Protocol (MCP) server designed to manage Volc Engine Function as a Service (veFaaS) functions. It provides an interface for AI agents and MCP clients to perform lifecycle management tasks for serverless functions.

Key Capabilities:

• Function Creation: Create new veFaaS functions with automatically generated random names.
• Code Updates: Update existing function code using several methods:
  • Base64-encoded zip files
  • TOS (Tencent Object Storage) objects
  • Container images
• Deployment: Release functions for production use.
• Monitoring: Check the existence and current status of functions.
• Utility: Create base64-encoded zip files directly from Python code.

go-stock is an open-source AI-powered stock analysis software. It allows users to subscribe to stock information and provides AI-driven analysis capabilities.

Note: Currently, it only supports Windows.