JARVIS AI Assistant

To run JARVIS on macOS, follow these steps to set up the backend, frontend, and environment:

1. Clone the repository:

   git clone https://github.com/yourusername/jarvis.git
   cd jarvis

2. Configure environment variables: Copy the example env file and populate it with your API keys:

   cp .env.example .env

   See the Configuration section for required keys.

3. Install Python dependencies:

   pip install -r requirements.txt

4. Install frontend dependencies:

   cd frontend && npm install && cd ..

5. Generate SSL certificates (Required for secure WebSockets):

   openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes -subj '/CN=localhost'

6. Run the application:

   • Terminal 1 (Backend):

     python server.py

   • Terminal 2 (Frontend):

     cd frontend && npm run dev

7. Access the UI: Open http://localhost:5173 in Google Chrome. Click the page once to enable audio before speaking.

# 1. Clone the repo
git clone https://github.com/yourusername/jarvis.git
cd jarvis

# 2. Set up environment
cp .env.example .env

# 3. Install Python dependencies
pip install -r requirements.txt

# 4. Install frontend dependencies
cd frontend && npm install && cd ..

# 5. Generate SSL certificates
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes -subj '/CN=localhost'

# 6. Start the backend
python server.py

# 7. Start the frontend
cd frontend && npm run dev

# 8. Open Chrome
open http://localhost:5173

The JARVIS frontend is a state-driven application that orchestrates several core modules to provide a voice-interactive experience:

• Orb Visualization: Uses createOrb to render a visual representation on a canvas. Its visual state is synchronized with the application's lifecycle via orb.setState(state).
• Voice Input: Managed by createVoiceInput. It captures speech and provides callbacks for successful transcriptions and errors. It can be start()ed, pause()ed, or resume()ed.
• Audio Playback: Managed by createAudioPlayer. It handles queuing audio data and provides an onFinished callback to signal when speech has ended. It also provides an Analyser used to drive the Orb's animations.
• WebSocket Communication: Managed by createSocket. It facilitates real-time bidirectional communication with the backend, handling message types like transcript, audio, status, and task_spawned.
• State Machine: The application transitions between four primary states: idle, listening, thinking, and speaking. Transitions trigger corresponding UI updates, orb animations, and voice input controls (e.g., pausing voice input while the system is thinking or speaking).

// Conceptual flow of the state machine and component interaction
transition("listening"); // Resumes voice input
transition("thinking");  // Pauses voice input
transition("speaking"); // Pauses voice input and plays audio
transition("idle");     // Resumes voice input

JARVIS features a specialized "Work Mode" for software development tasks.

Modes of Operation:

• Chat Mode: Standard conversational interaction using fast keyword detection and Haiku-based responses.
• Work Mode: Activated via specific commands or when a work_session is started. It uses claude -p (Claude Code) for high-power coding tasks.
• Planning Mode: A state where JARVIS asks clarifying questions to build a project plan. Once confirmed, it generates a CLAUDE.md instruction file and begins building.

Key Features in Work Mode:

• Stall Detection: If Claude Code asks too many clarifying questions instead of coding, JARVIS automatically pushes it to "start building now".
• Auto-open Localhost: If a response mentions a localhost URL, JARVIS automatically opens it in the browser.
• Project Scaffolding: When building, JARVIS creates a directory on the Desktop, writes a detailed CLAUDE.md with specific instructions (e.g., using React + Vite + Tailwind), and expects the final output to include a RUNNING_AT=http://localhost:PORT line.

The ClaudeTaskManager class manages background claude -p subprocesses. It handles task spawning, status tracking, and provides real-time updates to connected WebSocket clients.

Key features:

• Concurrency Control: Limits the number of simultaneous tasks (default is 3).
• Automatic Project Creation: If no working directory is provided, it generates a kebab-case folder name on the Desktop.
• Visual Execution: Uses AppleScript to open a new Terminal window and run the command visibly.
• Auto-QA: Automatically runs a QA verification process upon task completion.
• WebSocket Notifications: Pushes task_spawned, task_complete, and qa_result events to clients.

manager = ClaudeTaskManager(max_concurrent=3)
# Spawns a task and returns a unique task_id
task_id = await manager.spawn("Build a React todo app", working_dir="./my-project")

# Retrieve task details
task = await manager.get_status(task_id)
print(f"Status: {task.status}")

# List all tasks
all_tasks = await manager.list_tasks()