Note Companion Documentation

repository·master·Indexed 21 days ago

https://github.com/nexus-jpf/note-companion

An AI-powered Obsidian plugin for organizing, transcribing, and chatting with notes. Supports cloud-based workflows via Note Companion Cloud or self-hosted setups using provider keys (OpenAI, Claude, Gemini, Groq) and local LLMs via Ollama. Includes guides for installing the desktop plugin, deploying the web and mobile packages, and implementing atomic database updates to fix race conditions in file processing.

Tokens
96.6K
Snippets
236
Records
417
Agent score
74%

What's inside Note Companion

  1. What is AI Elements?

    master
    AI Elements is a component library and custom registry built on top of shadcn/ui. It provides prebuilt, composable, and accessible (ARIA/keyboard navigation) components specifically designed for AI interfaces. It is built to work with the Vercel AI SDK (v5/v6), leveraging its latest streaming improvements and unified message formats.
  2. Integrate with Obsidian themes using CSS variables

    master

    To support Obsidian's dark mode and user-defined themes, do not use hardcoded colors. Instead, use Obsidian's CSS variables within Tailwind classes using the [--variable-name] syntax.

    Commonly used variables:

    • --text-normal, --text-muted, --text-accent, --text-error
    • --background-primary, --background-secondary
    • --interactive-accent, --interactive-hover
    • --background-modifier-border

    Example Mapping:

    • Instead of bg-white, use bg-[--background-primary]
    • Instead of text-slate-600, use text-[--text-muted]
    • Instead of border-gray-200, use border-[--background-modifier-border]
    // Correct way to apply theme-aware colors
    <div className={tw("bg-[--background-primary] text-[--text-normal] border-[--background-modifier-border]")}>
      Themed Content
    </div>
  3. How Note Companion organizes files using the PARA method

    master

    Note Companion implements the PARA method to structure your digital knowledge into four distinct categories:

    • Projects: Active projects and immediate tasks.
    • Areas: Ongoing responsibilities and long-term interests.
    • Resources: Reference materials and topics of interest.
    • Archives: Completed or inactive items.

    This methodology is used by the AI-powered organization engine to automatically classify and categorize files based on their content and context.

  4. How Note Companion tools are executed

    master

    Note Companion uses a server-defined, client-executed pattern to perform actions within your Obsidian vault. This allows the AI to control local files securely without the server having direct access to your filesystem.

    1. Server Definition: The tool schema is defined on the server (packages/web/app/api/(newai)/chat/tools.ts).
    2. AI Decision: The AI model determines which tool to call based on your request.
    3. Client Execution: The Obsidian plugin (packages/plugin/views/assistant/ai-chat/) intercepts the tool call and executes the corresponding logic using the Obsidian API.
    4. Result Loop: The execution results are streamed back to the AI to complete the conversation loop.
  5. How Note Companion processes files

    master

    Note Companion automates file organization by monitoring a specific inbox folder. When you move files into the inbox, the system uses AI to perform several automated actions.

    The Workflow:

    1. Ingestion: Move any file or folder from your vault into the NoteCompanion/Inbox directory.
    2. Processing: Wait for the AI to process the item (text files are near-instant; audio and images take longer).
    3. Automated Actions (if enabled in plugin options):
      • Rename: The document title is updated based on content.
      • Tagging: Tags are added based on connections to existing tagged files.
      • Organization: The file is moved to the most appropriate folder. If no suitable folder is identified, it is moved to NoteCompanion/Processed.

    Special File Handling:

    • Audio files: A text transcription is appended to the processed document.
    • Image files: Annotations are added to the processed document.
    • Tagging: Every processed file is automatically tagged with #nc-processed to track its status.
  6. Implement Machine Learning Preference Learning

    master

    The PreferenceLearner service allows the system to learn user preferences (classification, folder, tags, or templates) based on file patterns. When a user confirms or rejects a suggestion, the learner updates its confidence. If confidence exceeds a threshold (e.g., 70-80%), the system can auto-apply the preference in future processing steps.

    interface UserPreference {
      pattern: string; // File pattern (e.g., "*.pdf", "meeting-*")
      classification?: string; // Preferred classification
      folder?: string; // Preferred destination
      tags?: string[]; // Preferred tags
      template?: string; // Preferred template
      confidence: number; // Learning confidence (0-100)
      appliedCount: number; // Times user confirmed this preference
      rejectedCount: number; // Times user rejected this preference
    }
    
    class PreferenceLearner {
      private preferences: Map<string, UserPreference> = new Map();
    
      learnFromAction(
        file: TFile,
        action: {
          type: 'folder' | 'tag' | 'classification' | 'template';
          value: any;
        }
      ) {
        const pattern = this.detectPattern(file);
        const key = `${pattern}_${action.type}`;
    
        const existing = this.preferences.get(key);
        if (existing && existing.value === action.value) {
          // User confirmed this preference
          existing.appliedCount++;
          existing.confidence = Math.min(100, existing.confidence + 5);
        } else {
          // New preference
          this.preferences.set(key, {
            pattern,
            [action.type]: action.value,
            confidence: 50,
            appliedCount: 1,
            rejectedCount: 0,
          });
        }
      }
    
      getRecommendation(
        file: TFile,
        type: 'folder' | 'tag' | 'classification' | 'template'
      ) {
        const pattern = this.detectPattern(file);
        const key = `${pattern}_${type}`;
        const pref = this.preferences.get(key);
    
        if (pref && pref.confidence > 70) {
          return {
            value: pref[type],
            confidence: pref.confidence,
            reason: `Based on ${pref.appliedCount} previous files matching "${pattern}"`,
          };
        }
    
        return null;
      }
    
      private detectPattern(file: TFile): string {
        const parts = file.basename.split(/[-_]/);
        if (parts[0].length > 3) {
          return `${parts[0]}-*.${file.extension}`;
        }
        return `*.${file.extension}`;
      }
    }
  7. Compare Obsidian vault strategies: Comprehensive Capture vs. Curated Insight

    master

    When organizing an Obsidian vault, you can choose between two primary mental models depending on whether your goal is exhaustive documentation or actionable insight.

    Approach 1: The Comprehensive Capture Vault

    Goal: To cast a wide net and ensure no information is lost.

    • Structure: Uses large inbox/ or dump/ folders for rapid capture of quotes, highlights, and thoughts.
    • Workflow: Minimal metadata at capture; processing is sporadic or non-existent.
    • Best for: Research-heavy projects, creative brainstorming, and archival/compliance needs.
    • Risk: Search overload and shallow connections due to unprocessed volume.

    Approach 2: The Curated Insight Vault

    Goal: To prioritize clarity, connection, and easy retrieval.

    • Structure: Purposeful folders like projects/, literature/, and reference/.
    • Workflow: Brief capture into an inbox/, followed by regular (e.g., weekly) processing sessions to summarize, distill, and tag notes before moving them to topic-specific folders.
    • Best for: Focused writing, professional/student workflows, and maintaining a manageable knowledge base.
    • Risk: High maintenance/discipline required; risk of over-pruning useful raw data.
  8. Locate file backups

    master

    Note Companion performs destructive formatting and automated changes (renaming, moving, tagging). To protect your data, a backup system is automatically triggered.

    Backups are stored in the _NoteCompanion/Backups/ folder. When a file is formatted, a link to its original version is appended to the newly formatted file. Note that these backups are currently managed internally and are not directly accessible via a dedicated 'Undo' button in the main UI.

  9. Handle Tool Invocations in the UI

    master

    To provide visual feedback when the AI calls a tool, implement a custom tool handler. This component should intercept tool-call message parts and render specific UI based on the toolName.

    Pattern:

    1. Iterate through message.parts looking for type === 'tool-call'.
    2. Use a switch statement on toolInvocation.toolName.
    3. For each tool, render a specialized handler that takes the args and state (e.g., 'call' or 'result').
    4. When the tool execution completes, call addToolResult (provided by useChat) to feed the result back into the conversation stream.
    export const CustomToolHandler: React.FC<CustomToolHandlerProps> = ({ toolInvocation, addToolResult }) => {
      const { toolName, toolCallId, args, state } = toolInvocation;
    
      switch (toolName) {
        case "searchNotes":
          return (
            <SearchHandler
              args={args}
              state={state}
              onComplete={(result) =>
                addToolResult({ toolCallId, result: JSON.stringify(result) })
              }
            />
          );
        default:
          return <div>Tool: {toolName}</div>;
      }
    };
  10. Understand the Note Companion processing pipeline

    master

    Note Companion automates document transcription and organization through a multi-step pipeline triggered when files are dropped into an inbox folder. The system follows three main phases:

    1. Detection & Queuing: An event handler detects file creation/renaming, waits for a 1-second delay, and enqueues the file. The system manages concurrency by limiting media files to 2 concurrent processes and regular files to 5.
    2. Processing Steps: The pipeline executes a sequence of up to 15 steps, including content extraction (PDF text, Image OCR, Audio transcription, or YouTube transcripts), classification, folder recommendation, name recommendation, and content formatting.
    3. Record Management: Every step is tracked via a RecordManager, which logs the status, errors, and metadata for each file.

    Files that fail processing are moved to an Error or Bypass folder, while successful files are moved to their destination folder.

    graph TB
        A[User drops file in Inbox] --> B[Event Handler detects creation]
        B --> C[Inbox Queue Manager]
        C --> D{File Type?}
        D -->|Media| E[Media Queue max 2 concurrent]
        D -->|Regular| F[Regular Queue max 5 concurrent]
        E --> G[Processing Pipeline]
        F --> G
        G --> H[Record Manager tracks steps]
        H --> I{Success?}
        I -->|Yes| J[Move to destination folder]
        I -->|No| K[Move to Error/Bypass folder]
        J --> L[Update sidebar UI]
        K --> L
  11. Optimize tw() function usage for performance

    master

    The tw() function (using twMerge) is fast, but you should avoid creating new tw() calls inside loops to prevent unnecessary overhead during component renders.

    Pattern to avoid: Creating a new tw() call for every item in a list.

    Recommended pattern: Extract common classes into a constant outside the loop, then combine them inside the loop.

    ```tsx
    // DO: Extract common classes
    const baseClass = tw("p-2");
    const activeClass = tw(baseClass, "bg-[--interactive-accent]");
    
    {items.map(item => (
      <div className={item.active ? activeClass : baseClass}>
        {item.name}
      </div>
    ))}
    ```埋