npcpy Documentation

npcpy is a framework designed for research and development in multimodal AI and agentic systems. Key features include:

• Multimodal Support: Primitives for working with multimodal language models.
• Agentic AI: Tools to build multi-agent teams.
• Knowledge Graphs: Integration with knowledge graph structures.
• NPC Context-Agent-Tool Data Layer: A framework to simplify context engineering and ensure compliance through software-defined layers rather than relying solely on prompts.
• Flexible Provider Support: Works with local providers (ollama, llama.cpp, omlx, LM Studio) and cloud-based LLM providers.

The npcpy.gen module provides specialized helpers for generating various media types and handling generation-related responses. These modules include:

• image_gen: Tools for image generation tasks.
• audio_gen: Tools for audio generation tasks.
• video_gen: Tools for video generation tasks.
• response: Utilities for routing and managing generation responses.
• embeddings: Tools for working with embeddings related to generation workflows.

NPCArray is a vectorized abstraction for populations of models (LLMs, scikit-learn estimators, PyTorch modules, and NPC agents). It allows you to broadcast prompts, chain transformations, and reduce results using a unified NumPy-like API.

Key Concept: Lazy Evaluation Operations in NPCArray are lazy. Calling methods like .infer(), .map(), or .filter() returns a LazyResult which builds a computation graph without executing anything. The actual computation only occurs when you call .collect() (or its alias .compute()). You can inspect the planned execution using .explain() before running it.

The knowledge_graph_skill provides four distinct search methods for querying the SQLite-backed Knowledge Graph (KG). Choose the method based on your specific information retrieval needs:

1. Keyword Search: Fast substring matching over fact statements. Use kg_search_facts.
2. Embedding Search: Semantic cosine similarity using vector embeddings. Use kg_embedding_search when you need semantic similarity without graph structure.
3. Link Search: Graph traversal (BFS/DFS) starting from keyword-matched seeds. Use kg_link_search to explore connected neighborhoods/relationships.
4. Hybrid Search: Combines keyword, embedding, and link search, boosting results found by multiple methods. Use kg_hybrid_search when a query spans facts, concepts, and their relationships.

When a request specifies an NPC name, the server resolves it using the following precedence order:

1. Registered team NPCs: Checks all teams passed via the teams parameter for a matching name.
2. Globally registered NPCs: Checks the npcs dictionary passed at startup.
3. Database / file fallback: Loads the NPC from disk using the npc_source field ("global" checks ~/.npcsh/npc_team/, while "project" checks the request's currentPath).

Note: Programmatically registered NPCs always take precedence over file-based definitions.

The knowledge_store_skill allows agents to interact with .knowledge.yaml files located within project directories. Unlike a global database, KnowledgeStore is directory-scoped: each directory containing a .knowledge.yaml file maintains its own independent, append-only memory graph. The YAML files serve as the single source of truth.

Memory Statuses

Memories exist in several states that dictate how they are used:

• pending_approval: Raw extractions requiring human review.
• human-approved: Confirmed knowledge available for LLM context injection.
• human-rejected: Discarded knowledge; agents should respect these and avoid repeating them.
• human-edited: A corrected version that supersedes the initial_memory.

An NPC is an AI agent defined by a persona, a model, and optional tools. It wraps LLM calls with consistent behavior driven by a primary_directive. You can define NPCs programmatically using the NPC class or via .npc YAML files stored in an npc_team/ directory.

from npcpy.npc_compiler import NPC

agent = NPC(
    name='Analyst',
    primary_directive='You analyze data and provide insights.',
    model='llama3.2',
    provider='ollama',
    tools=[my_function],  # optional
)

NPCs are defined in YAML files with a .npc extension. These files specify the persona, model, and available tools for an individual agent. You can also inherit team-level jinxes by setting jinxes: ["*"].

name: data_analyst
primary_directive: >
  You are a meticulous data analyst who provides
  insights from structured and unstructured data.
model: llama3.2
provider: ollama
jinxes:
  - "*"          # inherit all team jinxes
tools:
  - statistical_analysis
  - data_visualization

Steps in a Jinx reference inputs and prior step outputs using the {{ variable_name }} syntax. When a step completes, its output is stored in the context under its name and becomes available to all subsequent steps. You can also interact with the context dictionary directly in Python steps.

steps:
  - name: "load_data"
    engine: "python"
    code: |
      import pandas as pd
      df = pd.read_csv('{{ file_path }}')
      context['row_count'] = len(df)
      output = f"Loaded {len(df)} rows"

  - name: "analyze"
    engine: "natural"
    code: |
      The dataset at {{ file_path }} has {{ row_count }} rows.
      Previous step said: {{ load_data }}
      Provide analysis and insights.