Shadow Background Coding Agent

The Shadow Code Indexing System converts a repository into a searchable semantic format for agents through a five-step pipeline:

1. Graph Construction: Builds a semantic graph where GraphNode objects represent entities (files, classes, functions, documentation) and GraphEdge objects define relationships (e.g., CONTAINS, CALLS, DOCS_FOR).
2. Chunking: Breaks large code blocks into smaller, semantically coherent chunks to fit within language model context windows.
3. Embedding: Converts code chunks into high-dimensional vectors using embedding models (e.g., Jina or local transformers).
4. Storage: Uploads vectors and metadata (file paths, node IDs) to a vector database, specifically Pinecone.
5. Retrieval: Enables natural language queries by embedding the query and performing similarity searches against the Pinecone index.

This process allows an agent to find code based on semantic meaning rather than simple keyword matching.

The search and replace tool allows you to propose a modification to an existing file by replacing a specific old_string with a new_string.

Constraints and Best Practices

• Single Instance Only: The tool replaces exactly ONE occurrence per call. To modify multiple instances of the same string, you must make separate tool calls for each instance.
• Uniqueness via Context: To ensure the correct instance is targeted, the old_string must be unique. You must include significant context to avoid accidental matches elsewhere in the file:
  • Include at least 3-5 lines of context BEFORE the target change point.
  • Include at least 3-5 lines of context AFTER the target change point.
  • Match all whitespace, indentation, and surrounding code exactly as it appears in the source file.
• Verification Workflow:
  1. Identify if multiple instances of the target string exist.
  2. Gather unique context for each instance.
  3. Plan and execute separate tool calls for each unique instance.

Shadow can be deployed to AWS using Amazon EKS with Kata QEMU containers for hardware-level VM isolation. The deployment process involves two main stages: setting up the remote infrastructure (EKS cluster) and then deploying the Shadow application itself.

Prerequisites

Before running deployment scripts, ensure you have configured your AWS SSO profile:

aws configure sso --profile=YOUR_PROFILE_ID

Deployment Steps

1. Deploy the EKS Cluster with Kata Containers (Takes 25-35 minutes): Run the infrastructure deployment script to set up the EKS cluster, Kata QEMU runtime, and persistent storage.

   ./scripts/deploy-remote-infrastructure.sh

2. Deploy the Shadow Application: Once the infrastructure is ready, deploy the complete platform.

   ./scripts/deploy-full-infrastructure.sh

# Configure AWS SSO
aws configure sso --profile=ID

# Deploy infrastructure (25-35 minutes)
./scripts/deploy-remote-infrastructure.sh

# Deploy complete platform
./scripts/deploy-full-infrastructure.sh

The grep_search tool is designed for fast, exact regex searches using the ripgrep engine. It is preferred over semantic search when you know the exact symbol, function name, or text pattern you are looking for within specific directories or file types.

Key Constraints and Behaviors

• Match Type: Performs only exact regex searches. Do NOT attempt fuzzy or semantic matches.
• Output Limit: Results are capped at 50 matches to prevent overwhelming the output.
• Filtering: Use include or exclude patterns to filter the search scope by file type or specific paths.
• Input Requirement: You must return only a valid regex pattern string.

Regex Escaping Requirements

When providing a search string, you must escape special regex characters using a backslash (\). The characters that require escaping include: (, ), [, ], {, }, +, *, ?, ^, $, |, ., and \.

| Literal               | Regex Pattern            |
|-----------------------|--------------------------|
| function(             | function\\(              |
| value[index]          | value\\[index\\]         |
| file.txt               | file\\.txt                |
| user\|admin            | user\\\|admin             |
| path\\to\\file         | path\\\\to\\\\file         |
| hello world           | hello world              |
| foo\\(bar\\)          | foo\\\\(bar\\\\)          |