BambooAI Documentation

repository·main·Indexed 21 days ago

https://github.com/pgalko/bambooai

An open-source library for natural language-based data analysis using Large Language Models (LLMs). BambooAI enables users to interact with pandas DataFrames through conversation, automatically generating and executing Python code for analysis and visualization. It features a multi-agent system configurable via LLM_CONFIG.json, support for multiple LLM providers (OpenAI, Anthropic, Gemini, Ollama, VLLM, MiniMax), auxiliary dataset integration, and episodic memory via Pinecone or Qdrant vector databases.

Tokens
6.7K
Snippets
17
Records
31
Agent score
74%

What's inside BambooAI

  1. How BambooAI processes queries

    main

    BambooAI follows a six-step lifecycle to transform natural language into data insights:

    1. Initiation: Starts with a user prompt and maintains a conversation loop.
    2. Task Routing: Uses an LLM to classify questions and route them to either a text response handler or a code generation handler.
    3. User Feedback: The model can pause to ask for clarification if instructions are vague or if it encounters ambiguity during the process.
    4. Dynamic Prompt Build: Evaluates data needs, formulates an analysis plan, performs semantic searches for similar questions, and generates code using the selected LLM.
    5. Debugging and Execution: Executes the generated code and uses LLM-based self-healing to correct errors and retry until successful.
    6. Results and Knowledge Base: Ranks answers for quality, stores high-quality solutions in a vector database, and presents results or visualizations.
  2. Enable Vector DB for long-term memory (Episodic Memory)

    main

    BambooAI supports integrating with Pinecone or Qdrant to store and retrieve successful analyses. When a user ranks a solution highly (>6), the intent, plan, code, and metadata are vectorized and stored. For future similar tasks, the system queries the vector index (using a similarity threshold of 0.8) to guide agents using previous successful solutions.

    To enable this, set vector_db=True in the BambooAI constructor and configure the appropriate environment variables.

    from bambooai import BambooAI
    import pandas as pd
    
    # Initialize with vector database enabled
    bamboo = BambooAI(
        df=your_dataframe,
        vector_db=True
    )
  3. Quick Start with BambooAI

    main

    To get started with a basic data analysis session, follow these steps:

    1. Install the package: pip install bambooai
    2. Configure environment variables: Copy the example environment file and update it with your credentials.
      cp .env.example .env
    3. Configure agents and models: Copy the sample configuration file to define your preferred LLM combinations.
      cp LLM_CONFIG_sample.json LLM_CONFIG.json
    4. Run a session: Use the BambooAI class to initialize an instance with your dataframe and start a conversation.
    import pandas as pd
    from bambooai import BambooAI
    
    import plotly.io as pio
    pio.renderers.default = 'jupyterlab'
    
    df = pd.read_csv('titanic.csv')
    bamboo = BambooAI(df=df, planning=True, vector_db=False, search_tool=True)
    bamboo.pd_agent_converse()
  4. Install and run the BambooAI Web Application via pip

    main

    To run the Web UI using the pip package:

    1. Install the package:
      pip install bambooai
    2. Download the web_app folder from the repository.
    3. Configure the environment by copying .env.example to .env in the web_app directory.
    4. Configure LLM agents in web_app/LLM_CONFIG.json by copying LLM_CONFIG_sample.json and editing the agent_configs array.
    5. Run the app:
      cd <path_to_web_app>
      python app.py

    The interface is available at http://localhost:5000.

  5. Install BambooAI

    main

    You can install BambooAI using pip or by cloning the repository and installing the requirements manually.

    Using pip:

    pip install bambooai

    Using the repository source:

    git clone https://github.com/pgalko/BambooAI.git
    pip install -r requirements.txt
    pip install bambooai
  6. Configure Vector DB environment variables

    main

    Depending on your chosen provider, set the following in your .env file:

    For Pinecone:

    VECTOR_DB_TYPE=pinecone
    PINECONE_API_KEY=<YOUR API KEY HERE>
    PINECONE_CLOUD=aws
    PINECONE_REGION=us-east-1

    For Qdrant:

    VECTOR_DB_TYPE=qdrant
    QDRANT_URL=http://localhost:6333  # For local Qdrant
    QDRANT_API_KEY=<YOUR API KEY HERE>  # Optional for local, required for cloud
  7. Configure agents and models via LLM_CONFIG.json

    main

    BambooAI uses a multi-agent system where specialized agents handle different parts of the analysis. You can configure each agent's model, provider, and parameters by editing the LLM_CONFIG.json file located in your working directory.

    Important Requirements:

    1. LLM_CONFIG.json must be in the BambooAI working directory.
    2. All API keys for the models specified in agent_configs must be present in a .env file in the same working directory.
    3. If you assign a model to an agent in agent_configs, that model must also be defined in the model_properties section of the JSON.

    Agent Roles:

    • Expert Selector: Determines the best expert type for the query.
    • Analyst Selector: Selects the specific analysis approach.
    • Theorist: Provides theoretical background and methodology.
    • Dataframe Inspector: Analyzes data structure (requires ontology file).
    • Planner: Creates step-by-step analysis plans.
    • Code Generator: Writes Python code for analysis.
    • Error Corrector: Debugs and fixes code issues.
    • Reviewer: Evaluates solution quality.
    • Solution Summarizer: Creates concise result summaries.
    • Google Search Executor: Executes search queries.
    • Google Search Summarizer: Synthesizes search results.
  8. Configure LLM agents in LLM_CONFIG.json

    main

    The Web Application requires an LLM_CONFIG.json file to define which models and providers each agent uses. If this is missing or incomplete, execution will fail.

    Example structure for the agent_configs array:

    {
       "agent_configs": [
          {
             "agent": "Code Generator",
             "details": {
                "model": "your-preferred-model",
                "provider": "provider-name",
                "max_tokens": 4000,
                "temperature": 0
             }
          }
       ]
    }
  9. Integrate with SweatStack data

    main

    BambooAI supports loading longitudinal sports and health data from SweatStack.

    1. Authorize: Navigate to /sweatstack/authorize to begin the OAuth flow.
    2. Get Users: GET /sweatstack/get_users returns a list of accessible athletes/users.
    3. Load Data: POST /sweatstack/load_data fetches and combines data for selected users.
      • Request Body:
        • sports: List of sports to include.
        • metrics: List of metrics to include.
        • users: List of user IDs.
        • start_date / end_date: ISO format strings (YYYY-MM-DD).
    4. Remove Data: POST /sweatstack/remove_data clears the primary dataset and resets the BambooAI instance while preserving auxiliary datasets.
  10. Use BambooAI in Interactive or Single Query Mode

    main

    You can use BambooAI in two primary ways within a Jupyter Notebook or CLI:

    1. Interactive Mode: Uses pd_agent_converse() to start a conversational session.
    2. Single Query Mode: Passes a specific string to pd_agent_converse() to execute a one-off task.
    import pandas as pd
    from bambooai import BambooAI
    
    # Setup
    df = pd.read_csv('training_activity_data.csv')
    bamboo = BambooAI(df=df, search_tool=True, planning=True)
    
    # Option 1: Interactive Mode
    bamboo.pd_agent_converse()
    
    # Option 2: Single Query Mode
    bamboo.pd_agent_converse("Calculate 30, 50, 75 and 90 percentiles of the heart rate column")
  11. Configure agents with alternative providers (Ollama, VLLM, MiniMax)

    main

    You can use local or alternative LLM providers by specifying the appropriate provider in the agent_configs section of LLM_CONFIG.json.

    Ollama Example:

    {
      "agent": "Planner",
      "details": {
        "model": "llama3:70b",
        "provider": "ollama",
        "max_tokens": 2000,
        "temperature": 0
      }
    }

    VLLM Example:

    {
      "agent": "Code Generator",
      "details": {
        "model": "/path/to/model/DeepSeek-R1-Distill-14B",
        "provider": "vllm",
        "max_tokens": 2000,
        "temperature": 0
      }
    }

    MiniMax Example:

    {
      "agent": "Code Generator",
      "details": {
        "model": "MiniMax-M3",
        "provider": "minimax",
        "max_tokens": 8000,
        "temperature": 0.1
      }
    }
  12. Use the BambooAI class for data analysis

    main

    The BambooAI class is the primary entry point for natural language-based data analysis. It allows you to interact with a pandas DataFrame using a conversation loop.

    Key Parameters:

    • df: The pandas DataFrame you wish to analyze.
    • planning: (bool) Enables an optional planning agent for complex tasks.
    • vector_db: (bool) Enables/disables the use of a vector database for episodic memory.
    • search_tool: (bool) Enables/disables integration with internet searches and external APIs.

    Core Method:

    • pd_agent_converse(): Starts the interactive conversation loop where the agent processes natural language queries, generates Python code, and executes it to provide insights or visualizations.
    from bambooai import BambooAI
    import pandas as pd
    
    df = pd.read_csv('data.csv')
    bamboo = BambooAI(df=df, planning=True, vector_db=False, search_tool=True)
    bamboo.pd_agent_converse()