Local-Code-Interpreter

The response parser uses a strategy pattern via the ChoiceHandler class to process LLM response chunks. When a chunk is received, ChoiceHandler iterates through a predefined list of strategies. Each strategy implements a support() method to check if it can handle the current delta and an execute() method to perform the state updates.

Supported Strategies:

1. RoleChoiceStrategy: Updates the assistant's role name if delta.role is present.
2. ContentChoiceStrategy: Adds text content to the bot_backend and updates history if delta.content is present.
3. NameFunctionCallChoiceStrategy: Identifies the function name from tool_calls. It validates if the function exists in the jupyter_kernel.available_functions or additional_tools. If the function is missing, it appends an error to the history and sets whether_exit to True.
4. ArgumentsFunctionCallChoiceStrategy: Extracts function arguments. It includes special handling for:
   • python: Handles 'hallucinatory' calls where raw code is provided instead of JSON.
   • execute_code: Parses JSON arguments to extract the code string.
5. FinishReasonChoiceStrategy: Handles the end of a turn. If the finish_reason is tool_calls, it triggers the execution of the requested function (either via the Jupyter kernel or additional tools) and updates the history with the results.

The BotBackend class manages the lifecycle of an AI code interpreter session, including the conversation history, Jupyter kernel state, and tool availability. It inherits from GPTResponseLog to track real-time generation states.

Key capabilities:

• Conversation Management: Methods like add_text_message, add_file_message, and add_gpt_response_content_message update the internal conversation list used for chat completions.
• File Handling: add_file_message copies uploaded files into a unique Jupyter working directory and notifies the model via a system message. revoke_file allows removing the last uploaded file and its associated messages.
• Kernel Control: send_interrupt_signal interrupts the running Jupyter kernel, and restart resets the conversation, clears the working directory, and restarts the kernel.

backend = BotBackend()
# Add user input
backend.add_text_message("Calculate the square root of 144")
# Add a file to the session
backend.add_file_message("path/to/data.csv", "Initial bot message")
# Reset the session
backend.restart()

Configuration is handled via a config.json file located in the src directory. You should base your configuration on the examples provided in the config_example directory.

Key Configuration Rules

1. Model Selection (model_name)

The program requires models with function calling capabilities (version 0613 or newer).

Supported Models:

• gpt-3.5-turbo-0613 (and 16K version)
• gpt-3.5-turbo-1106
• gpt-3.5-turbo-0125
• gpt-4-0613 (and 32K version)
• gpt-4-1106-preview
• gpt-4-0125-preview
• gpt-4-turbo
• gpt-4o
• gpt-4o-2024-05-13
• gpt-4o-2024-08-06
• gpt-4o-mini
• gpt-4o-mini-2024-07-18

Note on Vision: gpt-4-vision-preview does not support function calling directly. To use vision, set the model to GPT-4V and set available to true (see Vision Model Settings).

Azure OpenAI Users:

• Set model_name to your deployment name.
• Ensure the deployment is a 0613 or newer version.
• Set API_VERSION to 2024-03-01-preview.

2. Vision Model Settings

To enable vision input using a non-end-to-end approach:

• Set model_name to GPT-4V.
• Set available to true.
• Setting available to false disables vision-related system prompts and reduces API costs.

3. Model Context Window

The model_context_window field is used to slice conversations when they exceed capacity.

• For Azure OpenAI, manually map deployment names to their integer context window:

  "<YOUR-DEPLOYMENT-NAME>": <context_window (integer)>

4. API Key Handling

You can either:

1. Provide the key directly in config.json under API_KEY.
2. Leave API_KEY as an empty string ("API_KEY": "") and set the OPENAI_API_KEY environment variable.
   • Windows: set OPENAI_API_KEY=<YOUR-API-KEY>
   • Linux: export OPENAI_API_KEY=<YOUR-API-KEY>

The Web UI is built using Gradio and relies on a state_dict to manage the BotBackend instance. Key lifecycle operations include:

• Initialization: The initialization(state_dict) function sets up the cache directory and instantiates the BotBackend if it doesn't exist. It also clears the OPENAI_API_KEY from the environment to ensure backend control.
• Restarting the UI: restart_ui(history) clears the chat history and resets all UI components (textboxes, buttons) to their default states.
• Restarting the Backend: restart_bot_backend(state_dict) triggers a full restart of the underlying BotBackend logic.
• Stopping Generation: stop_generating(state_dict) interrupts a running code execution or updates the stop state to halt GPT response generation.

# Conceptual usage of state management
state = {"bot_backend": None}
initialization(state)
# ... interaction ...
restart_ui(history)
restart_bot_backend(state)