repository·main·Indexed 27 days ago
https://github.com/microsoft/call-center-aiAn AI-powered call center solution built with Azure and OpenAI GPT. It enables developers to automate inbound and outbound phone calls, extract structured data (claims) from conversations, and manage customer interactions using LLMs like GPT-4. The system includes features for call recording, custom training data via Azure AI Search, and detailed conversation reporting.
The project uses the OpenAI SDK directly instead of a standard LLM framework. This was chosen because, at the time of development, existing frameworks could not handle the specific requirements of this project, including:
To develop and test the solution locally while using Azure resources for telephony, follow these steps:
Rust and uv, then run make install to set up the Python environment.make name=my-rg-name sync-local-config.config.yaml from config-local-example.yaml.make deploy-bicep deploy-post name=my-rg-name to deploy Azure resources (excluding the API server).devtunnel login and then make tunnel to connect to Azure Dev tunnels.make dev. The API server will be available at http://localhost:8080 and supports automatic reloading on file changes.Testing without Phone Calls:
You can test the application logic without Communication Services by running the local.py script:
python3 -m tests.local
Call recording is disabled by default. To enable it:
recordings in your Azure Storage account (this is automatically done if you used the standard Azure deployment).recording_enabled feature flag in Azure App Configuration to true.After a call is completed, you can access a detailed user report containing the conversation history, extracted claim data, and any generated reminders. The report is available at a specific URL pattern based on the phone number used during the call.
URL Pattern:
https://[your_domain]/report/[phone_number]
Note: The phone number in the URL must be URL-encoded (e.g., +133658471534 becomes %2B133658471534).
To move from the current implementation to a production-ready state, the following areas must be addressed:
You can instruct the AI agent to initiate a phone call to a specific number by making a POST request to the /call endpoint. The request body must define the bot's identity, the task to perform, the agent's phone number, and a structured claim schema to capture specific information during the conversation.
bot_company: The name of the company the bot represents.bot_name: The name of the AI assistant.phone_number: The destination phone number to call (E.164 format).task: A detailed description of the bot's objective and context.agent_phone_number: The phone number the bot will call from.claim: An array of objects defining the data points to be extracted. Each object requires a name and a type (e.g., text, datetime).# Ask the bot to call a phone number
data='{
"bot_company": "Contoso",
"bot_name": "Amélie",
"phone_number": "+11234567890",
"task": "Help the customer with their digital workplace. Assistant is working for the IT support department. The objective is to help the customer with their issue and gather information in the claim.",
"agent_phone_number": "+33612345678",
"claim": [
{
"name": "hardware_info",
"type": "text"
},
{
"name": "first_seen",
"type": "datetime"
},
{
"name": "building_location",
"type": "text"
}
]
}'
curl \
--header 'Content-Type: application/json' \
--request POST \
--url https://xxx/call \
--data $dataTo deploy the solution directly to Azure using pre-built container images, follow these steps:
config.yaml file at the project root by filling out the template from config-remote-example.yaml.az login.make. You can specify a specific image version using the image_version parameter to ensure stability (e.g., image_version=0.1.0).Prerequisites:
yqmakeContainer Images:
ghcr.io/clemlesne/call-center-ai:mainghcr.io/clemlesne/call-center-ai:0.1.0The bot supports multiple languages based on Azure Text-to-Speech. You can configure the default language and available language options (including pronunciations and specific voices) in config.yaml under conversation.initiate.lang.
If using an Azure Speech Custom Neural Voice (CNV), include the custom_voice_endpoint_id field.
# config.yaml
conversation:
initiate:
lang:
default_short_code: fr-FR
availables:
- pronunciations_en: ["French", "FR", "France"]
short_code: fr-FR
voice: fr-FR-DeniseNeural
- pronunciations_en: ["Chinese", "ZH", "China"]
short_code: zh-CN
voice: zh-CN-XiaoqiuNeuralTo enable SMS capabilities via Twilio, add your credentials to config.yaml under the sms section.
# config.yaml
sms:
mode: twilio
twilio:
account_sid: xxx
auth_token: xxx
phone_number: "+33612345678"Prompts can be customized in config.yaml.
prompts.tts: Templates for Text-to-Speech. These use {xxx} placeholders (e.g., {bot_name}, {bot_company}) which are replaced at runtime. Templates are provided as lists to allow for varied user experiences.prompts.llm: System prompts for the LLM (e.g., default_system_tpl, chat_system_tpl).Note: All TTS prompts should be written in English to serve as a pivot language for conversation translation.
The task provides context to the LLM about what the bot should accomplish. You can set a default task in config.yaml or provide a custom task string in the POST /call API request.
# config.yaml
conversation:
initiate:
task: |
Help the customer with their insurance claim. Assistant requires data from the customer to fill the claim. The latest claim data will be given. Assistant role is not over until all the relevant data is gathered.When running the application, you can use a Service Principal for Azure authentication by adding the following credentials to a .env file:
AZURE_CLIENT_ID=xxx
AZURE_CLIENT_SECRET=xxx
AZURE_TENANT_ID=xxx