Botium Core
repository·master·Indexed 42 days ago
https://github.com/codeforequity-at/botium-coreAn open-source testing framework for conversational AI, designed as the 'Selenium for Chatbots.' Botium Core version 1.15.16 provides a standardized automation layer to test chatbot interactions across various platforms using connectors, a fluent API via BotDriver, and custom asserter hooks for conversation lifecycle validation.
What's inside Botium
- Botium is an open-source testing framework designed specifically for conversational AI. It is conceptualized as the 'Selenium for Chatbots,' providing a standardized way to automate and test chatbot interactions across various platforms and technologies.
How to use Botium Connectors
masterBotium Core provides the underlying testing engine, but to interact with specific chatbot technologies (frameworks, APIs, SDKs, or cloud services), you must use Connectors.
Connectors act as the bridge between Botium and your chatbot. Each connector typically includes sample configurations and scripts to facilitate setup. You can find various connectors hosted on GitHub.
Use the integrated Inbound Endpoint
masterBotium Core can launch an inbound endpoint using an integrated HTTP server. This allows you to send input to a bot and wait for both synchronous and asynchronous responses.
To use this method:
- Start your chatbot endpoint (e.g.,
node jokebot.js). - Run the Botium script using a configuration file that defines the inbound endpoint (e.g.,
BOTIUM_CONFIG=botium-inbound.json node botiumFluent.js).
# 1. Start the chatbot node jokebot.js # 2. Start the Botium script with the inbound config BOTIUM_CONFIG=botium-inbound.json node botiumFluent.js- Start your chatbot endpoint (e.g.,
Run the custom connector sample
masterTo execute the custom connector sample project, install the necessary dependencies using
npm installand then run the test suite usingnpm test.$ npm install $ npm testUse the Redis Inbound Endpoint
masterYou can use a Redis-based inbound endpoint where the Botium CLI launches an inbound proxy and Botium Core connects to a shared Redis topic. This requires a running Redis server.
To use this method:
- Start a local Redis server.
- Launch the inbound proxy using the Botium CLI:
botium-cli inbound-proxy. - Start your chatbot and connect it to the Botium CLI inbound endpoint (e.g.,
node jokebot.js). - Run the Botium script using a configuration file that targets the Redis endpoint (e.g.,
BOTIUM_CONFIG=botium-redis.json node botiumFluent.js).
# 1. Start Redis server # 2. Start the inbound proxy via Botium CLI botium-cli inbound-proxy # 3. Start the chatbot node jokebot.js # 4. Start the Botium script with the Redis config BOTIUM_CONFIG=botium-redis.json node botiumFluent.jsTrigger asserters in Botium scripts
masterTo execute an asserter during a test, use its defined
reffollowed by arguments separated by the pipe|character. You can trigger asserters at three specific lifecycle points using these keywords:#begin: Before the conversation starts.#bot: After a bot response (on step).#end: After the conversation is finished.
Syntax:
[KEYWORD] [REF] [ARG1] | [ARG2] | ...#begin DUMMY arg1 | arg2 | arg3 #me hi #bot Hello! DUMMY argA | argB #end DUMMY argX | argYUse the Botium Box Inbound Endpoint
masterBotium Box provides an integrated inbound endpoint secured with an API key.
To use this method:
- Launch your chatbot (e.g.,
node jokebot.js). - In Botium Box, register the endpoint by importing the
_botium-box.json_file. - Update the
callbackUriwithin that file to include a valid API Key from Botium Box.
# 1. Launch the chatbot node jokebot.js # 2. Import _botium-box.json_ into Botium Box and update callbackUri with your API Key- Launch your chatbot (e.g.,
Run the Botium Polling HTTP/JSON example
masterThis sample demonstrates a Botium connector that uses polling to interact with an HTTP/JSON chatbot endpoint. The included demo chatbot (
jokebot.js) responds with random jokes. To run the full demonstration, you must first start the chatbot endpoint and then execute the Botium script.# 1. Start the chatbot endpoint node jokebot.js # 2. Start the Botium script to send input and poll for responses node botiumFluent.jsConfigure Botium via environment variables
masterBotium allows configuration through environment variables following specific naming conventions:
- Capabilities: Any environment variable starting with
BOTIUM_(e.g.,BOTIUM_PROJECTNAME) is treated as a capability. - Sources: If a
BOTIUM_variable matches a known source key (e.g.,BOTIUM_LOCALPATH), it is applied to thesourcesobject. - Environment Variables: Variables starting with
BOTIUM_ENV_are mapped to the driver'senvsobject. For example,BOTIUM_ENV_DATABASE_URL=localhostresults inenvs.DATABASE_URL = 'localhost'.
Note: If
BOTIUM_CONFIGis set, the driver will attempt to load configuration from that specific JSON file and its associated.local.jsonor environment-specific variants.- Capabilities: Any environment variable starting with
Use ConvoStep parameters via logic hooks
masterYou can pass parameters to a specific conversation step using a special logic hook named
CONVO_STEP_PARAMETERS. These parameters can be provided as a JSON string in the first argument or as key-value pairs separated by colons in subsequent arguments.These parameters are merged with
globalConvoStepParameters(defined in abeginstep) and are available to the step's execution context.// Example of a step with parameters in a JSON conversation format { "sender": "bot", "messageText": "Hello", "logicHooks": [ { "name": "CONVO_STEP_PARAMETERS", "args": ["stepTimeout:5000", "ignoreNotMatchedBotResponses:true"] } ] }Understand the Convo step structure
masterA conversation is composed of
ConvoStepobjects. Each step defines an interaction and can have several properties:sender: Determines the role of the step. Valid values areme(user input),bot(expected bot response),begin(setup),end(teardown), orinclude(importing other conversations).messageText: The text content for the step.sourceData: Used for asserting against non-textual data (e.g., JSON objects).not: A boolean flag. Iftrue, the step asserts that the expected condition/message did not occur.optional: A boolean flag. Iftrue, the step will not cause the entire conversation to fail if the assertion fails or if the bot does not respond (it will be skipped instead).asserters: A list of assertions to perform on the step.logicHooks: Custom logic to execute during the step.userInputs: Specific user input actions to perform.
Disable Botium installation tracking
masterBy default, this module sends anonymous installation information (operating system, architecture, and library version) to the Botium server to assist in project improvement.
You can disable this tracking by setting the
BOTIUM_ANALYTICS_environment variable tofalsebefore installing the package.export BOTIUM_ANALYTICS_=false npm install botium-core