venom

Venom supports several ways to inject variables into your testsuites. Variables can be strings, lists, or objects.

1. Command line flag (--var): Best for individual variables.
   • venom run --var="foo=bar"
   • venom run --var='foo_list=["biz","buz"]'
   • venom run --var='foo={"biz":"bar","biz":"barr"}'
2. Variable definition files (--var-from-file): Best for large sets of variables. The file must be a YAML dictionary.
   • venom run --var-from-file variables.yaml
3. Environment Variables: Venom looks for environment variables prefixed with VENOM_VAR_.
   • export VENOM_VAR_foo=bar && venom run *.yml

Precedence: Command line flags override --var-from-file, which overrides environment variables.

$ venom run --var="foo=bar"
$ venom run --var='foo_list=["biz","buz"]'
$ venom run --var='foo={"biz":"bar","biz":"barr"}'
$ venom run --var-from-file variables.yaml
$ export VENOM_VAR_foo=bar
$ venom run *.yml

Each command execution produces a result object. When multiple commands are executed in one testcase, results are indexed as commands0, commands1, etc.

Result Structure

• search: The state of the mail before the command was executed.
• mail: The state of the mail after the command was executed (only populated for commands that modify content, like append or flag).
• err: The error message if the command failed.
• timeseconds: Duration of execution.

Assertion Syntax Examples

To assert that a command succeeded and the mail subject changed:

• result.commands.commands0.err ShouldBeEmpty
• result.commands.commands0.mail.subject ShouldEqual "New Subject"

To assert specific flags in a list:

• result.commands.commands1.mail.flags.flags0 ShouldEqual "Flag1"

{
  "result": {
    "commands": [
      {
        "search": {
          "mailbox": "INBOX",
          "from": "from@mail-before-command-execution.com",
          "to": "to@mail-before-command-execution.com",
          "subject": "Title of mail BEFORE command execution",
          "body": "Body content of mail BEFORE command execution",
          "flags": [
            "Flag1",
            "Flag2"
          ]
        },
        "mail": {
          "from": "from@mail-after-command-execution.com",
          "to": "to@mail-after-command-execution.com",
          "subject": "Title of mail AFTER command execution",
          "body": "Body content of mail AFTER command execution",
          "flags": [
            "Flag1",
            "Flag2",
            "Flag3"
          ]
        },
        "err": "Error of the command",
        "timeseconds": 1.5
      }
    ]
  }
}

In venom, a TestSuite is a collection of TestCases defined in a single YAML file. Each TestSuite contains multiple TestCases, and each TestCase consists of an ordered set of steps. Each step is driven by an executor (the type field) that defines the behavior (e.g., executing a script, making an HTTP request, or querying a database).

TestSuite Structure

A typical YAML structure includes:

• name: Title of the suite.
• description: Markdown description.
• vars: Testsuite-level variables.
• testcases: A list of testcase objects.

Each testcase contains:

• name: Name of the testcase.
• steps: A list of steps, where each step can have a type, script, assertions, vars, and retry logic.

name: Title of TestSuite
description: A detailed description of the TestSuite, in markdown.
testcases:
- name: TestCase with default value, exec cmd. Check if exit code != 1
  steps:
  - script: echo 'foo'
    type: exec

- name: Title of First TestCase
  steps:
  - script: echo 'foo'
    assertions:
    - result.code ShouldEqual 0
  - script: echo 'bar'
    assertions:
    - result.systemout ShouldNotContainSubstring foo
    - result.timeseconds ShouldBeLessThan 1

The range attribute allows you to iterate over different data types within a step. During iteration, the following contextual variables are available:

• Arrays ([]interface{}): Iterates over each value. Use .index (or .key) for the current index and .value for the item.
• Maps (map[string]interface{}): Iterates over keys. Use .index for the current index, .key for the current key, and .value for the current value.
• Integers (int): Performs the target step n times. Use .index (or .key or .value) for the current iteration index.
• Templated Strings: Strings that resolve to one of the above types (e.g., from vars or previous step results).

- name: range with hardcoded array
  steps:
  - type: exec
    range:
      - actual: hello
        expected: hello
      - actual: world
        expected: world
    script: echo "{{.value.actual}}"
    assertions:
    - result.code ShouldEqual 0
    - result.systemout ShouldEqual "{{.value.expected}}"