Varnish Cache Documentation

Varnish Test Case (.vtc) files are used to describe testing scenarios involving scripted HTTP-talking entities and one or more Varnish instances.

Parsing Behavior:

• Files are read word-by-word with minimal tokenization.
• Syntax errors may not be detected until the specific action is reached during test execution.
• Parsing errors often trigger an assert. If an assertion fails, check the source file and line number provided in the error message.

Basic Rules:

• Words and Strings: Words are separated by whitespace. Single-line strings are enclosed in double-quotes ("..."). Multi-line strings are enclosed in curly brackets ({...}).
• Comments: Lines starting with # are comments. Empty lines and leading whitespace are ignored.
• Commands: Each line should contain at most one command. The first word is the command, followed by its arguments. To continue a command onto a new line without breaking the argument string, use a backslash (\) to escape the newline.

Varnish consists of several key binaries and processes:

Core Processes

• varnishd: The main Varnish cache program. When running, it spawns two processes:
  • master: A manager process that handles configuration, parameters, and VCL compilation. It does not handle HTTP traffic directly.
  • worker: The process that performs the actual work (handling HTTP traffic). If the worker dies, the master attempts to restart it.

Monitoring and Diagnostic Tools

• varnishstat: Displays Varnish statistics counters.
• varnishlog: Presents the Varnish transaction log in its native format.
• varnishtop: Provides a real-time "top-X" list view of the transaction log.
• varnishncsa: Presents the transaction log in NCSA format.
• varnishhist: Displays a response time histogram using ASCII art.
• varnishtest: A tool to test varnishd behavior by simulating backends and clients based on test scripts.

Varnish uses a multi-process architecture designed to isolate security risks through several barriers. The core design separates management tasks from traffic handling to ensure that a compromise or failure in the traffic-handling component does not compromise the entire system.

Key Components and Barriers

• Manager (MGT): The central process started by the Administrator. Its primary responsibility is to manage the lifecycle of the Child process, not to cache content. It does not trust the Child process.
• Child (CLD): The process that handles actual web traffic (HTTP requests/responses). It operates in an unprotected domain and is subject to potential exploits or DoS attacks from anonymous internet users. To mitigate risk, the Manager starts the Child with the lowest possible privileges and restricts its file descriptors.
• Administrator (ADMIN) vs. Operator (OPER):
  • Administrator: Has full control and authority over the system.
  • Operator: Used for monitoring and data collection. Varnish treats the Operator role as untrusted (a one-way relationship) because operators are often automated scripts that may contain bugs.
• VCC Compiler: Runs in a separate child process to prevent memory leaks or flaws in the compiler from affecting the Manager process.

Communication Channels

The Child process can only communicate back to the Manager through three strictly defended channels:

1. An exit code.
2. A CLI response.
3. Writing to the shared memory file (VSM) used for statistics and logging.

Varnish Configuration Language (VCL) uses several distinct objects to represent the lifecycle of an HTTP request and response. You can access and manipulate these objects within different VCL subroutines to control how requests are handled, how backends are queried, and how responses are delivered to clients.

Key objects include:

• req: The initial request object created when Varnish receives a request. Most logic in vcl_recv operates on this object.
• bereq: The backend request object. This is constructed based on req and is used when Varnish sends the request to a backend server.
• beresp: The backend response object. It contains the headers of the object received from the backend. Use this object in vcl_backend_response to modify responses coming from the server.
• resp: The HTTP response object immediately before it is delivered to the client. This is typically modified in vcl_deliver.
• obj: The object as it is stored in the cache. This object is read-only.