Overview of hog for consuming Hobbes logs
mainhog is a pre-written consumer designed to record structured data either locally to disk or to a remote process.repository·main·Indexed 22 days ago
https://github.com/morganstanley/hobbesA system for embedding dynamic expressions and evaluation within C++ processes, featuring strong type integration and binding capabilities. It provides tools for compiling expressions into C++ callables via hobbes::cc, binding C++ functions and class instances, and pushing application data to storage using the HSTORE macro and the hog utility for data consumption and fault recovery.
hog is a pre-written consumer designed to record structured data either locally to disk or to a remote process.Hobbes is a domain-targeting programming language and execution environment designed for ultra-low latency, high-performance integration with C/C++ applications. It is primarily used for managing the runtime of low-latency processes (like equities trading engines) that require dynamic, in-process rewriting of processing rules and persistence of structured logs without requiring restarts during working hours.
Key capabilities include:
Security and Safety Warning: Hobbes is designed for performance over sandboxing. It does not have a sandboxed runtime or runtime safety features. It provides direct access to memory and does not perform array bounds checks. It also supports remote compilation and execution of native code over a network (RPC), which should only be used within trusted internal networks due to the security implications of these design choices.
nil :: () -> (^x.(()+(a*x)))
nil _ = roll(|0=()|)
cons :: (a, ^x.(()+(a*x))) -> (^x.(()+(a*x)))
cons x xs = roll(|1=(x,xs)|)Hobbes is a specialized language designed for DevOps staff to manage the in-process configuration of extremely low-latency Order Managers (OM).
An Order Manager is responsible for maintaining the state of trade orders and executing logic based on market conditions (e.g., executing a limit order when a stock price hits a specific threshold).
Key requirements addressed by Hobbes:
Hobbes consists of two primary components designed to balance performance with flexibility:
Hobbes is designed to be hosted within C++ programs. You can achieve a hybrid architecture where:
This approach allows you to keep highly structured application parts in C++ while updating business logic in Hobbes without changing the core C++ codebase. For implementation details, refer to the Embedding Hobbes guide.
Hobbes implements polymorphism through Type Classes, which allow you to externally declare behaviors that a type supports. This enables writing generic functions that work across any data type implementing a specific capability (e.g., addition, multiplication, or printing).
You can define polymorphic functions using lambda syntax. The backslash \ starts the function, and the period . separates the argument list from the function body. Hobbes uses type inference to determine the necessary Type Class restrictions based on the operations used inside the function.
You can inspect the inferred type of an expression using the :t command. The resulting type notation follows the pattern:
Restrictions => (Input Types) -> Return Type
=> specifies the Type Classes required (e.g., Add a b c =>).=> describes the mapping from input to output (e.g., (a * b) -> c).Add: Required for types supporting the + operator.Equiv: For types supporting equivalence/equality.Multiply: For types supporting the * operator.Print: For types whose values can be printed.Hobbes supports pattern matching for classifying and destructuring data using match expressions. This generalizes C++ switch statements by allowing matching on multiple values simultaneously, binding variables to parts of the matched value, and using guard expressions for conditional branching.
match x y with).|paymentReceived=x|).where keyword to add a condition to a match row. The row is only selected if both the pattern matches and the guard evaluates to true._ pattern.match x y with
| 0 0 -> "foo"
| 0 1 -> "foobar"
| 1 0 -> "bar"
| 1 1 -> "barbar"
| 2 0 -> "chicken"
| 2 1 -> "chicken bar!"
| _ _ -> "beats me"Hobbes supports several aggregate types based on the ability to persist the underlying element type T:
T[]std::vector<T>std::tuple<T0, T1, ..., Tn>Hobbes uses a space-efficient binary format for persisted data. Files follow a header/body structure:
Because the header defines the schema, tools can perform efficient equality searches by calculating offsets based on the known sizes of the struct members. While the files are not human-readable, the hi REPL provides type-safe access to the data using the schema extracted from the header.
For complex parsing tasks that regular expressions cannot handle (like expression languages), hobbes provides a syntax to define LALR(1) parsers based on context-free grammars. You use the parse { RULES } syntax to construct a parser. Rules define both the syntax and "actions" (arbitrary hobbes code) that produce semantic values from the matched rules.
Key features include:
v:V) to use them in the rule's action.calc = parse {
E := x:E "+" y:T { x + y }
| x:E "-" y:T { x - y }
| x:T { x }
T := x:T "*" y:F { x * y }
| x:T "/" y:F { x / y }
| x:F { x }
F := "(" x:E ")" { x }
| x:V { x }
V := v:V d:D { v*10 + d }
| d:D { d }
D := "0" {0} | "1" {1} | "2" {2} | "3" {3} | "4" {4}
| "5" {5} | "6" {6} | "7" {7} | "8" {8} | "9" {9}
}Match expressions allow you to perform actions based on the value or type of an expression. They work top-down: the first valid case encountered is executed.
_ acts as a wildcard (default case) or an instruction not to bind a name to the matched element._).match is an expression, all branches must return the same type. Failing to do so results in a type unification error.When matching against multiple values (like a tuple), you must provide the correct number of underscores/patterns to match the expected column count.
Example of a simple match:
match 3 with
| 1 -> show("hello")
| 2 -> show("hobbes")
| _ -> show("oops!")Example of a match expression assigned to a variable:
hostport = match env with | "prod" -> "lnprd" | "qa" -> "euqa" | _ -> "ln123dev"match 3 with
| 1 -> show("hello")
| 2 -> show("hobbes")
| _ -> show("oops!")Hobbes handles arithmetic through Type Classes like Add, Subtract, Multiply, and Divide. These are available in the default namespace, allowing you to use operators like + on basic types (e.g., int, long) implicitly.
Note that operators are not hard-coded into the language; they are resolved by the compiler via these Type Class instances. You can extend arithmetic support to your own custom types by implementing the corresponding Type Class instance.
type counter = { count: int}
counterAdd = (\x y. { count = iadd(x.count,y.count)})
instance Add counter counter counter where
(+) = counterAdd