mFAST C++ Library

repository·master·Indexed 42 days ago

https://github.com/objectcomputing/mfast

A high-performance C++ library for encoding and decoding the FAST (FIX Adapted for STreaming) protocol, optimized for high-volume data exchange. It provides limited support for FAST 1.2, including define and type tags, and can be translated into asm.js for use in web environments via Socket.io. The library features fast_decoder and fast_encoder classes, a visitor pattern for field instructions, and a specialized exception hierarchy based on boost::exception.

Tokens
7.3K
Snippets
22
Records
34
Agent score
51%

What's inside mFAST

  1. Introduction to mFAST

    master
    mFAST is a high-performance C++ library designed for encoding and decoding the FAST (FIX Adapted for STreaming) protocol. It is suitable for high-volume data exchanges and can be used to stream graphs and feeds to web and mobile clients.
  2. mFAST Protocol Support and Limitations

    master

    mFAST provides limited support for FAST protocol version 1.2.

    Supported features for FAST 1.2:

    • define and type tags in the FAST XML specification.
    • New enum/boolean types.
    • Generated sequence types support iterators.

    Unsupported features:

    • SET types.
    • BIT GROUP types.
    • TIMESTAMP types.
  3. Using mFAST in Web Environments via asm.js

    master
    mFAST source code has been translated into asm.js (an efficient subset of JavaScript). When used with Socket.io, this allows for high-volume data exchanges between browsers and servers, making it an ideal format for streaming data to web and mobile clients (and vice versa).
  4. Migrating from mFAST 1.1 to newer versions

    master

    If you are upgrading from version 1.1, be aware of the following backward-incompatible changes:

    1. Field Absence: To make an existing field absent, use msg_mref.omit_fieldName() instead of the deprecated msg_mref.set_fieldName().as_absent().
    2. Custom Visitors: If you have implemented custom visitors for mFAST cref/mref types, your visitor must be updated to handle the new enum cref/mref types.
  5. Build mFAST on Unix or macOS

    master

    To build mFAST on Unix or macOS, ensure that the boost library is already installed on your system paths. Follow these steps to clone the repository and build using CMake:

    1. Clone the repository recursively.
    2. Create a build directory.
    3. Run CMake and make.
    git clone --recursive https://github.com/objectcomputing/mFAST.git
    mkdir mFAST/build
    cd mFAST/build
    cmake ..
    make
  6. Run performance evaluation programs

    master

    The examples/performance_test directory contains programs used for evaluating mFAST performance. You can use mf_generic_decode for generic decoding or mf_fixed_decode for fixed-format decoding.

    mf_generic_decode

    Used for generic decoding. Requires an XML schema file and a data file.

    Arguments:

    • -t <file>: The XML schema file.
    • -f <file>: The data file to decode.
    • -hfix <int>: A parameter for fixed-length header/field configuration (e.g., 4).

    mf_fixed_decode

    Used for fixed-format decoding. Requires a data file.

    Arguments:

    • -f <file>: The data file to decode.
    • -hfix <int>: A parameter for fixed-length header/field configuration (e.g., 4).
    # Generic decoding usage
    mf_generic_decode -t example.xml -f complex30000.dat -hfix 4
    
    # Fixed decoding usage
    mf_fixed_decode -f complex30000.dat -hfix 4
  7. Encode messages with fast_encoder

    master

    The fast_encoder provides two primary ways to encode a message_cref into a byte stream:

    1. Encode into a fixed-size buffer

    std::size_t encode(const message_cref &message, char *buffer, std::size_t buffer_size, bool force_reset = false)

    • Returns: The size of the encoded byte stream.
    • Throws: An exception if the provided buffer_size is smaller than the required size.
    • force_reset: If true, the encoder resets and discards all existing history values before encoding.

    2. Encode into a std::vector<char>

    void encode(const message_cref &message, std::vector<char> &buffer, bool force_reset = false)

    • Behavior: Appends the encoded FAST stream to the existing contents of the buffer.
    • force_reset: If true, the encoder resets and discards all existing history values before encoding.
    // Fixed buffer usage
    char buf[512];
    std::size_t len = encoder.encode(msg, buf, sizeof(buf), true);
    
    // Vector append usage
    std::vector<char> vec;
    encoder.encode(msg, vec, false);
  8. Use the `field_instruction` base class

    master

    The field_instruction class is the primary base class for defining how individual fields are constructed, destroyed, and copied within mFAST. It manages metadata such as the field's name, namespace (ns), ID, type, and presence (mandatory vs. optional).

    Key Methods

    • construct_value(value_storage &storage, allocator *alloc): (Virtual) Must be implemented by derived classes to initialize the field's value in storage.
    • destruct_value(value_storage &storage, allocator *alloc): Cleans up the field's value.
    • copy_construct_value(...): Performs a deep copy of the field from a source value_storage to a destination.
    • accept(field_instruction_visitor &, void *): Used for the Visitor pattern to process instructions.
    • clone(arena_allocator &alloc): Creates a copy of the instruction using an arena allocator.

    Metadata Accessors

    • name(): Returns the field name.
    • ns(): Returns the namespace.
    • id(): Returns the unique field ID.
    • field_type(): Returns the field_type_enum_t.
    • optional(): Returns true if the field is optional.
    • is_array(): Returns true if the field type is a string, byte vector, or sequence.
  9. Cast between field_cref and field_mref using dynamic_cast_as

    master

    mFAST provides a dynamic_cast_as utility to safely transition between read-only (field_cref) and mutable (field_mref) references.

    • If you attempt to cast a read-only reference to a mutable type and the instruction does not support mutability, it will throw std::bad_cast.
    • The function automatically handles the retrieval of the necessary allocator when casting to a mutable reference.
    // Example: Casting a read-only reference to a mutable one
    field_cref readonly_ref = ...;
    
    try {
        field_mref mutable_ref = dynamic_cast_as<field_mref>(readonly_ref);
        mutable_ref.omit();
    } catch (const std::bad_cast& e) {
        // Handle case where field is not mutable
    }
  10. Implement the field_instruction_visitor interface

    master

    To process or traverse mFAST field instructions, you must implement the field_instruction_visitor abstract base class. This interface uses the Visitor pattern, requiring you to override the visit method for each specific instruction type you wish to handle. Each visit method accepts a pointer to the specific instruction type and a void * context pointer for passing user-defined data through the traversal.

    Supported instruction types for overriding include:

    • Integer types: int32_field_instruction, uint32_field_instruction, int64_field_instruction, uint64_field_instruction
    • Vector types: int32_vector_field_instruction, uint32_vector_field_instruction, int64_vector_field_instruction, uint64_vector_field_instruction
    • Specialized types: decimal_field_instruction, ascii_field_instruction, unicode_field_instruction, byte_vector_field_instruction, enum_field_instruction, set_field_instruction
    • Structural types: group_field_instruction, sequence_field_instruction, template_instruction, templateref_instruction
    #include <mfast/field_instructions.h>
    
    class MyInstructionProcessor : public mfast::field_instruction_visitor {
    public:
        // Example: handling an int32 instruction
        void visit(const mfast::int32_field_instruction *inst, void *data) override {
            // Process the instruction using 'inst' and 'data'
        }
    
        // You must implement all other pure virtual visit methods defined in the interface...
        void visit(const mfast::uint32_field_instruction *inst, void *data) override {}
        // ... etc
    };
    
    // Usage:
    // MyInstructionProcessor processor;
    // instruction->accept(processor, my_context_data);