AsmTK Assembler Toolkit

AsmTK is an assembler toolkit built on top of AsmJit, designed primarily for AOT (Ahead-Of-Time) code generation. Key capabilities include:

• Runtime Architecture Selection: Supports both X86 and X64 modes that can be selected at runtime, independent of how the application was compiled.
• Comprehensive Parsing: The AsmParser supports all AsmJit instruction sets and named labels.
• Instruction Aliases: Provides extra instruction aliases (like movsb, cmpsb, sal) that are recognized by the parser, whereas AsmJit typically provides only the generic versions (like movs, cmps).
• Flexible Emission: Assembles to any BaseEmitter, allowing you to switch between Assembler and BaseBuilder at runtime and perform post-processing on the results.

The AsmTokenizer class is used to parse assembly text into a sequence of AsmToken objects. You can provide input via set_input(const uint8_t* input, size_t size), which supports non-null-terminated buffers like string views. To retrieve the next token, call next(AsmToken* token, ParseFlags parse_flags). If you need to re-parse the current token, use put_back(AsmToken* token) to rewind the tokenizer's cursor.

asmtk::AsmTokenizer tokenizer;
// Assuming 'buffer' is a uint8_t array of size 'len'
tokenizer.set_input(buffer, len);

asmtk::AsmToken token;
while (tokenizer.next(&token) != asmtk::AsmTokenType::kEnd) {
    // Process token
}

You can manually manage the input buffer of an AsmParser using the following methods:

• set_input(const char* input, size_t size = SIZE_MAX): Sets the source text for the parser. If size is SIZE_MAX, it uses strlen(input).
• input(): Returns a pointer to the current input buffer.
• is_end_of_input(): Returns true if the parser has reached the end of the input.
• current_command_offset(): Returns the offset of the current command within the input.

The asmtk::AsmParser class is used to parse assembly text and emit instructions into an asmjit::BaseEmitter.

To use it, construct an AsmParser with a pointer to an asmjit::BaseEmitter. You can then use the parse() method to process a complete buffer of assembly text, or parse_command() to parse individual commands sequentially.

If the parser encounters a symbol it does not recognize, it will trigger the UnknownSymbolHandler if one has been provided via set_unknown_symbol_handler().

// Example usage pattern
asmjit::BaseEmitter* emitter = /* ... */; 
asmtk::AsmParser parser(emitter);

// Parse a full buffer
asmtk::Error err = parser.parse("mov eax, 1\nret", SIZE_MAX);
if (err != asmjit::Error::kOk) {
    // Handle error
}

When AsmParser encounters a symbol that is not defined, it calls a user-provided UnknownSymbolHandler callback. This allows you to resolve symbols dynamically (e.g., mapping them to specific memory addresses or external functions).

UnknownSymbolHandler signature:

typedef Error (ASMJIT_CDECL* UnknownSymbolHandler)( 
    AsmParser* parser, 
    asmjit::Operand* out, 
    const char* name, 
    size_t size
);

To register a handler, use set_unknown_symbol_handler(handler, data). You can pass a void* pointer as data to provide context to your callback function.

asmtk::Error my_handler(asmtk::AsmParser* parser, asmjit::Operand* out, const char* name, size_t size) {
    // Logic to resolve 'name' and populate 'out'
    return asmjit::Error::kOk;
}

// ...
parser.set_unknown_symbol_handler(my_handler, nullptr);

An AsmToken contains the type, the raw data pointer, and the size of the token. It also provides helper methods to retrieve parsed numeric values or character representations.

// Get metadata
asmtk::AsmTokenType type = token.type();
const uint8_t* data = token.data();
size_t size = token.size();

// Get numeric values (if applicable)
double f = token.f64_value();
int64_t i = token.i64_value();
uint64_t u = token.u64_value();

// Get character/byte representations
const char* chars = token.value_chars();
const uint8_t* bytes = token.value_bytes();

The ParseFlags enum allows you to modify the behavior of the next() method during tokenization.

enum class ParseFlags {
  kNone = 0,
  //! Don't attempt to parse number (always parse symbol).
  kParseSymbol = 0x00000001u,
  //! Consider dashes as text in a parsed symbol.
  kIncludeDashes = 0x00000002u
};

The AsmTokenType enum defines the categories of tokens identified by the tokenizer. These include symbols, numbers, operators, and delimiters.

enum class AsmTokenType : uint8_t {
  kEnd,      // End of input
  kNL,       // New line
  kSym,      // Symbol
  kNSym,     // Non-symbol
  kU64,      // Unsigned 64-bit integer
  kF64,      // 64-bit float
  kLCurl,    // '{'
  kRCurl,    // '}'
  kLBracket, // '['
  kRBracket, // ']'
  kLParen,   // '('
  kRParen,   // ')'
  kAdd,      // '+'
  kSub,      // '-'
  kMul,      // '*'
  kDiv,      // '/'
  kComma,    // ','
  kColon,    // ':'
  kOther,    // Other character
  kInvalid   // Invalid token
};