msgpackr

msgpackr provides aliases for users who prefer encoder/decoder terminology. The following mappings are available:

By enabling the structuredClone: true option in a Packr instance, you can support structured cloning. This allows for:

• Preserving object identity (circular references).
• Preserving certain typed objects like Error, Set, RegExp, and TypedArray instances.

Note: This option is disabled by default because reference checking and extensions degrade performance by approximately 25-30%.

import { Packr } from 'msgpackr';

let obj = {
	set: new Set(['a', 'b']),
	regular: /a\spattern/
};
obj.self = obj;

let packr = new Packr({ structuredClone: true });
let serialized = packr.pack(obj);
let copy = packr.unpack(serialized);

copy.self === copy; // true
copy.set.has('a'); // true

The record extension distinguishes between arbitrary maps (like JS Map) and well-defined object structures (records). When multiple objects share the same structure, msgpackr reuses the structure definition instead of re-encoding keys, resulting in more compact encodings and 2-3x faster decoding performance.

By default, a new Packr instance has the record extension enabled. For large datasets with repeating nested objects, simply using Packr provides these benefits automatically.

import { Packr } from 'msgpackr';
let packr = new Packr();
packr.pack(bigDataWithLotsOfObjects);

The Record Structure extension (using extension ID 0x72 or 'r') allows for efficient encoding of objects by defining a schema once and then using a single identifier byte for subsequent instances.

How it works:

1. Declaration: An extension declaration is followed by a MessagePack array defining the field names.
2. Identifier: The extension data byte identifies the start of a record. This identifier must be between 0x40 and 0x7f.
3. Encoding: Subsequent uses of the identifier in the stream are parsed as record instances. The parser reads the next n values (where n is the number of fields defined in the schema) as the field values.

Example Binary Layout: [0xd4 (fixext1)][0x72 ('r')][0x40 (ID)][Array of field names] ... [0x40 (ID)][Value 1][Value 2]

To achieve maximum performance, you can use the following techniques:

Buffer Reuse

Avoid expensive buffer allocations by reusing existing buffers. When using Node addons, you can use memcpy to copy data into existing buffers. When calling unpack, you can pass a second parameter indicating the effective size of the available data in the buffer.

Arena Allocation (useBuffer())

Use the useBuffer method to provide a buffer for the serialization process. This allows you to reuse the same buffer for multiple serialization operations, reducing GC pressure and allocation overhead. This is optional; msgpackr handles buffer cleanup automatically via GC if useBuffer is not used.

In version 2.0, the randomAccessStructure option and the internal struct.js module have been removed.

If you relied on randomAccessStructure: true, you should now use the standalone structon package.

Compatibility Note: Data encoded with randomAccessStructure: true in older versions of msgpackr is fully compatible with structon, and vice versa, as they share the same binary format.

import { Packr } from 'msgpackr';
import { createStructon } from 'structon';

const Structon = createStructon(Packr);
const codec = new Structon({ structures: [] });

To use msgpackr in your project, install it using npm:

npm i msgpackr

msgpackr works in modern browsers. You can load it via a UMD script which creates a msgpackr global:

<script src="node_modules/msgpackr/dist/index.js"></script>

For module-based development, it is recommended to import only the specific functions you need to minimize bundle size:

import { unpack } from 'msgpackr/unpack';

Available Bundles:

• dist/index.js: Standard UMD bundle.
• dist/index.min.js: Minified bundle.
• dist/index-no-eval.js: A version that excludes dynamic code evaluation (eval/Function) for environments with strict Content Security Policies (CSP). Note that using this version may reduce performance for record structures.

msgpackr is compatible with Deno and Bun:

• Deno: Load via the deno.land registry or use the NPM module loader: import { unpack } from 'npm:msgpackr'.
• Bun: Use it like any other NPM package.

To improve storage efficiency in databases or files, you can use shared structures. This allows multiple objects with common structures to reference a single shared definition.

To use this, provide a structures array to the Packr constructor. If you are persisting data, you must implement getStructures and saveStructures to load and save the shared structure definitions. msgpackr automatically adds new structures (up to a limit) in an incremental, compatible way.

import { Packr, unpack, pack } from 'msgpackr';

let packr = new Packr({
	getStructures() {
		// Load existing structures from a file, DB, or KV store
		return unpack(readFileSync('my-shared-structures.mp')) || [];
	},
	saveStructures(structures) {
		// Persist the updated structures array
		writeFileSync('my-shared-structures.mp', pack(structures));
	}
});

You can enable or disable the record structure extension when creating a Packr, Unpackr, PackrStream, or UnpackrStream instance using the useRecords option.

• useRecords: true (Default): Uses the record extension for optimized encoding/decoding.
• useRecords: false: Disables the extension (standard/compatibility mode). All objects are serialized as MessagePack maps and deserialized as JS Objects.