Neon Bindings Examples

The gzip-stream example demonstrates how to build a Node.js "through" stream using Neon. This implementation creates a Transform stream that is both readable and writeable, offloading CPU-intensive gzip compression to the Node worker thread pool. This pattern allows you to integrate high-performance Rust-based processing into standard Node.js stream pipelines while maintaining backpressure and asynchronous behavior.

// The example uses a glue function to wrap the Neon CompressStream into a Node.js Transform stream
const stream = compress();
// 'stream' can now be used in standard Node.js piping: 
// sourceStream.pipe(stream).pipe(destinationStream);

The repository contains several specialized examples:

• async-sqlite: Provides an async interface to a SQLite database.
• cpu-count: Returns the number of CPUs available on the system.
• gzip-stream: Demonstrates how to asynchronously compress a stream of data.
• hello-world: A basic example that returns a JS String with a greeting message.
• tokio-fetch: Demonstrates asynchronously fetching a Node.js release date.

Because SQLite is a synchronous interface that blocks the executing thread, this implementation uses a multi-threaded pattern to maintain JavaScript responsiveness:

1. Dedicated Database Thread: A single Rust thread is spawned specifically for performing database operations. This avoids the need for a connection pool since SQLite is naturally single-threaded.
2. Communication via MPSC: A multi-producer, single-consumer (mpsc) channel is used for one-way communication from the JavaScript main thread to the database thread. The sender side of this channel is stored within a JavaScript class (using #[neon::export(class)]).
3. Event Callbacks: To return data back to JavaScript, the database thread uses a neon::event::Channel. This allows the Rust thread to send events back to the JavaScript main thread to resolve promises or trigger callbacks once the query is complete.

To use the CompressStream Neon module within a standard Node.js Transform stream, you must implement the transform and flush methods. The glue code adapts the CompressStream Promise-based API to the callback-based requirements of Node.js streams.

function compress() {
    const stream = new CompressStream();

    return new Transform({
        transform(chunk, encoding, callback) {
            stream
                .compress(encoding, chunk)
                .then(data => callback(null, data))
                .catch(callback);
        },

        flush(callback) {
            stream
                .finish()
                .then(data => callback(null, data))
                .catch(callback);
        }
    });
}

The examples repository uses the npm 7 [workspaces] feature. To set up the development environment, ensure you are using npm 7 or higher, clone the repository, and run npm install from the root directory.

# Ensure npm 7+ is installed
npm --version

git clone https://github.com/neon-bindings/examples.git
cd examples
npm install

The cpu-count example provides a single function, getNumCpus, which returns the number of CPUs available on the current platform. This is useful for determining concurrency limits or optimizing workload distribution in Node.js applications.

const { getNumCpus } = require(".");

console.log(getNumCpus());

The hello-world example demonstrates a basic Neon binding by exposing a single function named hello. This function returns the string "hello". It serves as a minimal template for how to expose simple functions from Rust to JavaScript.

const { hello } = require(".");

console.log(hello());

The async-sqlite example provides a Database class that allows you to perform asynchronous SQLite operations from JavaScript. This pattern is useful for preventing the JavaScript main thread from blocking during synchronous SQLite queries by offloading work to a dedicated Rust thread.

To use it, instantiate the Database class and use await on methods like insert and byId to handle the asynchronous results returned from the database thread.

const { Database } = require(".");

(async () => {
    const db = new Database();

    const id = await db.insert("Marty McFly");
    const name = await db.byId(id);

    console.log(name);
})();

The nodeReleaseDate function asynchronously fetches the release date for a specified Node.js version from nodejs.org. In the underlying Rust implementation, this is an async fn that utilizes TryFromJs and TryIntoJs for ergonomic data conversion between JavaScript and Rust.

// Returns a Promise that resolves to the release date string
const date = await nodeReleaseDate('v18.0.0');

The CompressStream class is the core Neon module for gzip compression. It provides methods to process data in chunks and finalize the compression stream.

// Create a new instance
const stream = new CompressStream(level);

// Compress a chunk
const buffer = await stream.compress(chunk, encoding);

// Finalize compression
const finalBuffer = await stream.finish();