website·Indexed 33 days ago
https://docs.scramjet.org/A data-oriented framework for building 'Sequences'—small, asynchronous programs deployed as 'Transform as a Service'. The platform includes the Scramjet Cloud Platform (SCP) API, a CLI tool (`si`), and a framework supporting TypeScript and Python. It features specialized stream types such as :DataStream, :BufferStream, :NumberStream, and :MultiStream for functional data transformations and distributed parallel processing in JavaScript.
valueOf method. Because it extends DataStream, it supports all standard stream operations such as map and reduce, while adding numeric-specific utility methods.Scramjet provides several specialized stream classes to handle different data types. These streams support writing, reading, or transform modes (exclusively). Scramjet optimizes performance by chaining multiple transforms (e.g., .map().map().filter()) into a single transform stream rather than creating multiple intermediate streams.
Available stream classes:
DataStream: Basic object stream of any type.StringStream: Stream of strings.BufferStream: Stream of buffers.NumberStream: Stream of numbers.WindowStream: Stream of windows of objects.MultiStream: A group of streams.Scramjet provides predefined templates to help users apply standard settings and quickly start building stream-processing applications. Templates are available for the following languages:
const fs = require('fs');
const { BufferStream } = require('scramjet');
fs.createReadStream("pixels.rgba")
.pipe(new BufferStream()) // pipe a buffer stream into scramjet
.breakup(4) // split into 4 byte fragments
.parse((buffer) => [
buffer.readInt8(0), // the output is a stream of R,G,B and Alpha
buffer.readInt8(1), // values from 0-255 in an array.
buffer.readInt8(2),
buffer.readInt8(3),
]);Scramjet Cloud Platform simplifies the integration of cloud and local processing systems, removing the need for complex VPNs or public API gateways. A core feature is 'Sequences'—long-lived cloud functions designed for high-performance data processing. Key benefits include:
StringStream.from(async () =>
(await fetch("https://example.com/data/article.txt")).text()
)
.lines()
.append("\r\n")
.pipe(fs.createWriteStream("./path/to/file.txt"));DataStream..window method and is used to perform moving window calculations.StreamWorker class is intended for internal use within the Scramjet framework. It provides low-level control over subprocesses, specifically handling the spawning of workers, communication between processes, and the delivery of data streams to child processes.The repo-checker sequence retrieves the following metrics from supported platforms: