nip (Node Input/output Piper)

The core usage of nip involves passing a JavaScript function as an argument to process input lines. The syntax for the js-function argument supports three styles:

1. Expression style: A single return statement.
2. Function style: A full function definition.
3. Closure style: A block of code that returns a function.

Arguments available to the function:

• line: The current line content.
• index: The current line index.
• cols: Number of columns (if using a splitter).
• lines: The array of all lines (useful with the -1 flag).

Return value behavior:

• If you return false: Nothing is sent to the output stream.
• If you return a string or number: That value is sent to the output stream.
• If you return anything else: The original line is sent to the output stream.

nip 'return line.substr(0, 10) + index'
nip 'function(line, index) { return line.toUpperCase(); }'
nip '/* setup code */ return function(line, i) { return line + i; }'

To use nip, you must provide JavaScript code that defines how input lines are transformed. You can provide this logic in two ways:

1. As a function-like string: Pass a string directly as the js-function argument.
2. Via a file: Use the --file flag to point to a .js file containing your transformation logic.

When using a string, you can use one of three formats:

• A simple return statement: 'return line.toUpperCase()'
• A full function definition: 'function(line, lineIndex) { return line.toUpperCase(); }'
• A setup block that returns a function: 'var setup = 123; return function(line) { return line; }'

Available Variables: When using the string format, the following variables are available in your scope:

• line: The current line being processed.
• index: The current line index.
• cols: The number of columns.
• lines: The total number of lines (only available if using the --first-line-only flag).

Return Values:

• Return a string to provide the transformed line.
• Return false to omit the line from the output.

# Example using a function-like string
nip 'return line.toUpperCase()' input.txt

# Example using a file
nip --file transform.js input.txt

The transformation logic (the [js-code] argument or the content of the --file) must be valid JavaScript that evaluates to a function. nip supports three formats for this function:

1. Return statement: A string starting with return (e.g., return function(line, index, cols, lines) { ... }).
2. Function declaration: A string starting with function (e.g., function(line) { return line.toUpperCase(); }).
3. Expression: A raw function expression (e.g., (line) => line.trim()).

The callback function receives the following arguments:

• line: The current line being processed.
• index: The index of the current line.
• cols: An array of columns (split by the column splitter).
• lines: The full content/context of the lines (depending on mode).

# Example using a function expression via CLI
nip "(line) => line.toUpperCase()" input.txt

# Example using a file containing a function
nip --file transform.js input.txt

The processAsStream function provides a way to read data from a file or stdin, split it into lines based on a regular expression, and process each line through a callback. The results of the callback are then written to an output stream.

Input Sources

• If options.filename is set to '-', the function reads from process.stdin.
• Otherwise, it creates a read stream from the file path provided in options.filename.

Callback Behavior

The options.callback is executed for every line. It receives the following arguments:

1. line: The raw string content of the current line.
2. lineNumber: The zero-based index of the current line.
3. columns: An array of strings created by splitting the line using options.colSeparatorRegex.

Return Values of the Callback:

• false: The line is skipped, and no newline is written to the output stream.
• string or number: The value is converted to a string and written to options.outStream, followed by the options.lineSeparator on the next line.
• Any other type: The original line is written to options.outStream, followed by the options.lineSeparator on the next line.

Options Object

const processAsStream = require('./lib/process-as-stream');
const fs = require('fs');

const options = {
  filename: 'input.txt',
  lineSeparatorRegex: /\r?\n/,
  colSeparatorRegex: /,/,
  outStream: fs.createWriteStream('output.txt'),
  lineSeparator: '\n',
  callback: function(line, lineNumber, columns) {
    // Process columns
    const transformed = columns[0].toUpperCase();
    return transformed;
  }
};

processAsStream(options, (err) => {
  if (err) console.error(err);
  else console.log('Processing complete');
});

The nip function is the primary entrypoint for processing files with custom line and column separators. It accepts an options object and processes a list of files sequentially.

Required options:

• files: An array of filenames to process.
• callback: A function called after each file is processed.

Optional options:

• lineSeparator: A string representing the line separator.
• lineSeparatorRegex: A regular expression for identifying line separators.
• colSeparatorRegex: A regular expression for identifying column separators.
• firstLineOnly: If true, only the first line of each file is processed (uses processAsBuffer internally).
• inPlace: If true, the output is written back to the original filename (provided the filename is not -). Note that for non-stdin files, it temporarily writes to filename + ".nip" and then renames it.
• outStream: The destination stream for output (defaults to process.stdout).
• doneCallback: A function called after all files in the files array have been processed. It receives the total number of files processed as an argument.

const nip = require('nip');

nip({
  files: ['input.txt', 'data.csv'],
  callback: (err) => {
    if (err) console.error(err);
    else console.log('File processed');
  },
  lineSeparator: '\n',
  colSeparatorRegex: /,/,
  inPlace: true,
  doneCallback: (total) => {
    console.log(`Finished processing ${total} files`);
  }
});