msnodesqlv8

repository·master·Indexed 41 days ago

https://github.com/timelorduk/node-sqlserver-v8

Microsoft Driver for Node.js SQL Server compatible with all versions of Node. Version 5.5.0 provides a native ODBC driver for SQL Server and Sybase ASE, featuring a promises API, parameterized queries, and high-performance bulk inserts via the native BCP protocol. Includes specialized Docker environments for testing SQL Server 2022 with Always Encrypted functionality.

Tokens
38.8K
Snippets
135
Records
158
Agent score
100%

What's inside msnodesqlv8

  1. What is msnodesqlv8?

    master
    The msnodesqlv8 npm module is a NodeJS driver for connecting to SQL Server databases. It is a fork of the Microsoft SQL server ODBC module msnodesql, reworked for compatibility with modern versions of Node.js. It uses a mixture of C++ and JavaScript and provides a JavaScript API that is a superset of the original Microsoft library functionality.
  2. Manage database connections with sql.Pool

    master

    The sql.Pool class manages a set of connections to handle concurrent work efficiently. It includes features like periodic heartbeats to check connection integrity, inactivity timeouts, and the ability to pause, resume, or cancel individual queries within the pool.

    const sql = require('msnodesqlv8')
    
    const pool = new sql.Pool({
      connectionString: 'Driver={ODBC Driver 13 for SQL Server};Server=(localdb)\node;Database=scratch;Trusted_Connection=yes;'
    })
    
    pool.on('open', (options) => console.log('Pool opened', options))
    pool.on('error', e => console.error(e))
    
    pool.open()
    
    // Submitting a query
    const q = pool.query('SELECT * FROM MyTable')
    q.on('done', () => console.log('Query finished'))
    
    // Controlling a query
    q.cancelQuery() // Stops the query
    q.pauseQuery()  // Pauses the query
    q.resumeQuery() // Resumes the query
  3. Use the Promise-based API for modern async/await

    master

    The library provides a modern asynchronous API where all promise-based methods are collected under a .promises property on the respective objects. This allows you to use async/await syntax instead of callbacks.

    Key promise interfaces include:

    • PoolPromises: Provides open() and close() for connection pools.
    • SqlClientPromises: Provides query(), callProc(), and open() for direct SQL client operations.
    • ConnectionPromises: Provides query(), callProc(), prepare(), getTable(), close(), and cancel() for active connections.
    • BulkTableMgrPromises: Provides select(), insert(), delete(), and update() for bulk table operations.
    • PreparedPromises: Provides query() and free() for prepared statements.
  4. Difference between query() and queryRaw()

    master

    The query() method returns results as an array of JavaScript objects (one object per row). The queryRaw() method returns the raw database rows in a structured object containing meta and rows, where rows is an array of arrays.

    // query() returns: [{ col1: val, col2: val }, ...]
    conn.query("SELECT * FROM Employee", (err, res) => { ... });
    
    // queryRaw() returns: { meta: [...], rows: [[val1, val2], [val1, val2]] }
    conn.queryRaw("SELECT * FROM Employee", (err, res) => { ... });
  5. Important usage constraint: Do not use in UI threads

    master
    msnodesqlv8 is a native addon. Do not call it from a UI thread (such as an Electron renderer process or Next.js client components). Instead, execute database logic within a server route, API handler, or a Node.js worker_thread.
  6. Register Stored Procedures for non-SQLServer ODBC connections

    master

    When using SQL Server, the library automatically fetches and caches stored procedure definitions. However, for other ODBC connections (such as Sybase), you must manually register the stored procedure using the procedureMgr to ensure the library understands the parameter mapping.

    To register a procedure:

    1. Access the procedure manager via connection.procedureMgr().
    2. Define parameters using pm.makeParam(spName, paramName, type, size, isOutput).
    3. Add the procedure to the manager using pm.addProc(spName, params).
    4. Set the appropriate dialect using proc.setDialect(pm.ServerDialect.Sybase) (or the relevant dialect for your connection).
    const pm = connection.procedureMgr();
    const params = [
      pm.makeParam(spName, '@param1', 'varchar', 30, false),
      pm.makeParam(spName, '@param2', 'int', 0, true) // true for output
    ];
    const proc = pm.addProc(spName, params);
    proc.setDialect(pm.ServerDialect.Sybase);
  7. Use the Table Manager for bulk operations

    master

    The Table Manager (conn.tableMgr()) provides a high-level API for interacting with a specific table. It allows you to perform bulk inserts, updates, deletes, and selects within a single transaction, reducing network overhead.

    Workflow:

    1. Get the manager: var tm = conn.tableMgr();
    2. Bind to a table: tm.bind('TableName', callback);
    3. Use the bulkMgr provided in the callback to perform operations.

    Key Capabilities:

    • Bulk Insert: bulkMgr.insertRows(arrayOfObjects, callback)
    • Bulk Update: bulkMgr.updateRows(arrayOfObjects, callback). Note: You must call bulkMgr.setUpdateCols(cols) before updating to specify which columns are being modified.
    • Bulk Delete: bulkMgr.deleteRows(rows, callback). Requires setWhereCols to define the criteria.
    • Bulk Select: bulkMgr.selectRows(keys, callback). Requires setWhereCols for non-primary key selections.
    • Batching: Use bulkMgr.setBatchSize(size) to split large vectors into smaller chunks for submission.
    var tm = conn.tableMgr();
    
    tm.bind('Employee', function(bulkMgr) {
        var data = [{ Name: 'John', ID: 1 }];
        
        // Bulk Insert
        bulkMgr.insertRows(data, (err, res) => {
            // Handle completion
        });
    });
  8. Supported Platforms for msnodesqlv8

    master

    Windows

    msnodesqlv8 runs on both 32-bit and 64-bit versions of Node.js hosted on Windows. It supports all major Node.js versions (tested for >= 10).

    Linux

    If running on Linux, you must install the Microsoft ODBC driver. Version >= 17.5 is recommended and has been tested. Supported distributions include:

    • Ubuntu 18.04 / 20.04
    • Alpine 3.12
    • Debian 10.5
    • Fedora 32
    • Windows Subsystem for Linux 2 (WSL2)

    macOS (darwin)

    Native binaries are supplied via NPM. The driver has been tested on macOS High Sierra.

  9. Use BCP for fast bulk inserts

    master

    BCP (Bulk Copy Program) allows for high-speed data insertion by binding columns in memory and performing a memory copy to the server. This requires ODBC Driver 17 or 18.

    To use BCP, you must use the tableMgr to get a table object and then call setUseBcp(true) on that table.

    // Example pattern for BCP
    // 1. Get the table via the connection's table manager
    const tm = theConnection.tableMgr();
    const table = await tm.getTable(this.tableName);
    
    // 2. Enable BCP on the table
    table.setUseBcp(true);
    
    // 3. Insert rows using the table instance
    await table.insertRows(rows);
  10. How statement types and lifecycles work

    master

    The project uses an interface-based architecture for SQL statements, categorized into three types to handle different execution patterns:

    1. TransientStatement: Used for one-off query executions.
    2. PreparedStatement: Used for parameterized, reusable statements.
    3. TvpStatement: Used for Table-Valued Parameters (TVP).

    All statements follow a standard lifecycle:

    1. Creation: Created via OdbcStatementFactory.
    2. Preparation: (Specific to PreparedStatement) The statement is prepared for execution.
    3. Execution: The statement is executed with provided parameters.
    4. Result Processing: Metadata for the result is collected.
    5. Fetching: Rows are retrieved in batches.
    6. Next Result Set: Moving to subsequent result sets if they exist.
    7. Cleanup: The statement is freed.
  11. Use the 'efficient' scaling strategy for optimized connection pooling

    master

    The efficient scaling strategy optimizes pool growth by only creating new connections when existing idle connections are exhausted. This is ideal for variable or bursty workloads where you want to minimize connection overhead and maximize reuse of idle connections.

    Behavior

    • Initial creation: Creates connections only up to the floor value on pool open.
    • Conservative growth: Only grows the pool when the work queue exceeds the number of available idle connections.
    • Incremental scaling: Adds connections based on actual demand using the scalingIncrement value.

    Configuration Options

    When using scalingStrategy: 'efficient', you can tune the following parameters:

    • floor: The number of connections to create immediately upon pool open.
    • ceiling: The maximum number of connections allowed in the pool.
    • scalingStrategy: Set to 'efficient' to enable this behavior.
    • scalingIncrement: The number of connections to add when the pool needs to grow.
    const pool = new sql.Pool({
      connectionString: '...',
      floor: 5,           // Start with 5 connections
      ceiling: 20,        // Maximum 20 connections  
      scalingStrategy: 'efficient',    // Use efficient strategy
      scalingIncrement: 3              // Grow by 3 when needed
    })
  12. Handle SQL Variant types

    master

    When the driver detects a sql_variant type, it automatically queries the underlying type information and interprets the column as that specific type. For example, a sql_variant containing an integer will be returned as a JavaScript number type.

    c.query("select cast(10000 as sql_variant) as data;",  (err, res) => {});