ssh2-sftp-client

repository·master·Indexed 21 days ago

https://github.com/theophilusx/ssh2-sftp-client

A promise-based wrapper for the ssh2 library providing a high-level API for SFTP operations in Node.js. It includes the SftpClient class for managing connections, file transfers (including fastGet and fastPut for concurrency), directory management, and file attribute retrieval. Requires Node.js version 20.x or higher.

Tokens
18.5K
Snippets
54
Records
84
Agent score
73%

What's inside ssh2-sftp-client

  1. Overview of ssh2-sftp-client

    master

    The ssh2-sftp-client package provides a high-level, promise-based SftpClient class for Node.js. It acts as a wrapper around the ssh2 library, which is a pure JavaScript event-based SSH2 implementation.

    Because this package is a decorator for ssh2, many of the underlying configuration options and behaviors are inherited from the ssh2 project. For deep technical details on SSH2 protocols or specific low-level options, refer to the ssh2 documentation.

  2. Avoid concurrent operations on a single connection

    master
    Although the SFTP protocol technically supports concurrent operations, executing multiple requests in parallel over the same connection (e.g., using Promise.all() on multiple get() calls) is often unreliable and provides little performance benefit in Node.js. It is recommended to execute SFTP requests sequentially to ensure stability.
  3. Handle events and exceptions in ssh2-sftp-client

    master

    Because ssh2-sftp-client provides a Promise-based API over an event-based module (SSH2), it uses internal strategies to manage asynchronous events like error, end, and close:

    • Promise Management: When a method is called, the client attaches listeners to these events to reject the enclosing promise. It tracks if an error has already been handled to ignore subsequent, less informative errors.
    • Connection Delays: A small delay (approx. 500ms) is used during connect() to ensure all late-firing events are caught before handlers are removed.
    • Termination: The end() method sets up listeners to ignore additional events during connection teardown.
    • Global Handlers: Global handlers catch unhandled events, reset the sftp property, and log errors to the console to prevent script crashes.
  4. Use fastPut() and fastGet() with caution

    master

    The fastPut() and fastGet() methods are significantly faster than put() and get() when they work, but they are highly dependent on SFTP server capabilities. Some servers struggle with concurrent connections or specific packet sizes.

    If you are developing code that must run against various unknown SFTP servers, consider:

    1. Using the standard get() and put() methods for better compatibility.
    2. Providing an option for users to choose the method.
    3. Tweaking options like the number of concurrent connections or specific packet sizes if issues arise.
  5. How error and event handling works in SftpClient

    master

    The SftpClient wraps the event-based ssh2 API into a Promise-based API.

    Promise-based interactions

    During an active API call, the client attaches temporary event listeners (error, end, close) to the underlying socket. If an event occurs while a promise is pending, the promise is rejected to communicate the failure to your try/catch or .catch() block.

    Global event handlers

    If an event (like a lost connection) occurs outside the context of an active API call, no promise exists to catch it. In these cases, the client uses global event handlers which, by default, log the event and invalidate the current connection.

    To handle these 'out-of-band' events, you can provide custom callbacks when instantiating the SftpClient constructor:

    // Example of providing global event listeners
    let sftp = new Client({ 
      error: (err) => console.error('Global error:', err),
      end: () => console.log('Connection ended'),
      close: () => console.log('Connection closed')
    });

    Note: The error callback receives the error object as an argument; end and close callbacks receive no arguments.

  6. Avoid mixing Promise chains and Async/Await

    master

    While you can mix Promise chains (.then()) and async/await, it is highly discouraged as it creates complex, bug-prone code. A common bug occurs when an await call is made inside a .then() block without returning the resulting promise, causing the chain to proceed to .finally() or the next .then() before the awaited task is finished.

    Choose one paradigm and stick to it. Using async/await is generally considered more natural for most developers.

    // Recommended: Pure async/await pattern
    async function doSftp() {
      try {
        let sftp = await sftp.connect(conf);
        let d = await sftp.cwd();
        console.log(`remote dir is ${d}`);
        await sftp.fastGet(`${d}/foo.txt`, 'bat.txt');
      } catch (e) {
        console.error(e.message);
      } finally {
        await sftp.end();
      }
    }
  7. Avoid re-using SftpClient objects (especially with Windows servers)

    master

    When connecting to Windows-based SFTP servers, an ECONNRESET error is often raised after a delay when calling end(). To prevent uncaught exceptions, ssh2-sftp-client must keep error handlers attached to the object.

    If you re-use the same SftpClient object for multiple connections (e.g., connect() -> end() -> connect()), these handlers accumulate. After 11 handlers are added, Node.js will trigger a memory leak warning.

    Best Practice: Always generate a new SftpClient object for each new connection. You can perform multiple operations (uploads/downloads) within a single connection, but do not call connect() again on the same instance after calling end().

  8. Connect through a SOCKS 5 Proxy

    master

    To connect via a SOCKS 5 proxy, you must create the connection using a SOCKS client (like the socks package) and then pass the resulting socket to the connect method via the sock option. Note that the SOCKS client connection must be ingested by ssh2-sftp-client immediately to avoid timeouts.

    import { SocksClient } from 'socks';
    import SFTPClient from 'ssh2-sftp-client';
    
    const host = 'my-sftp-server.net';
    const port = 22;
    
    // connect to SOCKS 5 proxy
    const { socket } = await SocksClient.createConnection({
      proxy: {
        host: 'my.proxy',
        port: 1080,
        type: 5,
      },
      command: 'connect',
      destination: { host, port }
    });
    
    const client = new SFTPClient();
    client.connect({
      host,
      sock: socket, // pass the socket to proxy here
      // other config options
    });
  9. Install ssh2-sftp-client

    master

    To use this package, install it via your preferred package manager. This package provides the SftpClient class, which is a promise-based decorator around the ssh2 module.

    Requirements:

    • Node.js version 20.x or higher (Node versions prior to v20.x are not supported).
    # Example installation command
    npm install ssh2-sftp-client
  10. Specifying remote file paths

    master

    The client uses 'nix' style paths (using / as a separator) regardless of whether the remote server is Windows or Linux.

    Path Formats

    • Absolute Paths: e.g., /absolute/path/to/file (Recommended for efficiency).
    • Relative Paths: Use ./ for the current directory or ../ for the parent directory. Note that using relative paths incurs a small performance penalty as the client must query the server to resolve the absolute path.
    • Windows Paths: If the server is Windows, use /C:/Users/name instead of C:\Users\name.

    Important Constraints

    • No Shell Expansion: The tilde (~) and environment variables like $HOME are not supported.
    • Full Paths Required: When performing operations like put, you must include the target filename in the destination path. The module will not automatically append the local filename to a directory path.

    Incorrect:

    client.put('/local/file.txt', '/remote/dir'); // Will fail or not behave as expected

    Correct:

    client.put('/local/file.txt', '/remote/dir/file.txt');
    // OR rename during upload:
    client.put('/local/file.txt', '/remote/dir/new-name.txt');
  11. Authenticate without a password using SSH keys

    master

    You can avoid using passwords by setting up SSH keys and using one of the following two methods in your connection configuration:

    1. Using an SSH Agent: Pass the SSH_AUTH_SOCK environment variable to the agent option.
    2. Using a Private Key directly: Pass the contents of your private key file to the privateKey option using fs.readFileSync().
    // Option 1: Using SSH Agent
    let sftp = new Client();
    sftp.connect({
      host: 'YOUR-HOST',
      port: 'YOUR-PORT',
      username: 'YOUR-USERNAME',
      agent: process.env.SSH_AUTH_SOCK
    }).then(() => {
      sftp.fastPut(/* ... */)
    });
    
    // Option 2: Using Private Key directly
    let sftp = new Client();
    sftp.connect({
      host: 'YOUR-HOST',
      port: 'YOUR-PORT',
      username: 'YOUR-USERNAME',
      privateKey: fs.readFileSync('/path/to/ssh/key')
    }).then(() => {
      sftp.fastPut(/* ... */)
    });
  12. Limit upload/download speed using streams

    master

    To limit bandwidth usage, you can pipe the data through a throttling stream (e.g., using the throttle package).

    Important: When using a stream for the dst argument in get(), you must set the { autoClose: false } option to avoid a _get Permission Denied error in ssh2-streams caused by an extra _read() call on a closed stream.

    const Throttle = require('throttle');
    const progress = require('progress-stream');
    
    // limit download speed
    const throttleStream = new Throttle(config.throttle);
    
    // download progress stream
    const progressStream = progress({
      length: fileSize,
      time: 500,
    });
    progressStream.on('progress', (progress) => {
      console.log(progress.percentage.toFixed(2));
    });
    
    const outStream = createWriteStream(localPath);
    
    // pipe streams together
    throttleStream.pipe(progressStream).pipe(outStream);
    
    try {
      // set autoClose to false
      await client.get(remotePath, throttleStream, { autoClose: false });
    } catch (e) {
      console.log('sftp error', e);
    } finally {
      await client.end();
    }