DOCSEARCHbyAlgolia
Submit Docs

NYU Research Technology Services Documentation

website·Indexed 19 days ago

https://services.rt.nyu.edu/

Documentation for NYU's research computing resources, including the Torch HPC cluster, the Secure Research Data Environment (SRDE), cloud computing via GCP, and Generative AI tools. It provides guides on SLURM job scheduling, PyTorch Distributed Data Parallel (DDP), shell scripting, secure remote connections, and the use of REDCap for research data capture.

Tokens
59.6K
Snippets
318
Records
510
Agent score
99%

What's inside NYU Research Technology Services

  1. Introduction to Apptainer for HPC

    Apptainer (formerly Singularity) is a container platform designed for shared HPC environments. Unlike Docker, it does not require a privileged daemon, allowing users to run software in isolated environments without administrator privileges on the host system. It is the supported container platform for Torch and can utilize images distributed through Docker registries.

  2. Overview of the NIH STRIDES Initiative for NYU Researchers

    The NIH Science and Technology Research Infrastructure for Discovery, Experimentation, and Sustainability (STRIDES) Initiative provides NIH-funded NYU researchers with access to cost-effective cloud computing infrastructure (GCP/AWS) to streamline data use and collaboration. Benefits of participation include pre-negotiated favorable pricing for cloud services, cloud training, professional service engagements, and guidance on data storage, governance, and controlled access. NYU researchers with an active NIH award are eligible to participate.

  3. Overview of NFSv4 Access Control Lists (ACLs) on Torch

    Torch uses NFSv4 ACLs for fine-grained file and directory access control, which are more flexible than POSIX ACLs. Users are strongly discouraged from using chmod 777 to share files, as it can lead to accidental or malicious data loss.

  4. Overview of Torch Data-Transfer Nodes (DTN)

    The Torch Data Transfer Nodes (DTN) are specialized nodes optimized for moving data between cluster filesystems (such as /home, /scratch, /archive, and the HPC Research Project Space) and external endpoints like laptops or desktops. They feature 100-Gb/s Ethernet connections to the High Speed Research Network (HSRN) and are connected to the HDR Infiniband fabric.

  5. Overview of ML and AI workflows on NYU HPC

    The NYU HPC resources support Machine Learning (ML) and Artificial Intelligence (AI) workflows. Supported frameworks and tasks include:

    • Deep Learning Frameworks: PyTorch and TensorFlow, ranging from single-GPU setups to sophisticated multi-GPU configurations.
    • LLM Inference: Basic tasks using the Hugging Face transformers library and high-throughput serving via vLLM (which provides an OpenAI-compatible API).
    • LLM Serving: Use of the vllm command-line tool for serving models and interactive chat sessions.
    • LLM Fine-tuning: Practical guidance on fine-tuning models (e.g., the Gemma model) to improve instruction-following and response quality.

  6. Overview of Apptainer for HPC

    Apptainer (formerly the FOSS version of Singularity) is a container-based Linux kernel workspace designed for HPC environments. It allows users to run pre-built programs in isolated environments without requiring root access on the host cluster, making it a secure alternative to Docker in academic and research settings. Apptainer is fully compatible with Docker images.

  7. Overview of Portkey LLM Gateway capabilities

    Portkey is an enterprise-grade LLM gateway that provides a centralized layer for managing Large Language Model interactions. Its primary capabilities include:

    • Request Routing: Configure fallbacks and load balancing for LLM requests.
    • Observability: Access usage statistics and logs via a control pane or API.
    • Prompt Management: Use a prompt playground to compare responses across multiple LLMs, collaborate with teams, and export prompts.
    • Guardrails: Define internal guardrails or integrate with third-party guardrail services.
    • Agentic Workflows: Integration support for frameworks such as LangChain and LlamaIndex.
    • Security & Governance: Manage API keys for workspaces with specific budget and rate limits.

  8. Overview of the NYU Secure Research Data Environment (SRDE)

    The NYU Secure Research Data Environment (SRDE) is a centralized secure computing platform for research projects that require storage, sharing, and analysis of high-risk datasets. It is designed to meet security controls outlined in NIST 800-171 to safeguard Controlled Unclassified Information (CUI) and helps researchers comply with security requirements of research grants and Data Use Agreements.

  9. Overview of NYU Dataproc Hadoop Cluster

    NYU Dataproc is a Google-managed, cloud-based Hadoop distribution. It provides a distributed framework for storing and processing big data using three core components: HDFS (Hadoop Distributed File System), MapReduce, and YARN (Yet Another Resource Negotiator). The NYU configuration is designed to be cloud-agnostic, utilizing HDFS and non-proprietary HBase components, while integrating with Google Cloud features like autoscaling, Google Cloud Storage (GCS), and BigQuery.

  11. Overview of Research Project Space (RPS)

    Research Project Space (RPS) volumes provide persistent working space for sharing data and code among project or lab members. They are built on the VAST parallel file system and mounted on cluster Compute Nodes, allowing them to be accessed by running jobs. Unlike HPC Scratch, RPS directories are backed up and have no old file purging policy. Storage and inode usage in RPS do not count toward quotas in Home, Scratch, or Archive file systems.

  12. Overview of Environment Modules on NYU HPC

    Environment modules are used on HPC systems to manage software incompatibilities, versioning, and dependencies. A module is a self-contained description of a software package that contains the settings required to run it and its dependencies. Loading a module typically modifies shell environment variables (such as $PATH and library paths) to make the software accessible without moving large amounts of data. The system supports both TCL modules and Lmod, both of which use the module command.