Open OnDemand Documentation

repository·master·Indexed 19 days ago

https://github.com/osc/ondemand

A browser-based portal providing remote access to High-Performance Computing (HPC) systems without requiring command-line or SSH clients. Includes documentation for the Open OnDemand Dashboard, Job Composer, OOD Shell, and core components such as mod_ood_proxy, nginx_stage, ood_auth_map, and the ood-portal-generator.

Tokens
15.8K
Snippets
45
Records
81
Agent score
67%

What's inside Open OnDemand

  1. What is Open OnDemand?

    master

    Open OnDemand is an open-source, web-based client portal designed for High-Performance Computing (HPC) centers. It aims to bridge the accessibility gap in HPC by providing advanced web and graphical interfaces, reducing the need for users to have advanced Linux knowledge or to manually configure client software like SSH or VNC.

    Key capabilities include:

    • Per-user NGINX (PUN) architecture: Provides secure, federated authentication (e.g., via CILogon) and uses Unix domain sockets for secure server-side communication between a web proxy and individual user instances.
    • File Management: Includes a web-based file browser and file editor.
    • Terminal Access: Provides a web-based terminal interface.
    • Accessibility Apps: A suite of applications including a Dashboard, Job Constructor, Job Status, System Status, VDI, and iHPC apps.
    • App Development: Uses the Rails-based AweSim AppKit, which enables the creation, sharing, and publishing of both workflow and interactive applications.
  2. Overview of nginx_stage

    master

    The nginx_stage component is responsible for managing Per-User NGINX (PUN) processes within Open OnDemand. Its primary responsibilities include:

    • Process Management: Staging and launching PUN processes for individual users, as well as listing and stopping running PUNs.
    • Configuration Management: Staging, listing, and cleaning up web application NGINX configuration files.
  3. Overview of mod_ood_proxy

    master
    mod_ood_proxy is an Apache HTTP Server module that implements the Open OnDemand proxy API. It allows Apache to act as a proxy for Open OnDemand services, facilitating communication between the web server and the backend components of the Open OnDemand ecosystem.
  4. What is ood_auth_map and when to use it

    master

    The ood_auth_map library provides scripts designed to map an authenticated username (typically provided by an Apache proxy via the REMOTE_USER environment variable) to a local system username.

    This is a critical component in Open OnDemand deployments where an Apache proxy needs to route requests to specific backend NGINX processes. Each backend NGINX process listens on a unique Unix domain socket and is dedicated to a specific local system user. The mapping ensures the proxy can correctly identify which socket to use for a given authenticated user.

  5. Overview of OOD Ruby Gems

    master

    OOD relies on several key Ruby gems:

    • ood_core: The primary backend engine. It provides Scheduler Adapters, batch_connect templates for the three types of OOD apps, ACL functionality, cluster interactivity, and job interaction.
    • ood_appkit: Provides an interface for scientific apps, including a dataroot for writing data and common helper objects/assets.
    • ood_support: Provides an interface to interact with the local OS installed on the HPC center's web node.
    • ood_packaging: Primarily used by the internal OOD team for packaging and distribution.
  6. How views use ERB and model data

    master

    Views in the Project Manager are typically .html.erb (Embedded Ruby) files. They use data provided by the controller (stored in instance variables like @project or @workflow) to render the UI.

    View Patterns

    • Passing Data to JavaScript: Hidden input fields are often used to pass IDs and URL paths from the server-side Ruby view to client-side JavaScript.
    • Embedded Ruby (ERB): You can execute Ruby code within the HTML to perform logic, such as conditionally applying CSS classes.
    • Rails Helpers: Standard Rails helpers like project_workflow_path or select_tag are available to generate URLs and form elements.
    # apps/dashboard/app/views/workflows/show.html.erb
    
    <%= javascript_include_tag 'workflows', nonce: true, defer: true %>
    
    <input type="hidden" id="project-id" value="<%= @project.id %>">
    <input type="hidden" id="workflow-id" value="<%= @workflow.id %>">
    <input type="hidden" id="base-workflow-url" value="<%= project_workflow_path(@project.id, @workflow.id) %>">
    <input type="hidden" id="base-launcher-url" value="<%= project_launchers_path(@project.id) %>">
    
    <div id="workflows_app">
      <div class="toolbar" aria-label="toolbar">
        <% hidden_class = @workflow.editable? ? '' : 'd-none' %>
        <%= select_tag "select_launcher", options_from_collection_for_select(@launchers, :id, :title), include_blank: false, class: "form-control w-25 #{hidden_class}" %>
        <button id="btn-add" class="<%= hidden_class %>">Add Launcher</button>
  7. Understanding OOD Scheduler Adapters

    master

    OOD uses Adapters to interact with different HPC schedulers. These adapters reside in ood_core.

    Commonly used or notable adapters include:

    • k8s: For using Kubernetes as a scheduler.
    • LinuxHost: Used to mimic a scheduler or resource manager, typically for remote desktops or IDEs.
    • SystemD: A community-contributed adapter.
  8. How models manage data and logic

    master

    Models are responsible for managing data and providing the core logic that controllers and views interact with. They isolate complexity, allowing controllers and views to remain simple.

    Model Responsibilities

    • Data Persistence: In Open OnDemand, models often manage files (like YAML manifests) rather than database rows.
    • Logic Encapsulation: Models provide methods to determine state (e.g., editable?) or perform complex operations (e.g., update or submit).
    • Abstraction: By using model methods, you can change the underlying implementation (e.g., how a file is checked for writability) without changing the controller or view code.
    # apps/dashboard/app/models/workflow.rb
    
      def manifest_file
        Workflow.workflow_dir(@project_dir).join("#{@id}.yml")
      end
    
      def update(attributes, override = false)
        update_attrs(attributes, override)
        return false unless valid?(:update)
    
        save_manifest(:update)
      end
    
      def update_attrs(attributes, override = false)
        [:name, :description, :launcher_ids, :metadata].each do |attribute|
          next unless override || attributes.key?(attribute)
          instance_variable_set("@#{attribute}".to_sym, attributes.fetch(attribute, ''))
        end
      end
    
      def editable?
        manifest_file.writable? || !shared?(manifest_file)
      end
  9. Use Nightly versions for development

    master

    Nightly packages (.rpm and .deb) are built every night from the current commit in the main branch and released to the nightly repository.

    Warning: Stability varies depending on the development cycle. While builds closer to a release candidate are generally more stable, no stability guarantees are provided for nightly builds.

  10. Understand the MVC pattern in the Project Manager

    master

    The Project Manager component follows the Model-View-Controller (MVC) paradigm. It is composed of three main entities: Projects, Launchers, and Workflows. Each entity has its own Model, View, and Controller that interact to manage data and relationships.

    Entity Relationships

    • A User has many Projects.
    • A Project has many Launchers and many Workflows.
    • A Workflow has many Launchers.

    Note: Unlike typical web applications, the Open OnDemand dashboard does not use a database; instead, it manages data and relationships through the filesystem. Relationships are established during routing using Rails resource routing.