DOCSEARCHbyAlgolia
Submit Docs

nbgitpuller

repository·main·Indexed 42 days ago

https://github.com/jupyterhub/nbgitpuller

A tool for educators and developers to distribute git-based content to users via simple URLs. It automates cloning and pulling repositories directly into Jupyter environments, including JupyterHub, classic Notebook Server, and Jupyter Server, handling git operations and automatic merging behind the scenes.

Tokens
5.5K
Snippets
15
Records
33
Agent score
52%

What's inside nbgitpuller

  1. What is nbgitpuller?

    main
    nbgitpuller is a tool designed to distribute content from a git repository to students or users via a simple link. When a user clicks the link, the content is automatically pulled into a live Jupyter session. It features automatic merging to ensure users are not exposed to direct git commands. While primarily used with JupyterHub, it can also function on local computers.

  2. How `nbgitpuller` links work

    main

    An nbgitpuller link is a URL that, when clicked by a user, automates the process of synchronizing a Git repository with a JupyterHub instance.

    Key behaviors:

    • First click: Creates a local copy of the specified Git repository for the student/user.
    • Subsequent clicks: Fetches the latest version of the remote repository and automatically merges it with the user's local copy using predefined rules to prevent merge conflicts or data loss.

    Link components: An nbgitpuller link typically contains:

    1. The JupyterHub location.
    2. The Git repository URL.
    3. (Optional) A specific file or directory to open automatically after the pull (note: the entire repository is still cloned).
    4. (Optional) The desired UI to open (e.g., classic notebook, JupyterLab, RStudio, or Linux Desktop), depending on what is installed in your JupyterHub.

  3. Understand nbgitpuller automatic merging behavior

    main

    nbgitpuller is designed to minimize manual git interaction for end users (students) by applying opinionated merging rules when their local environment diverges from the instructor's remote repository.

    Key behaviors include:

    • Non-conflicting changes: Instructor changes to files the student hasn't touched, or changes to different lines in the same file, are automatically merged into the student's local copy.
    • Conflicting lines: If the student and instructor change the same lines in the same file, nbgitpuller always keeps the student's changes to prevent data loss.
    • File deletions: If a student deletes a file locally that still exists in the remote repo, clicking the nbgitpuller link again will pull the file back into the student's directory (useful for 'starting over').
    • Filename collisions: If a student creates a file that later appears in the instructor's remote repository with the same name, nbgitpuller will rename the student's file to Untitled141_<timestamp>.ipynb and pull the instructor's version to preserve both.

  4. When to use or avoid nbgitpuller

    main

    Use nbgitpuller when:

    • You want to distribute notebooks or markdown files easily without requiring users to use git commands.
    • You have a separate workflow for collecting user contributions (since nbgitpuller only performs "pull" operations).

    Do NOT use nbgitpuller when:

    • Users need to push: If users need to push directly to a git repository, they should use git directly. Using nbgitpuller for pulling while users use git for pushing can lead to unexpected behavior.
    • Manual Git operations: If users need to perform manual git operations locally, avoid nbgitpuller. Mixing manual git commands with nbgitpuller's automatic operations will cause conflicts and unexpected results.

  5. Activate a specific tab in the link generator

    main

    When a user lands on the link generator page, you can force the form to start on a specific authentication tab using the tab query parameter.

    Supported values:

    • ?tab=binder: Activates the Binder tab.
    • ?tab=canvas: Activates the Launch from Canvas tab.

  6. Generate nbgitpuller links

    main

    You can create custom nbgitpuller links using the web-based link generator. These links allow users to launch a JupyterHub instance (or Binder) with a specific Git repository and branch pre-configured.

    To generate a link, you need to provide:

    • JupyterHub URL: The URL of the JupyterHub instance where nbgitpuller is installed.
    • Git Repository URL: The URL of the repository to pull (supports git, https, or SSH formats).
    • Branch: The specific branch to pull from (defaults to main).
    • File to open (Optional): A specific file or directory within the repo to open upon launch.
    • Application to Open: Choose between Classic Jupyter Notebook, JupyterLab, RStudio, Shiny, RetroLab, or a Custom URL.
    • Target Path (Optional): The directory name to clone the repository into.
    • Named Server to open (Optional): A specific JupyterHub NamedServer instance.

    Alternatively, you can use the nbgitpuller-link-generator browser extension available for Firefox and Chrome.

  7. Pre-populate the nbgitpuller link generator

    main

    You can pre-fill the link generator form by appending query parameters to the generator URL. This is useful for creating shared links that are already partially configured for your users.

    Supported query parameters:

    • hub: The URL of the JupyterHub.
    • repo: The URL of the Git repository.
    • branch: The branch name to pull from.

    Example: To pre-populate the JupyterHub URL for UC Berkeley DataHub, use: https://nbgitpuller.readthedocs.io/link.html?hub=https://datahub.berkeley.edu

    https://nbgitpuller.readthedocs.io/link.html?hub=https://datahub.berkeley.edu

  8. Set up a local development environment for nbgitpuller

    main

    Since nbgitpuller works with both the classic Notebook Server and the newer Jupyter Server, you can develop it locally without a JupyterHub instance. Follow these steps to prepare your environment:

    1. Clone the repository: Fork and clone the nbgitpuller repo.
    2. Create a virtual environment: Use python3 -m venv venv and activate it.
    3. Install Python dependencies:
      • Install development dependencies: pip install -r dev-requirements.txt
      • Install nbgitpuller in editable mode: pip install -e .
    4. Install JavaScript dependencies: Run npm install to install packages from package.json.
    5. Build assets: Run npm run webpack to create the JS and CSS bundles.

    Note: If you modify Python code, you must restart the notebook server process. Changes to JavaScript or CSS do not require a server restart.

    # Setup steps summary
python3 -m venv venv
source venv/bin/activate
pip install -r dev-requirements.txt
pip install -e .
npm install
npm run webpack

  9. Manage repository size to prevent timeouts

    main

    Large git repositories increase the risk of timeouts, intermittent failures, and inconsistent states (e.g., contents fetched but not checked out).

    To ensure a smooth experience:

    • Aim for a repository size under 100MB.
    • Keep the size under 1GB if possible.
    • Avoid storing large datasets directly in the git repo. Instead, distribute datasets via other methods, use data subsets, or use compression.

  10. How to use nbgitpuller to distribute content

    main

    Use nbgitpuller to distribute Jupyter Notebooks or other files to users without requiring them to understand git. The workflow follows these steps:

    1. Installation: Ensure nbgitpuller is installed on the user's environment (e.g., installed for all users of a JupyterHub).
    2. Link Generation: Create an nbgitpuller link that points to the specific content you want to distribute (such as a notebook in a GitHub repository).
    3. User Access: Provide the link to your users. When they click it, nbgitpuller automatically pulls the content into their live Jupyter session and onto their file system.

    Note: nbgitpuller is a "pull" tool only. It does not "push" changes back to a repository. You must have an alternative method for collecting content/changes from your users.

  11. Exclude `.ipynb_checkpoints` and temporary files from your git repo

    main

    Committing Jupyter's autosave directory (.ipynb_checkpoints) or other temporary files (like .DS_Store or __pycache__) can cause hard-to-debug conflicts between the user's local environment and the JupyterHub server.

    To prevent this, add these patterns to your .gitignore file. You can use a standard Python .gitignore to cover most common temporary files.

  12. Build nbgitpuller documentation

    main

    Documentation is maintained using sphinx and is located in the docs/ directory. It uses a mix of reStructuredText and MyST Markdown.

    1. Install doc requirements: pip install -r docs/doc-requirements.txt
    2. Build HTML: Navigate to the docs folder and run make html.
    3. Preview: Open _build/html/index.html in your browser.

    To quickly open the preview from the docs folder:

    • MacOS: open _build/html/index.html
    • Linux: xdg-open _build/html/index.html
    # Build documentation
pip install -r docs/doc-requirements.txt
cd docs
make html

# Preview (MacOS)
open _build/html/index.html