nbgrader

nbgrader provides a suite of command line tools categorized by user role and purpose.

• Basic commands: Core utilities for running nbgrader and managing configuration.
• Instructor commands: Tools for creating assignments, validating them, autograding, and managing solutions/feedback.
• Database commands: Utilities to manage the nbgrader database, including adding, importing, removing, and listing students and assignments.
• Shared server commands (Instructor): Commands for releasing assignments/feedback and collecting submissions in a shared environment (e.g., JupyterHub).
• Shared server commands (Student): Commands for students to list, fetch, validate, and submit assignments in a shared environment.

nbgrader is a system designed for assigning and grading Jupyter notebooks. It provides several interfaces for different user roles:

• Instructors: Can use a toolbar extension within Jupyter notebooks to guide assignment and grading tasks, or the 'formgrader' extension to manage the full lifecycle of an assignment (generating student versions, releasing, collecting, autograding, and manual grading).
• Students: Can use the assignment list extension to view, fetch, submit, and validate their assignments.
• Automated/Power Users: Can use command line tools to efficiently generate, assign, release, collect, and grade notebooks.

nbgrader provides three primary interfaces for different user roles:

1. Student Assignment List Extension: A Jupyter extension for students to view, fetch, submit, and validate assignments, as well as review feedback.
2. Instructor Toolbar Extension: A Jupyter extension that guides instructors through assignment and grading tasks directly within the notebook interface.
3. Instructor Formgrader Extension: A Jupyter extension that provides access to core nbgrader functionality (generating, releasing, collecting, autograding, and manual grading) via a form-based interface.

Autograding in nbgrader functions by executing the notebook, similar to the "Restart and run all cells" command.

Mechanism:

1. The notebook is run from top to bottom.
2. nbgrader looks for cells marked as autograder tests that produced an error output.
3. An error is detected if there is any text on the standard error stream (stderr) for that cell.
4. If an error is found, the point value associated with that specific test cell is subtracted from the total.

Important Notes:

• Kernel Support: The Jupyter kernel must write error messages to the stderr stream for autograding to function. Some language kernels may not support this.
• Partial Credit: While not natively implemented as a complex system, you can simulate partial credit by dividing a single task into multiple autograded test cells.

nbgrader stores its configuration and state within the Jupyter notebook source itself. All nbgrader-specific information is stored at the cell level within the cell's metadata field, under a dictionary keyed as nbgrader. This structure allows nbgrader to track grading requirements, solutions, and integrity via commands like nbgrader generate_assignment, nbgrader validate, and nbgrader autograde.

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "nbgrader": {
                    "...
                }
            },
            "source": ["an example cell\n"]
        }
    ]
}

nbgrader allows for limited customization of its behavior through predefined plugin methods or by implementing your own custom plugins.

Supported plugin types include:

• late-plugin: For logic that needs to run late in the process.
• export-plugin: For customizing how notebooks are exported.
• zipcollect-plugin: For customizing how files are collected into zip archives.

If the existing plugin interfaces do not meet your requirements, you should open an issue on GitHub to request new functionality.

nbgrader uses a database to store assignment information.

• SQLite (Default): nbgrader will automatically create a SQLite database at {course_directory}/gradebook.db. You do not need to create this manually.
• MySQL or PostgreSQL: Since nbgrader uses SQLAlchemy, you can use these backends. However, unlike SQLite, you must create the database ahead of time before running nbgrader commands.

The validate extension provides a student-facing interface within the Jupyter notebook view.

Functionality:

• It executes the student's notebook from top to bottom.
• It reports any errors encountered during execution.
• It is functionally equivalent to "Restart and run all", but it does not stop execution when an error is encountered.
• Limitation: Validation only has access to the actual notebook file the student possesses; it cannot access or execute hidden tests. If an instructor wants a test to be visible to students, it must be part of the notebook file.

nbgrader exchange functions can be invoked through three primary interfaces:

1. Command Line Interface (CLI): Direct execution of commands, for example: nbgrader release_assignment assignment1.
2. formgrader server extension: This extension generally invokes methods defined in the nbgrader/apps/{foo}app.py modules.
3. assignment_list server extension: This extension generally calls the exchange methods directly.