nbformat

repository·main·Indexed 43 days ago

https://github.com/jupyter/nbformat

Reference implementation of the Jupyter Notebook format, providing Python APIs for manipulating notebooks and JSON Schemata for versions 3 and above. It includes tools for reading, writing, and validating notebooks using NotebookNode objects, as well as machinery for managing notebook signatures and trust via NotebookNotary.

Tokens
12.1K
Snippets
49
Records
63
Agent score
55%

What's inside nbformat

  1. Understand the Jupyter Notebook file format structure

    main

    A Jupyter notebook is a JSON-formatted dictionary. At the highest level, it contains the following keys:

    • metadata (dict): Contains information about the kernel and language.
    • nbformat (int): The major version of the notebook format.
    • nbformat_minor (int): The minor version of the notebook format.
    • cells (list): A list of cell dictionaries containing the notebook content.

    Important Note on Multi-line Strings: Fields like source or text may be written to disk as a single string or a list of strings. When working with notebook files directly, you must handle both types. However, programmatic APIs (like the Python or JavaScript nbformat libraries) will always re-join these into a single multi-line string when reading them into memory.

    {
        "metadata": {
            "kernel_info": {
                "name": "the name of the kernel"
            },
            "language_info": {
                "name": "the programming language of the kernel",
                "version": "the version of the language",
                "codemirror_mode": "The name of the codemirror mode to use [optional]"
            }
        },
        "nbformat": 4,
        "nbformat_minor": 0,
        "cells": []
    }
  2. Use Code cells for executable content

    main

    Code cells are the primary content of notebooks. They contain source code and a list of outputs generated by execution.

    Key fields:

    • cell_type: Must be "code".
    • execution_count: An integer or null.
    • source: The code (string or list of strings).
    • outputs: A list of output dictionaries.
    • metadata: Can include "collapsed": true/false or "scrolled": true/false/"auto".
    {
        "cell_type": "code",
        "execution_count": 1,
        "metadata": {
            "collapsed": true,
            "scrolled": false
        },
        "source": "[some multi-line code]",
        "outputs": [
            {
                "output_type": "stream",
                "name": "stdout",
                "text": "[multiline stream text]"
            }
        ]
    }
  3. Understand the Jupyter Notebook Format

    main
    Jupyter notebook files are JSON documents that store text, source code, rich media output, and metadata. The document is structured into segments called cells, where each cell contains a specific piece of the notebook's content.
  4. Manage notebook signatures and trust

    main

    The nbformat.sign module provides machinery for recording which notebooks are trusted. This is used by the notebook web application to safely handle dynamic output.

    NotebookNotary

    Use NotebookNotary to sign, unsign, and check the signatures of notebooks. It provides methods like sign(), unsign(), check_signature(), mark_cells(), and check_cells().

    Custom Signature Storage

    Signatures are managed via a SignatureStore. By default, NotebookNotary uses SQLiteSignatureStore (if SQLite bindings are available) or MemorySignatureStore.

    To implement a custom storage backend, subclass SignatureStore and override its methods, then configure NotebookNotary.store_factory to use your implementation.

    # Conceptual usage of NotebookNotary
    from nbformat.sign import NotebookNotary
    
    notary = NotebookNotary()
    # notary.sign(nb) ...
  5. Use Raw NBConvert cells

    main

    Raw cells contain content that should be included unmodified in nbconvert output (e.g., LaTeX or reStructuredText). The notebook authoring environment does not render these cells.

    To control which nbconvert formats include the cell, use the format key in the cell's metadata. If format is defined, the cell is only included when outputting to that specific mime-type. If undefined, the cell is included in all formats.

    {
        "cell_type": "raw",
        "metadata": {
            "format": "mime/type"
        },
        "source": ["some nbformat output text"]
    }
  6. Work with NotebookNode objects

    main

    The primary data structure in nbformat is the NotebookNode. These objects behave like dictionaries but allow for attribute access (e.g., nb.cells instead of nb['cells']). The structure of a NotebookNode follows the standard notebook format specification.

    You can convert a standard dictionary into a NotebookNode using nbformat.from_dict(dict).

    # Example of attribute access on a NotebookNode
    print(nb.cells)
  7. Use Markdown cells for body text

    main

    Markdown cells are used for non-code text and support GitHub-flavored markdown. They follow this structure:

    {
        "cell_type": "markdown",
        "metadata": {},
        "source": "[multi-line *markdown*]"
    }

    Attachments: Markdown cells can include inline images via the attachments field in the cell's metadata. The attachments dictionary contains mime-bundles keyed by filename.

    {
        "cell_type": "markdown",
        "metadata": {},
        "source": ["Here is an *inline* image ![inline image](attachment:test.png)"],
        "attachments": {"test.png": {"image/png": "base64-encoded-png-data"}}
    }
  8. Supported Markdown flavors in Jupyter Notebooks

    main

    The Jupyter Notebook format supports Markdown within text cells. While there is no strict specification for the exact flavor, most Jupyter Notebook interfaces utilize the marked.js JavaScript library for rendering. This means you can expect behavior consistent with:

    • CommonMark
    • GitHub Flavored Markdown (GFM)

    Note that as the marked.js specification evolves, the rendering behavior in various notebook interfaces may change accordingly.

  9. How notebook trust is determined

    main

    A notebook is considered 'trusted' if its signature is found in the local SignatureStore.

    Individual code cells can also be trusted independently. A cell is considered trusted if:

    1. It is explicitly marked as trusted in its metadata (cell.metadata.trusted == True).
    2. OR, it has no potentially unsafe rich outputs (e.g., execute_result or display_data containing non-whitelisted keys). Simple print statements or cells with no output are always trusted.

    NotebookNotary.check_cells(nb) returns True if all code cells in the notebook meet these criteria.

  10. Configure the JSON schema validator

    main

    By default, nbformat uses fastjsonschema for validation. If you need to use jsonschema instead, set the NBFORMAT_VALIDATOR environment variable to jsonschema before running your application.

    export NBFORMAT_VALIDATOR=jsonschema