docstring_parser

repository·master·Indexed 42 days ago

https://github.com/rr-/docstring_parser

A Python library for parsing docstrings in reST, Google, Numpydoc, and Epydoc formats into structured objects. Version 0.18.0 provides programmatic access to descriptions, parameters, return values, and raised exceptions via the parse() and parse_from_object() functions.

Tokens
5.8K
Snippets
20
Records
32
Agent score
51%

What's inside docstring_parser

  1. Install docstring_parser via pip

    master

    You can install the package directly using pip:

    pip install docstring_parser

    If you are using a virtual environment, follow these steps:

    python -m venv venv # create environment
    source venv/bin/activate # activate environment
    python -m pip install docstring_parser
  2. Install docstring_parser via conda

    master

    To install using conda, use the conda-forge channel. You can either install it into your current environment or create a new one:

    Install into current environment:

    conda install -c conda-forge docstring_parser

    Create a new environment with the package:

    conda create -n my-new-environment -c conda-forge docstring_parser
  3. Numpydoc Section Types

    master

    The NumpydocParser uses specialized Section subclasses to handle different parts of the docstring. Common ones include:

    • ParamSection: Handles Parameters, Args, Attributes, etc. Extracts arg_name, type_name, is_optional, and default.
    • RaisesSection: Handles Raises or Warns. Extracts the exception type and description.
    • ReturnsSection / YieldsSection: Handles Returns or Yields. Extracts return_name, type_name, and whether it is a generator.
    • ExamplesSection: Handles Examples. Parses code snippets (lines starting with >>>) and their accompanying descriptions.
    • DeprecationSection: Handles Sphinx-style .. deprecated:: blocks, extracting the version and description.
  4. Configure section cardinality with SectionType

    master

    The SectionType enum tells the parser how to interpret the content following a section title. This determines whether the parser expects a single block of text or multiple individual entries (like parameters).

    Members:

    • SINGULAR (0): For sections containing a single block of text (e.g., Examples).
    • MULTIPLE (1): For sections containing multiple distinct entries (e.g., Args, Parameters, Raises, Attributes).
    • SINGULAR_OR_MULTIPLE (2): For sections that might contain either a single entry or multiple entries (e.g., Returns, Yields).
    from docstring_parser import SectionType
  5. Epydoc metadata field mapping

    master

    When parsing Epydoc docstrings, the following metadata types are extracted into the Docstring.meta list as specific objects:

    PrefixObject TypeDescription
    @param, @keyword, @typeDocstringParamRepresents function parameters, including arg_name, type_name, is_optional, and default values.
    @ivar, @cvar, @varDocstringParamRepresents class/instance attributes.
    @raiseDocstringRaisesRepresents exceptions raised, including the type_name and description.
    @return, @rtype, @yield, @ytypeDocstringReturnsRepresents return values. If @yield or @ytype is used, is_generator is set to True.
    @<any_other>DocstringMetaGeneric metadata for any other @ prefixed fields.
  6. How Numpydoc parameter defaults are detected

    master

    The Numpydoc parser attempts to extract default values for parameters in two ways:

    1. In the type declaration: It looks for patterns like default True, default=True, or default: True within the type string.
      • Example: copy : bool, default True
    2. In the description: If no default is found in the type declaration, it searches the parameter's description for natural language patterns like Defaults to 'value' or is 'value'.

    When a default is detected, the parameter is automatically marked as is_optional=True.

  7. Parse Python docstrings with parse()

    master

    Use the parse function from docstring_parser to convert docstring text into a structured object. The parser supports ReST, Google, Numpydoc-style, and Epydoc formats.

    The resulting object provides access to:

    • long_description: The multi-line description of the docstring.
    • params: A list of parameter objects (access arg_name for the name).
    • raises: A list of exception objects (access type_name for the exception type).

    Example usage:

    from docstring_parser import parse
    
    docstring = parse(
        '''
        Short description
    
        Long description spanning multiple lines
        - First line
        - Second line
        - Third line
    
        :param name: description 1
        :param int priority: description 2
        :param str sender: description 3
        :raises ValueError: if name is invalid
        '''
    )
    
    print(docstring.long_description)
    print(docstring.params[1].arg_name)  # 'priority'
    print(docstring.raises[0].type_name) # 'ValueError'
    from docstring_parser import parse
    
    docstring = parse(
        '''
        Short description
    
        Long description spanning multiple lines
        - First line
        - Second line
        - Third line
    
        :param name: description 1
        :param int priority: description 2
        :param str sender: description 3
        :raises ValueError: if name is invalid
        '''
    )
    
    print(docstring.long_description)
    print(docstring.params[1].arg_name)
    print(docstring.raises[0].type_name)
  8. Render a parsed Docstring back to text

    master

    The compose function allows you to take a Docstring object (the output of a parser) and convert it back into a formatted string. This is useful for docstring manipulation or re-formatting.

    You can specify a rendering_style (e.g., RenderingStyle.COMPACT) and an indent string.

    from docstring_parser import compose, parse, RenderingStyle
    
    docstring_text = """
    Summary.
    
    Parameters
    ----------
    x : int
        Description.
    """
    parsed = parse(docstring_text)
    new_text = compose(parsed, rendering_style=RenderingStyle.COMPACT)
  9. Parse Numpydoc-style docstrings with NumpydocParser

    master

    To parse a docstring written in the Numpydoc format, use the NumpydocParser class or the convenience parse function. The parser breaks the docstring into a Docstring object containing a short description, long description, and various metadata sections like parameters, returns, raises, and examples.

    By default, the parser recognizes standard Numpydoc sections such as Parameters, Returns, Raises, Examples, Notes, and See Also. You can extend or override these sections using add_section().

    from docstring_parser import parse
    
    docstring_text = """
    Add a description.
    
    Parameters
    ----------
    arg1 : int
        Description of arg1.
    arg2 : str, optional
        Description of arg2, defaults to 'hello'.
    """
    
    parsed = parse(docstring_text)
    print(parsed.short_description)
    print(parsed.params[0].arg_name)
  10. Get attribute docstrings from a component with get_attr_docs

    master

    The AttributeDocstrings.get_attr_docs method provides a way to manually extract attribute documentation from a class or module using AST analysis.

    Parameters:

    • component: The class or module object to process.

    Returns: A dictionary where keys are attribute names and values are tuples containing (description, type, default):

    • description (str): The literal string following the attribute.
    • type (Optional[str]): The type annotation string (e.g., 'int').
    • default (Optional[str]): The string representation of the default value (e.g., '30').
    from docstring_parser.attrdoc import AttributeDocstrings
    
    class Example:
        """Class doc."""
        port: int = 8080
        """The network port."""
    
    extractor = AttributeDocstrings()
    docs = extractor.get_attr_docs(Example)
    # docs will be: {'port': ('The network port.', 'int', '8080')}
  11. Parse ReST-style docstrings with parse()

    master

    Use the parse function to convert a ReST-style docstring string into a structured Docstring object. The parser handles short descriptions, long descriptions, and metadata fields like :param:, :type:, :returns:, :rtype:, :raises:, and :deprecated:.

    Key features:

    • Automatically cleans indentation using inspect.cleandoc.
    • Supports mapping :type: and :rtype: declarations to their respective parameters or return values.
    • Detects optional types if they end with ? (e.g., int?).
    • Extracts default values from descriptions containing the phrase defaults to <value>.
    • Handles generator detection via :yields: keywords.
    from docstring_parser import parse
    
    docstring_text = """
    Summary line.
    
    Longer description here.
    
    :param x: description of x
    :type x: int
    :returns: description of return
    :rtype: str
    """
    
    parsed = parse(docstring_text)
    print(parsed.short_description)
    print(parsed.long_description)
    for meta in parsed.meta:
        print(meta)
  12. Render a parsed Docstring with compose()

    master

    The compose function takes a Docstring object (produced by parse) and converts it back into a formatted string. You can control the output using the rendering_style and indent parameters.

    Parameters:

    • docstring: The Docstring object to render.
    • rendering_style: An instance of RenderingStyle (e.g., RenderingStyle.COMPACT or RenderingStyle.EXPANDED).
    • indent: A string defining the indentation used for multi-line descriptions (default is " ").
    from docstring_parser.epydoc import parse, compose, RenderingStyle
    
    docstring_text = """
    Summary.
    
    @param x: description
    """
    parsed = parse(docstring_text)
    rendered = compose(parsed, rendering_style=RenderingStyle.EXPANDED)
    print(rendered)