cxxheaderparser

repository·main·Indexed 41 days ago

https://github.com/robotpy/cxxheaderparser

A pure Python C++ header parser supporting C++11 and beyond. It provides a visitor-style API for inspecting header structures and facilitating the creation of C++ bindings. It includes a CLI for dumping parsed data in formats like JSON and pprint, and supports Doxygen comment extraction and optional preprocessing via PCPP, GCC, or MSVC.

Tokens
12.2K
Snippets
28
Records
65
Agent score
99%

What's inside cxxheaderparser

  1. Overview of cxxheaderparser

    main

    cxxheaderparser is a pure Python C++ header parser. It uses a mildly naive parsing approach designed to handle many C++ constructs, including modern features from C++11 and beyond.

    Important Note on Macros: By default, cxxheaderparser does not use a C preprocessor. If the C++ headers you are parsing contain macros, you must provide a preprocessor function via the preprocessor attribute in cxxheaderparser.options.ParserOptions.

  2. Understand changes to member pointer types in v2.0

    main

    In version 2.0, the representation of member object pointers and member function pointers has changed.

    • New Representation: Both member object pointers and member function pointers are now represented by the cxxheaderparser.types.MemberPointer class.
    • Legacy Representation (Pre-v2.0): Previously, member function pointers were represented by a cxxheaderparser.types.Pointer containing a cxxheaderparser.types.FunctionType where the classname attribute identified the owning class.

    This is a breaking change to the parsed type dataclasses. Note that FunctionType is still included in TypeId for contexts such as aliases, typedefs, template arguments, and parse_typename. DecoratedType continues to be the set of object/declarator types used by variables, fields, adjusted parameters, and function returns.

  3. How cxxheaderparser works and its limitations

    main

    cxxheaderparser is a pure Python C++ header parser designed to handle many C++ constructs, including modern C++11 and beyond. It is intended to provide enough information for binding generators to wrap C++ code.

    Core Concepts

    • Incomplete Headers: The parser can process incomplete headers and does not require processing #include directives.
    • Two API Styles:
      1. A visitor-style interface for building custom data structures.
      2. A simple visitor that stores all parsed data in a single large data structure.

    Important Limitations

    • No Full AST: It does not produce a full Abstract Syntax Tree (AST). If you require a full AST, use Clang.
    • No Validation: It is not a C++ validator and will not reject all invalid C++ headers. Use a compiler for validation.
    • Preprocessor Requirement: The parser requires a C++ preprocessor for headers containing macros. While support for PCPP, GCC, and MSVC is implemented in the cxxheaderparser.preprocessor module, it is not enabled by default. You must manually enable it if your headers require preprocessing.
  4. Generate unit tests with the test generator

    main

    You can automatically generate unit test code for cxxheaderparser by using the cxxheaderparser.gentest module.

    To use it:

    1. Ensure your C++ header content is saved in a file.
    2. Run the generator command specifying the header filename and the desired test name.
    3. Copy the resulting output from stdout into a test file within your project's tests directory.
    python -m cxxheaderparser.gentest FILENAME.h TESTNAME
  5. Implement custom parsing with CxxVisitor

    main

    When the simple API is insufficient, you can implement a custom parsing mechanism by defining a visitor class that implements the CxxVisitor protocol.

    To use a custom visitor:

    1. Define a class that implements the CxxVisitor protocol (you can use SimpleCxxVisitor as a reference for available callbacks).
    2. Instantiate your visitor.
    3. Pass the visitor instance to the CxxParser constructor along with the filename and content.
    4. Call parser.parse() to trigger the callbacks in your visitor.
    visitor = MyVisitor()
    parser = CxxParser(filename, content, visitor)
    parser.parse()
    
    # do something with the data collected by the visitor
  6. Parse C++ using directives, declarations, and type aliases

    main

    The parser handles three types of using statements:

    1. Using Directive: using namespace IDENTIFIER; or using namespace ::IDENTIFIER;.
    2. Using Declaration: using typename IDENTIFIER; or using ::IDENTIFIER;.
    3. Using Type Alias: using IDENTIFIER = type_id; (often used with templates).

    When parsing these, the visitor pattern is used to notify the consumer of the specific type of using statement encountered via on_using_namespace, on_using_declaration, or on_using_alias.

  7. Parse C++ enums and enumerators

    main

    The parser supports parsing C++ enumerations, including enum, enum class, and enum struct.

    Key features include:

    • Base Types: Supports enums with an explicit base type (e.g., enum E : int).
    • Forward Declarations: Recognizes enum forward declarations.
    • Enumerators: Parses individual enumerators, including those with explicit values (e.g., VAL = 1) and those with Doxygen comments or attributes.
    • Attributes: Supports attributes attached to the enum or individual enumerators.
  8. Parse C++ function and method declarations

    main

    The parser identifies functions and methods by looking for a return type, a name (or operator), and a parameter list.

    • Methods: When parsing inside a ClassBlockState, the parser identifies methods and can distinguish between constructors, destructors, and regular methods based on the name matching the class name.
    • Functions: Functions declared outside of a class are parsed as Function objects.
    • Trailing Return Types: The parser supports C++11 trailing return types (e.g., auto func() -> int) by detecting the ARROW token.
    • Deduction Guides: It also supports parsing C++20 deduction guides.

    When a function or method is successfully parsed, the parser notifies the registered visitor via on_method_impl, on_class_method, or on_function depending on the context.

  9. Parse C++ typedefs

    main

    The parser handles typedef declarations. A valid typedef must follow these rules:

    • It must have a single name segment (no nested-name-specifiers).
    • The name must not be empty.
    • It cannot be constexpr, extern, static, inline, or a template.
    • It cannot have a function body (it must be a declaration).
    • It must have a return type.

    Upon successful parsing, the on_typedef visitor method is called with a Typedef object.

  10. Generate dependency files (depfiles) with preprocessors

    main

    Both the GCC and PCPP preprocessors support generating dependency files (depfiles) that list the include files parsed during preprocessing.

    To use this, you must provide both depfile and deptarget:

    • depfile: A pathlib.Path where the dependency list will be written.
    • deptarget: A List[str] of targets to include in the depfile.

    Note: For make_pcpp_preprocessor, depfile is not compatible with retain_all_content=True.

  11. Represent C++ types with PQName

    main

    A PQName (Possibly Qualified Name) represents a C++ type name, which may be segmented by ::. It is composed of one or more PQNameSegment objects. If a name refers to the global namespace, the first segment is an empty NameSpecifier.

    Segments can be:

    • FundamentalSpecifier: Basic types like int, char, void.
    • NameSpecifier: A named segment, which may include a TemplateSpecialization.
    • DecltypeSpecifier: The contents of a decltype(...) expression.
    • AutoSpecifier: The auto keyword.
    • AnonymousName: An integer-based ID for anonymous classes.

    You can use the .format() method to reconstruct the C++ string representation.

    # Example of how PQName segments might be structured conceptually
    # (Actual instantiation depends on the parser output)
    # pq_name.format() -> "std::vector<int>"