DOCSEARCHbyAlgolia
Submit Docs

pycparser Documentation

repository·main·Indexed 25 days ago

https://github.com/eliben/pycparser

A pure Python C parser designed for easy integration into applications, commonly used for static code checking, C code obfuscation, and generating FFIs. It provides tools to convert C source code into an Abstract Syntax Tree (AST), traverse the AST using NodeVisitor, and reconstruct C source code via CGenerator. The library includes a standalone CLexer and utility functions like fix_switch_cases and fix_atomic_specifiers to refine AST structures.

Tokens
3.2K
Snippets
1
Records
32
Agent score
85%

What's inside pycparser

  1. Generate preprocessed C files for benchmarking

    main
    The benchmarking suite uses preprocessed files located in the inputs directory. These files are sourced from open-source projects like Redis, TCC, and SQLite. To generate these .pp files for testing, use gcc with the -E flag (preprocess only), -nostdinc to ignore standard includes, and -D'__attribute__(x)=' to strip C attributes that pycparser might not support. You must also include a directory containing fake libc headers (e.g., $HOME/eli/pycparser/utils/fake_libc_include) to satisfy dependencies.

  2. Handle C preprocessor requirements for realistic C code

    main
    When parsing realistic C code samples with pycparser, you must run the C preprocessor before passing the code to the parser. This is necessary because pycparser expects preprocessed code. Refer to the main README.rst or external guides on using fake headers to manage preprocessor dependencies.

  5. Parse C files using the preprocessor

    main

    For most C code, pycparser requires the code to be preprocessed first. If you use the top-level parse_file function, it will automatically interact with the C preprocessor (cpp) if it is in your PATH.

    Alternatively, you can use gcc -E or clang -E as the preprocessor.

    Note: pycparser does not include standard C library headers in the pip package. To parse code that includes standard headers (like stdio.h), it is recommended to use the 'fake' standard includes provided in utils/fake_libc_include. These headers contain only the bare necessities required for valid parsing and can improve performance.

  6. Regenerate AST nodes after modifying configuration

    main
    The code for pycparser's AST nodes is automatically generated from the _c_ast.cfg configuration file using the _ast_gen.py script. If you modify the AST configuration in _c_ast.cfg, you must re-generate the code by running the _ast_gen.py script from the repository root or the pycparser directory.

  7. Explore pycparser AST nodes and API

    main

    For developers looking to build on top of pycparser:

    • Public API Documentation: The public interface is documented via comments within pycparser/c_parser.py.
    • AST Node Reference: For a detailed overview of the various Abstract Syntax Tree (AST) nodes created by the parser, refer to pycparser/_c_ast.cfg.
    • Configuration: Detailed information regarding configuration and compatibility arguments can be found in the docstring of the CParser class constructor.

  8. Preprocess a C file using `preprocess_file`

    main

    Use preprocess_file to run the C preprocessor (cpp) on a file and retrieve the resulting text. This is useful for resolving macros and includes before parsing.

    Arguments:

    • filename: The name of the file to preprocess.
    • cpp_path: The path to the cpp executable (defaults to "cpp").
    • cpp_args: Command line arguments for cpp. Can be a string or a list of strings (e.g., r'-I../include').

    Returns: The preprocessed file's contents as a string.

    Errors: Raises RuntimeError if cpp cannot be invoked.

  9. Parse a C file using `parse_file`

    main

    Use parse_file to convert a C source file into an Abstract Syntax Tree (AST).

    Arguments:

    • filename: The name of the file to parse.
    • use_cpp: Set to True to run the C preprocessor on the file before parsing.
    • cpp_path: Path to the cpp executable (used if use_cpp is True).
    • cpp_args: Arguments for cpp. Use a raw string (e.g., r'-Ipath') or a list of strings.
    • parser: An optional parser object to use instead of the default CParser.
    • encoding: The encoding to use when reading the file.

    Returns: An AST object.

    Errors: Raises ParseError if the file fails to parse.