DOCSEARCHbyAlgolia
Submit Docs

Cog Documentation

repository·main·Indexed 19 days ago

https://github.com/nedbat/cog

Cog is a content generation tool for executing Python snippets in source files to automate the creation or updating of static file content. It provides a command-line interface and a programmatic API via the Cog and CogGenerator classes to extract and execute code within markers, manage output via the cog module, and verify content integrity using HashHandler checksums.

Tokens
11K
Snippets
42
Records
51
Agent score
58%

What's inside cog

  1. Overview of Cog

    main
    Cog is a content generation tool designed for performing small bits of computation for static files. It is used to automate the generation of parts of files that might otherwise need to be updated manually.

  2. What is Cog and how does it work?

    main

    Cog is a content generation tool that allows you to embed small snippets of Python code within static files to generate text dynamically.

    How it works:

    1. Cog identifies chunks of Python code wrapped in specific marker lines.
    2. It executes the embedded Python code.
    3. It replaces the space between the markers with the output produced by the Python code.
    4. All text outside of these markers is passed through unchanged.

    This allows you to maintain a single source file that contains both the logic (Python) and the template (the surrounding text), making it easy to generate repetitive code like C++ function declarations, configuration files, or documentation.

  3. Protect output with checksums

    main

    Use the -c flag to append a compact base64 checksum to the end of the generated output. This protects the output against accidental manual edits. If a developer edits the generated code, cog will detect the mismatch during the next run and stop to prevent overwriting the manual changes.

    Cog also supports the older hex checksum format: [[[end]]] (checksum: <hex_string>) and will automatically upgrade these to the shorter base64 format upon regeneration.

    --[[[cog
--   import cog
--   for i in range(10):
--      cog.out("%d " % i)
--   ]]]
0 1 2 3 4 5 6 7 8 9
--[[[end]]] (sum: vXcVMEUp9m)

  4. How to use Cog markers in files

    main

    To embed Python code in a file, use triple square bracket markers. The lines between [[[cog and ]]] contain the Python code, and the lines between ]]] and [[[end]]] define the output area.

    Marker Syntax:

    • Start block: [[[cog
    • End code: ]]]
    • Start output area: ]]]
    • End block: [[[end]]]

    Pro-tip: Marker lines can contain additional text (like comment characters) to ensure the file remains valid for other tools (e.g., a C++ compiler or a linter) while the Python code is present.

    /*[[[cog
import cog
fnames = ['DoSomething', 'DoAnotherThing']
for fn in fnames:
    cog.outl("void %s();" % fn)
]]]*/
//[[[end]]]

  5. How to write Cog source files

    main

    Cog source files are primarily plain text files containing chunks of Python code wrapped in specific marker lines. When Cog runs, it executes the Python code and replaces the area between the end-marker and the end-marker with the code's output.

    Marker Syntax

    Markers are identified by the presence of the [[[cog and [[[end]]] sequences. The text preceding these sequences is ignored, allowing you to use markers that match your file's comment style (e.g., #, --, //, or /*).

    Python Chunk Execution

    • Sequential Execution: Each chunk is executed in sequence.
    • Shared State: Multiple chunks in the same file share the same globals dictionary, allowing you to define functions or imports in one chunk and use them in subsequent chunks.
    • Indentation Handling: Cog automatically manages indentation. It identifies a common whitespace prefix from the output block and re-indents the entire block so that the leftmost non-whitespace character matches the indentation of the [[[cog marker line. This allows you to indent your Python code to match the surrounding file structure.
    # Example of a standard Python chunk in a Python file
[[[cog
import cog
print("Hello from Cog!")
]]] 
[[[end]]]

  6. How Cog's code generation model works

    main

    Cog is a general-purpose code generation tool that uses Python to transform text files. Instead of using separate input and output files, Cog works by reading a host file (e.g., C++, Java, or any text-based language) and looking for specially-marked sections containing Python code.

    When Cog runs, it:

    1. Identifies sections delimited by /* [[[cog and ]]] */ (or similar marker patterns).
    2. Discards any existing generated content between the ]]] marker and the [[[end]]] marker.
    3. Executes the Python code found within the cog block.
    4. Captures the output (typically via cog.outl calls).
    5. Inserts that captured output into the file, effectively "splicing" the generated code back into the host file.

    This allows the generator code and the generated output to live side-by-side in the same file, making the process an edit-time activity rather than a complex build-time step.

  7. How Cog manages source and output files

    main
    Cog uses an in-place update model. Instead of requiring a separate source file and a separate output file, Cog writes its results back into the original file while retaining the code it executed. This allows a single file to serve as both the source of the generation logic and the final output. Because of this design, Cog can be run multiple times on the same file without creating duplicate or separate files.

  8. Managing Cog files in source control

    main
    Cog uses marker lines that are designed to accommodate any language syntax. These markers can effectively 'hide' the Cog Python code within your text file. This enables you to check Cog files directly into source control without needing to manage separate source and output files or modify complex build procedures.

  9. Using Cog for code generation without a templating engine

    main
    Unlike traditional code generation tools that rely on templating engines (which can struggle with whitespace and complex logic), Cog uses standard Python code to generate text. This allows you to use the full power of the Python language for text generation, avoiding the mental overhead of converting Python logic into a specific templating syntax.

  10. Use Cog markers with file-specific comment styles

    main

    To accommodate different file formats, Cog recognizes any line containing the [[[cog sequence as a start marker. You can prefix the marker with your language's comment syntax.

    Examples of valid markers:

    • Python/Shell: # [[[cog
    • C/C++/Java: // [[[cog or /* [[[cog */
    • SQL: -- [[[cog

    If you need custom markers that do not follow these patterns, use the --markers CLI option to define them.

    # Example markers for different languages
//[[[cog
/* [[[cog */
-- [[[cog
#if 0 // [[[cog

  11. Run cog via CLI or Python module

    main

    Cog can be used as a standalone command-line utility or invoked as a Python module. When using the module approach, the package name is cogapp.

    CLI Usage:

    $ cog [options] [INFILE | @FILELIST | &FILELIST] ...

    Module Usage:

    $ python3 -m cogapp [options] [arguments]
    # CLI example
$ cog -h

# Module example
$ python3 -m cogapp -h