Fileseq

repository·master·Indexed 43 days ago

https://github.com/justinfx/fileseq

A Python library for parsing and managing frame ranges and file sequences commonly used in VFX and Animation pipelines. It supports various padding syntaxes (Standard, Printf, Houdini, and UDIM), subframes, and complex frame range shorthands. Key features include the FrameSet container for sequence management, FileSequence for path parsing and formatting, and utilities for discovering sequences on disk.

Tokens
13.1K
Snippets
43
Records
51
Agent score
55%

What's inside Fileseq

  1. Understand the Fileseq grammar patterns

    master

    The Fileseq grammar defines four primary patterns for identifying file sequences. These patterns are shared across Python, Go, and C++ implementations to ensure consistent parsing behavior.

    PatternDescriptionExample
    sequenceFull sequence with a frame range and padding/path/file.1-100#.exr
    patternOnlyPadding without an explicit frame range/path/file.#.exr
    singleFrameA single frame file with padding/path/file.0100.exr
    plainFileA standard file with no frame pattern/path/file.txt
  2. How FrameSet works for managing frame ranges

    master

    A FrameSet is an immutable, ordered, and deduplicated sequence of frame numbers parsed from a range string. It supports set-algebra operations (union, intersection, etc.) and can handle fractional (subframe) numbers using Decimal or float iterables.

    Range Syntax Examples:

    • 1-10: Standard range (1, 2, ..., 10)
    • 1-10x2: Step (1, 3, 5, 7, 9)
    • 1-10y2: Fill/Inverse step (2, 4, 6, 8, 10)
    • 1-10:3: Stagger (1-10x3, 1-10x2, 1-10)
    • -5-5: Negative frames
    • 1-5,10-20: Comma-separated ranges

    Subframe Support: Pass Decimal values or a subframe range string (e.g., '1-3x0.5') to include fractional frames.

    from fileseq import FrameSet
    
    # Construction
    fs = FrameSet('1-10x2')
    fs_sub = FrameSet('1-3x0.5') # 1.0, 1.5, 2.0, 2.5, 3.0
    
    # Iteration and indexing
    list(fs)    # [1, 3, 5, 7, 9]
    fs[0]       # 1
    len(fs)     # 5
    5 in fs     # True
    
    # Set operations
    a = FrameSet('1-5')
    b = FrameSet('3-7')
    
    a | b  # union:        1-7
    a & b  # intersection: 3-5
    a - b  # difference:   1-2
    a ^ b  # symmetric difference: 1-2, 6-7
  3. Supported Frame Range Shorthand Syntax

    master

    Fileseq supports a wide variety of frame range shorthand syntaxes used in VFX and animation pipelines. This includes standard ranges, comma-delimited lists, chunked, filled, and staggered sequences, as well as support for negative frame numbers and subframes.

    Padding Syntax

    • Standard Padding: #=4 (4-digit padded), @=1 (1-digit padded)
    • Alternate Padding: #=1, @=1
    • Printf Syntax: %04d=4 (4-digit padded), %01d=1 (1-digit padded)
    • Houdini Syntax: $F4=4 (4-digit padding), $F=1 (1-digit padding)
    • UDIM Syntax: <UDIM> or %(UDIM)d (always 4-digit padded)
  4. Use Python-specific subframe notation

    master

    The Python implementation of Fileseq supports advanced subframe notation. While these patterns are valid according to the grammar, they are currently ignored by Go and C++ implementations.

    Supported Python patterns:

    • Dual range: Main frames and subframes (e.g., file.1-5#.10-20@@.exr)
    • Composite padding: Frame and subframe padding (e.g., file.1-5@.#.exr)
    • Pattern only: Wildcard for both components (e.g., file.#.#.exr)
    # Dual range
    file.1-5#.10-20@@.exr
    
    # Composite padding
    file.1-5@.#.exr
    
    # Pattern only
    file.#.#.exr
  5. Regenerate the Python parser from the grammar

    master

    If you modify the ANTLR4 grammar file located at src/fileseq/grammar/fileseq.g4, you must regenerate the Python parser to apply the changes. You can do this using the hatch build tool or by calling the ANTLR Java JAR directly.

    Requirements:

    • Java 11+ must be in your PATH.
    • The ANTLR 4.13.1 JAR (provided in the tools/ directory) must be available.
    # Using hatch
    hatch run generate
    
    # Or directly with Java
    java -jar tools/antlr-4.13.1-complete.jar \
        -Dlanguage=Python3 \
        -visitor \
        -o src/fileseq/parser \
        src/fileseq/grammar/fileseq.g4
  6. Extend FileSequence with custom subclasses

    master

    You can subclass FileSequence to implement custom logic. To ensure your subclass is used during discovery, you can:

    1. Call methods directly on the subclass: This automatically uses your subclass's hooks (_preprocess_sequence, _resolve_padding, getPaddingNum).
    2. Pass the subclass via the klass argument: If calling from a generic utility that uses FileSequence methods, pass your subclass to the klass parameter to override the construction class.
    # Approach 1: Call directly on subclass
    seqs = VRayFileSequence.findSequencesOnDisk('/renders')
    
    # Approach 2: Use the 'klass' argument
    seqs = FileSequence.findSequencesOnDisk('/renders', klass=VRayFileSequence)
  7. Install and develop Fileseq

    master

    Fileseq uses hatch for development and builds.

    Installation

    pip install hatch
    pip install -e ".[dev]"

    Running Tests

    hatch run test
    hatch run test-cov

    Documentation

    hatch run docs:build
    hatch run docs:serve

    Regenerating the Parser

    If you modify the grammar file (src/fileseq/grammar/fileseq.g4), you must regenerate the parser (requires Java 11+):

    hatch run generate
  8. Subclass FileSequence to support custom syntax

    master

    You can extend fileseq.FileSequence or fileseq.FilePathSequence by subclassing fileseq.BaseFileSequence. This allows you to support non-standard notation for frame ranges, padding, or path types by overriding specific internal hooks.

    To create a fully round-trippable custom sequence (where str(seq) returns the original custom syntax), you typically need to override three methods:

    1. _preprocess_sequence: Translate custom syntax into a format the fileseq grammar understands.
    2. _resolve_padding: Translate the parsed grammar padding back into your custom token for internal storage.
    3. getPaddingNum: Ensure setPadding and other methods recognize your custom padding token.

    Additionally, you can use _postprocess_sequence to restore custom syntax (like range delimiters) that isn't related to the padding token itself.

    import re
    import fileseq
    
    class VRayFileSequence(fileseq.FileSequence):
        """Translate VRay `<frameNN>` padding tokens natively."""
    
        _VRAY_PAD_RE = re.compile(r'<frame(\d+)>')
        _used_custom_padding = False
    
        def _preprocess_sequence(self, sequence: str) -> str:
            """Translate `<frameNN>` → `%0Nd` so the grammar accepts it."""
            def replace(m):
                width = int(m.group(1))
                self._used_custom_padding = True
                return '%0{}d'.format(width) if width > 0 else '%d'
            return self._VRAY_PAD_RE.sub(replace, sequence)
    
        def _resolve_padding(self, parsed_pad: str, zfill: int, pad_style) -> str:
            """Translate the parsed printf form back to the canonical VRay token."""
            if zfill > 0 and self._used_custom_padding:
                return '<frame{:02d}>'.format(zfill)
            return parsed_pad
    
        @classmethod
        def getPaddingNum(cls, chars: str, **kwargs) -> int:
            """Recognise `<frameNN>` tokens in addition to the built-in formats."""
            m = cls._VRAY_PAD_RE.match(chars)
            if m:
                return int(m.group(1))
            return super().getPaddingNum(chars, **kwargs)
    
    # Usage
    seq = VRayFileSequence('/render/beauty.1-100<frame04>.exr')
    print(seq.padding())     # '<frame04>'
    print(str(seq))          # '/render/beauty.1-100<frame04>.exr'
    print(seq.frame(42))     # '/render/beauty.0042.exr'
  9. Configure FileSequence padding styles

    master

    You can control how the # token is interpreted using the pad_style argument during FileSequence construction.

    • PAD_STYLE_DEFAULT: # maps to 4-digit zero-padded (e.g., 0001).
    • PAD_STYLE_HASH1: # maps to 1 digit, and #### maps to 4 digits (Houdini convention).
    from fileseq import FileSequence, PAD_STYLE_DEFAULT, PAD_STYLE_HASH1
    
    # Default: '#' maps to 4-digit zero-padded
    seq = FileSequence('/out/f.1-10#.exr', pad_style=PAD_STYLE_DEFAULT)
    
    # Hash1: '#' maps to 1 digit, '####' to 4 digits (Houdini convention)
    seq = FileSequence('/out/f.1-10#.exr', pad_style=PAD_STYLE_HASH1)
  10. Create a FrameSet from various inputs

    master

    A FrameSet is an immutable, ordered, and unique set of frames. You can instantiate it using several types of input:

    • Frame range strings: Supports shorthand like 1-10, 1-100x5 (every 5th frame), 1-100y5 (fills in missing frames), and 1-100:5 (staggered frames).
    • Iterables: A list, set, or any iterable of frame numbers (integers, floats, or decimal.Decimal).
    • Individual values: A single int, float, or decimal.Decimal.
    • Existing FrameSets: Using a FrameSet as input acts as a copy constructor.

    Note: If the range is too large (exceeding fileseq.constants.MAX_FRAME_SIZE), a fileseq.exceptions.MaxSizeException is raised. If the string format is invalid, a fileseq.exceptions.ParseException is raised.

    from fileseq import FrameSet
    
    # From string
    fs = FrameSet('1-100x5')
    
    # From iterable
    fs = FrameSet([1, 2, 3, 4, 5])
    
    # From single value
    fs = FrameSet(10)
    
    # From range (inclusive)
    fs = FrameSet.from_range(1, 100, step=2)
  11. Initialize a FileSequence

    master

    The FileSequence class represents an ordered sequence of files. You can instantiate it by passing a sequence string (e.g., dir/path.1-100#.ext). It automatically parses the directory, basename, extension, padding, and frame range.

    Arguments:

    • sequence (str): The file path pattern containing frame ranges and padding.
    • pad_style (optional): The padding style to use (e.g., PAD_STYLE_DEFAULT, PAD_STYLE_HASH1, PAD_STYLE_HASH4).
    • allow_subframes (bool): Whether to allow subframes in the sequence.
    from fileseq import FileSequence
    
    # Standard sequence
    seq = FileSequence('/foo/bar.1-10#.exr')
    
    # Sequence with subframes
    seq_sub = FileSequence('/foo/bar.1-10.001-005#.#.exr', allow_subframes=True)
  12. Use _postprocess_sequence to restore custom range syntax

    master

    The _postprocess_sequence hook is called by __str__ and format just before the sequence pattern string is returned. Use this to restore custom syntax for parts of the string that are not the padding token, such as custom frame range delimiters (e.g., using = instead of -).

    Note: _postprocess_sequence is not called when resolving individual frame paths via .frame() or iteration. Those methods use the internally stored padding directly to ensure correctness.

    import re
    import fileseq
    
    class EqualRangeFileSequence(fileseq.FileSequence):
        """Accept `1=100` range syntax in addition to the standard `1-100`."""
    
        _EQUAL_RE = re.compile(r'(\d+)=(\d+)')
    
        def _preprocess_sequence(self, sequence: str) -> str:
            return self._EQUAL_RE.sub(r'\1-\2', sequence)
    
        def _postprocess_sequence(self, sequence: str) -> str:
            # FrameSet always emits '1-100'; restore the custom delimiter on output.
            return sequence.replace('-', '=', 1)
    
    seq = EqualRangeFileSequence('/render/beauty.1=100#.exr')
    print(str(seq))       # '/render/beauty.1=100#.exr'
    print(seq.frame(42))  # '/render/beauty.0042.exr'