Understand the Skim Item Ingestion Pipeline
masterSkim uses a unified parallel pipeline (parallel_bufread) to ingest all inputs, including plain stdin, shell commands, and flags like --ansi or --nth.
Pipeline Stages:
- I/O Reader: Reads 256 KB chunks, splits at line boundaries, and assigns sequence numbers.
- Workers: Validate UTF-8 and create
DefaultSkimIteminstances. This stage handles ANSI stripping and field transformations (e.g.,--nth,--hide-nth). - Reorder: Collects items and emits them in the correct sequence.
- Killer: Ensures child processes are terminated if the pipeline is interrupted.
Item Construction Behavior:
When using --ansi and --nth (transformed fields), the text field contains the transformed content, while orig_text preserves the original line for output purposes.