DOCSEARCHbyAlgolia
Submit Docs

Oils Documentation

repository·master·Indexed 25 days ago

https://github.com/oils-for-unix/oils

Oils is a modern upgrade path from bash, providing a better language and runtime via OSH for shell script compatibility and YSH for Python or JavaScript backgrounds. This documentation covers the Oils builtin architecture, J8 notation implementations, Docker image usage, and the mycpp Python-to-C++ translator, including its specific patterns for switch statements, RAII, and downcasting.

Tokens
117.1K
Snippets
390
Records
822
Agent score
85%

What's inside Oils

  2. Overview of framer for C extension boilerplate generation

    master
    framer is a tool designed to automate the generation of boilerplate C code required for C extension types. It generates function definitions, argument handling code, and type objects based on a specification object written in Python. The specification utilizes Python's class statement to describe the extension module and its constituent extension types.

  4. Overview of Yaks

    master
    Yaks is a minimal, TYPED language designed to compile to the mycpp runtime. It uses the NIL8 syntax, which is a Lisp-ish Intermediate Representation (IR) with imperative semantics. Yaks is designed to be human-readable and can be written by hand or generated losslessly from other languages (like Python) to provide precise type errors including file, line, and column information.

  5. Overview of the mycpp Runtime

    master

    The mycpp translator targets a custom-built runtime that implements garbage-collected data structures and core Python-like functionality. Key components include:

    • Typed Records: Supports Python classes and ASDL product/sum types.
    • Built-ins: mycpp/gc_builtins.{h,cc} provides functionality similar to Python's __builtin__ (e.g., int(), str()).
    • Standard Library: mycpp/gc_mylib.{h,cc} provides utility functions (e.g., mylib.BufWriter similar to cStringIO.StringIO).

    Key Differences from CPython:

    • Integers: Uses C int or mylib.BigInt instead of Python's arbitrary-precision integers.
    • String Handling: s.strip() uses ASCII whitespace (consistent with JSON/J8) and does not include characters like \v.
    • Syscalls: NUL bytes are permitted in arguments to syscalls like open().

  6. Overview of TSV8 Data Format

    master

    TSV8 is the primary exterior data format used in the Oils design for representing tables. It is designed to be both human-readable and machine-readable, serving as a bridge between raw lines and structured rows.

    Key characteristics:

    • Aligned/SSV8: Supports an aligned format for better readability.
    • Typed Columns: Columns can have specific types and attributes.
    • Lazy Parsing: Designed for partial parsing, allowing tools to decode only the necessary columns (e.g., using table --by-col).
    • Dual Representation: Can be viewed as a stream of rows (JSONLines-style) or as column-oriented data.

  8. Overview of Lazy, Lossless Lexing Libraries

    master
    The project implements lazy, lossless lexing libraries designed to process structured text formats (like HTML, TSV8, and JSON8) with high efficiency. Unlike DOM-based parsers that construct full trees and allocate significant memory, these lexers provide an iterator of events or spans. This approach minimizes allocations, preserves whitespace for diff-based testing, and allows for multiple transformation passes without losing the original document structure or 'stack'.

  9. Overview of Oils repository structure

    master

    The Oils repository is organized into several functional areas:

    Interpreters

    • bin/: Main entry points (e.g., bin/osh).
    • frontend/: Common input and lexing for OSH and YSH.
    • osh/: OSH parsers and evaluators.
    • ysh/: YSH parser and evaluator.
    • data_lang/: JSON-based languages.
    • core/: Shared code between OSH and YSH.
    • builtin/: Builtin commands and functions.
    • pylib/: Python standard library components.
    • tools/: User-facing tools like the osh2oil translator.
    • display/: User interface components.

    DSLs and Code Generators

    • asdl/: ASDL implementation.
    • pgen2/: Parser Generator.
    • mycpp/: Experimental translator from typed Python to C++ (requires MyPy).
    • pea/: Experimental cleaner version of mycpp.
    • opy/: Obsolete Python compiler.

  10. Overview of Oils Data Notations

    master

    Oils shell programs can be built using several well-defined data notations and text interchange formats. The primary notations supported include:

    • J8 Notation: A data language used for strings, lines, and structured data.
    • JSON8: An extension or variant of JSON.
    • TSV8: A tab-separated values format with specific column attributes and types.

    All J8 notation is encoded in UTF-8.

  11. Overview of QSN (Quoted String Notation)

    master

    QSN is a data format designed for representing byte strings. It is an adaptation of Rust's string literal syntax, using single quotes instead of double quotes to avoid confusion with JSON.

    Note: As of January 2024, QSN has been replaced by J8 Notation, which is more harmonized with JSON. Use J8 Notation for new projects if possible.