Phobos Standard Library

repository·master·Indexed 22 days ago

https://github.com/dlang/phobos

The standard library for the D Programming Language, providing a comprehensive set of modules and functions bundled with the D compiler. The repository includes the current Phobos 2 library and Phobos 3, a next-generation version under active development featuring a split-level design to minimize allocations and exceptions, a hybrid single-root/multi-trunk package structure, and standardized UTF-8 internals.

Tokens
2.9K
Snippets
7
Records
13
Agent score
79%

What's inside Phobos

  1. Status of Phobos 3

    master

    Phobos 3 is the next generation of the D Standard Library. It is intended to supersede the existing Phobos 2 library (found in the std folder).

    Warning: Phobos 3 is currently under active development and is not ready for use at any level.

  2. Memory Management and Split-Level Design

    master

    Phobos 3 follows a 'split-level' design pattern to minimize memory allocations and exceptions, making the library more usable in performance-critical or @nogc contexts.

    Split-Level Design Pattern

    1. Low-level functions: These take in a user-provided buffer instead of allocating memory internally and return error codes instead of throwing exceptions.
    2. High-level functions: These wrap the low-level functions, providing a simpler API by managing buffers and converting error codes into exceptions.

    Key Principles

    • Minimize Allocations: Library routines should avoid deciding allocation strategies for the user. For example, std.path returns ranges that allow users to emit results into their own chosen buffers.
    • Minimize Exceptions: Exceptions are avoided where possible because they require the GC and prevent nothrow usage. The library prefers returning error codes or using Option/Sum types for error returns.
  3. Understand the Phobos 3 Proposed Package Structure

    master

    The Phobos 3 package map outlines a reorganized structure for the standard library. It categorizes modules into several top-level namespaces: core.*, etc.*, std.*, platform, based, phobos.sys, phobos.data, phobos.crypto, phobos.io, phobos.text, and phobos.net.

    Key organizational changes include:

    • based modules: These contain a mix of wrapped based functions, new functions, and public imports of based symbols. Many common utilities (like math, time, and traits) are being moved or reorganized here.
    • phobos.sys: Contains low-level system-oriented modules such as algorithm, array, datetime, exception, random, and range.
    • phobos.data: Focuses on data formats like base64, csv, json, toml, sdl, and zip.
    • phobos.crypto: Provides cryptographic primitives including digest, hash, kdf, rsa, and symmetric algorithms.
    • phobos.io, phobos.text, and phobos.net: Handle I/O, text processing (ASCII, UTF, encoding), and networking (HTTP, sockets, TLS, URI).

    Note: This structure is proposed and not final. Modules marked with an asterisk () are new.*

    core.*
    etc.*
    std.*
    platform
      | freebsd
      | linux
      | macos
      | stdc
      | win
    based
      | bigint
      | bitmanip
      | checkedint (core.checkedint)
      | complex
      | compiler
      | console (std.stdio)
      | demangle (core.demangle)
      | file
      | int128
      | math
        | algebraic
        | constants
        | exponential
        | hardware
        | operations
        | remainder
        | rounding
        | traits
        | trigonometry
        | special
        | numeric
      | meta
      | optional* (std.typecons)
      | stdint
      | sumtype
      | system
      | time (core.time)
      | traits
    phobos.sys
      | algorithm
        | comparison
        | iteration
        | mutation
        | searching
        | setops
        | sorting
      | array
      | checkedint
      | conv
      | datetime
        | date
        | interval
        | stopwatch
        | systime
        | timezone
      | demangle
      | exception
      | functional
      | meta
      | outbuffer
      | process
      | random
      | range
      | signals
      | traits
      | uuid
      | variant
    phobos.data
      | base64
      | csv
      | json*
      | toml*
      | sdl*
      | zip
    phobos.crypto
      | digest
        | crc
        | murmurhash
      | ecc*
      | hash*
        | sha*
        | hmac*
      | kdf*
      | random*
      | rsa*
      | symmetric*
    phobos.io
      | console (std.stdio)
      | stream* (iopipe)
      | mmfile
      | path
    phobos.text
      | ascii
      | encoding
      | format
      | string
      | uni
      | utf
    phobos.net
      | http*
      | socket
      | tls*
      | uri
  4. Working with Strings in Phobos 3

    master

    Phobos 3 has moved away from several legacy string behaviors to improve performance and predictability:

    No Autodecoding

    Strings no longer automatically decode. Users must explicitly request decoding using filters like utf.byDchar.

    UTF-8 Standard

    Internals (algorithms, ranges, and functions) are standardized on UTF-8. Support for wchar and dchar is provided via specific algorithms:

    • utf.byChar
    • utf.byWchar
    • utf.byDchar

    Handling Invalid Unicode

    Phobos 3 avoids throwing exceptions for invalid Unicode to allow for @nogc and nothrow code. Instead of failing, APIs allow callers to specify behavior, such as:

    • Returning an 'error' result.
    • Replacing invalid sequences with the Unicode substitution character.
    • Using std.utf.validate to check if a ubyte[] is valid UTF-8.
    • Using by-code-point decoding to report errors for individual operations.
  5. Understand the Phobos 3 Package Structure

    master

    Phobos 3 uses a hybrid single-root/multi-trunk design. It uses a single package root phobos. which contains multiple 'trunk' packages. This allows the library to be split into smaller, manageable components that can be compiled and linked separately, reducing executable weight.

    Key characteristics:

    • Core Roots: The foundation of the library. Currently includes core, etc, and phobos.sys.
    • Layering: Higher-level packages are built upon the core roots.
    • Platform Support: Not all trunks must be implemented for a platform to be considered supported; if only core roots are required, porting becomes easier.
    • Constraints: The phobos.sys trunk is prohibited from importing from non-core trunks.
    • Compatibility: The old std root is maintained for compatibility but is built independently of the new phobos. root.
  6. Install Phobos Standard Library

    master

    Phobos is the standard library for the D Programming Language and is packaged together with the D compiler. To use Phobos, you should download the complete precompiled package from the official D website.

    If you need to build the library from source, refer to the DMD building instructions in the D wiki.

    http://dlang.org/download.html
  7. Access Phobos API Documentation

    master

    The complete API documentation for the Phobos standard library is available online. Use this resource to explore available modules, functions, and types.

    http://dlang.org/phobos/
  8. How to add a new changelog entry

    master

    To add a new entry to the pending changelog, create a new file in the changelog folder with a .dd extension. The file format should resemble a git commit message:

    1. Title: The first line is the title of the change. It cannot contain links.
    2. Description: After an empty line, provide a long description of the change.
    3. Examples: You can include code examples using ------- as a separator.
    4. Links: Use Ddoc syntax for internal and external links:
      • $(REF module, function) or $(REF_ALTTEXT alt_text, module, function) for internal references.
      • $(LINK2 path, text) for linking to local files (e.g., $(LINK2 $(ROOT_DIR)spec/module.html, this)).
      • $(LINK2 https://url.com, text) for external resources.
    My fancy title of the new feature
    
    A long description of the new feature in `std.range`.
    It can be followed by an example:
    -------
    import std.range : padLeft, padRight;
    import std.algorithm.comparison : equal;
    
    assert([1, 2, 3, 4, 5].padLeft(0, 7).equal([0, 0, 1, 2, 3, 4, 5]));
    
    assert("Hello World!".padRight('!', 15).equal("Hello World!!!!"));
    -------
    and links to the documentation, e.g. $(REF drop, std, range) or
    $(REF_ALTTEXT a custom name for the function, drop, std, range).
    
    Links to the spec can look like this $(LINK2 $(ROOT_DIR)spec/module.html, this) 
    and of course you can link to other $(LINK2 https://forum.dlang.org/, external resources).
  9. Overview of zlib data compression

    master

    zlib 1.3.1 is a general-purpose, thread-safe data compression library. It supports several data formats defined by IETF RFCs:

    • zlib format: Described by RFC 1950.
    • deflate format: Described by RFC 1951.
    • gzip format: Described by RFC 1952.

    For full API documentation, refer to the zlib.h header file in the source tree.

  10. Install and compile zlib

    master

    To compile zlib and run the test programs on most Unix-like systems, use the following sequence:

    1. Run ./configure to prepare the build.
    2. Run make test to compile and verify the library works correctly.
    3. Run make install to install the library to the system.

    Platform Specifics:

    • Windows: Use the special makefiles located in win32/ or contrib/vstudio/.
    • VMS: Use make_vms.com.
    • 64-bit Irix: deflate.c must be compiled without optimization (avoid -O) to prevent failures in certain tests.
    ./configure
    make test
    make install