DOCSEARCHbyAlgolia
Submit Docs

rhubarb-lip-sync

repository·master·Indexed 25 days ago

https://github.com/danielswolf/rhubarb-lip-sync

A repository containing documentation and build instructions for several integrated libraries, including libogg (Ogg bitstream format), Guidelines Support Library (GSL), utf8proc (Unicode normalization and UTF-8 operations), and UTF8-CPP (portable handling of UTF-8 encoded Unicode strings).

Tokens
34.4K
Snippets
75
Records
179
Agent score
80%

What's inside rhubarb-lip-sync

  2. Introduction to UTF8-CPP

    master
    UTF8-CPP is a small, C++98 compatible generic library designed for portable handling of UTF-8 encoded Unicode strings. It is designed to work naturally with STL algorithms and iterators. The library is feature-complete and provides tools for validating UTF-8, calculating Unicode code point distances, and converting between UTF-8 and UTF-16 encodings.

  3. Overview of Vorbis libraries

    master

    Vorbis is a public domain, general-purpose audio and music encoding format. The package provides several libraries for different use cases:

    • libvorbis: The core BSD-style implementation of the Vorbis specification.
    • libvorbisfile: A convenience library built on top of Vorbis designed to simplify common audio file operations.
    • libvorbisenc: A library providing a simple, programmatic interface for setting up audio encoding.

    Note: You must have libogg installed to compile these libraries, as it is a required dependency.

  4. Use utf8proc for Unicode normalization and operations

    master

    utf8proc is a C library providing Unicode normalization, case-folding, and other UTF-8 operations.

    After successful compilation, the library is available as libutf8proc.a (static) or libutf8proc.so (dynamic). It supports Unicode version 11.0.0.

    For Unicode normalizations, use the following option combinations:

    • Normalization Form C: STABLE, COMPOSE
    • Normalization Form D: STABLE, DECOMPOSE
    • Normalization Form KC: STABLE, COMPOSE, COMPAT
    • Normalization Form KD: STABLE, DECOMPOSE, COMPAT

    For most string mapping tasks, use the utf8proc_map function unless you prefer to manage memory allocation manually. Detailed API documentation is located in the utf8proc.h header file.

  5. Understand Rhubarb Lip Sync mouth shapes

    master

    Rhubarb Lip Sync uses a set of mouth positions to represent speech. You can use six basic shapes or include up to three optional extended shapes.

    Basic Mouth Shapes (Required)

    • A: Closed mouth for "P", "B", and "M" sounds.
    • B: Slightly open with clenched teeth (most consonants like "K", "S", "T", and the "EE" vowel).
    • C: Open mouth for vowels like "EH" (men) or "AE" (bat).
    • D: Wide open mouth for vowels like "AA" (father).
    • E: Slightly rounded mouth for vowels like "AO" (off) or "ER" (bird).
    • F: Puckered lips for "UW" (you), "OW" (show), or "W" (way).

    Extended Mouth Shapes (Optional)

    • G: Upper teeth touching lower lip (for "F" or "V" sounds).
    • H: Long "L" sounds (tongue raised behind upper teeth).
    • X: Idle position (closed but relaxed lips, used for pauses in speech).

  6. Handle formatting errors and type safety

    master

    C++ Format is fully type-safe and manages memory automatically to prevent buffer overflows.

    Runtime Errors

    If a format string is incompatible with the provided arguments (e.g., using an integer format code for a string), the library throws a FormatError exception.

    Compile-time Errors

    Where possible, the library catches errors at compile time. For example, attempting to format a wide character into a narrow string will trigger a compiler error. In such cases, use a wide format string (e.g., L"...") instead.

  7. Understand Format String Syntax

    master

    The fmt::format() and fmt::print() functions use a replacement field syntax based on curly braces {}.

    Key Rules:

    • Replacement Fields: Use {} to indicate where an argument's value should be inserted.
    • Literal Braces: To include a literal { or } in your text, escape them by doubling them: {{ or }}.
    • Argument IDs:
      • You can specify an argument by index (e.g., {0}, {1}) or by identifier.
      • If you omit the ID (e.g., {}), the library automatically uses the next available index in sequence (0, 1, 2...).
      • If you omit IDs, you must omit all of them in the string to use implicit indexing.
    • Format Specifications: You can add a colon : followed by a format_spec inside the braces to control width, alignment, padding, and precision (e.g., {:>10}).
    • Nested Fields: You can nest replacement fields inside a format_spec to dynamically specify formatting parameters, provided the nested fields only contain an argument index.
    "First, thou shalt count to {0}" // References the first argument
"Bring me a {}"                  // Implicitly references the first argument
"From {} to {}"                  // Same as "From {0} to {1}"

  8. Format Specification Mini-Language Syntax

    master

    The cppformat library uses a format syntax similar to printf, but replaces the % character with {} and uses : to separate arguments from format specifiers. For example, "%03.2f" is written as "{:03.2f}".

    Key syntax features include:

    • Positional Arguments: Access arguments by index using {0}, {1}, etc. Indices can be repeated.
    • Alignment and Width:
      • {:<N}: Left align with width N.
      • {:>N}: Right align with width N.
      • {:^N}: Center align with width N.
      • {:*^N}: Center align with width N using * as a fill character.
    • Sign Specification:
      • {:+f}: Always show the sign (both + and -).
      • {: f}: Show a space for positive numbers and a sign for negative numbers.
      • {:f} or {: -f}: Show only the minus sign for negative numbers.
    • Base Conversion:
      • {0:d}: Decimal.
      • {0:x}: Hexadecimal.
      • {0:o}: Octal.
      • {0:b}: Binary.
      • Use the # prefix (e.g., {0:#x}) to include type-specific prefixes like 0x, 0, or 0b.
    • Special Formatting:
      • {:,}: Use a comma as a thousands separator.
      • {:.2%}: Express a value as a percentage with specified precision.
      • {:%Y-%m-%d}: Type-specific formatting (e.g., for datetime objects).

  9. Portability and Platform Consistency

    master

    C++ Format is highly portable and works across various operating systems and compilers, including:

    • Linux (64-bit/32-bit): GCC 4.4.3+, Intel C++ Compiler (ICC) 14.0.2
    • Mac OS X: GCC 4.2.1, Clang 4.2, 5.1.0
    • Windows (64-bit/32-bit): Visual C++ 2010, 2013

    One major benefit is consistent output across platforms. For example, formatting a floating-point infinity always results in inf, whereas printf behavior can vary by platform.

  10. Use utf8::iterator for STL-compatible UTF-8 iteration

    master

    The utf8::iterator adapter allows you to iterate over UTF-8 encoded strings using standard STL algorithms. It works by wrapping an octet iterator (like char* or std::string::iterator).

    Important Safety Note: utf8::iterator is a checked iterator. It is bound to the range specified in its constructor. Any attempt to increment or decrement outside this range, or to compare iterators constructed from different ranges, will result in an exception. Typically, you should initialize it using the begin() and end() methods of a sequence container.

    std::string s = "example";
utf8::iterator i (s.begin(), s.begin(), s.end());

  11. Configure Sign and Alternate Forms for Numbers

    master

    The sign and # options are used to control the presentation of numeric types.

    Sign Options:

    OptionMeaning
    +Always show a sign for both positive and negative numbers.
    -Show a sign only for negative numbers (default).
    (space)Use a leading space for positive numbers and a minus sign for negative numbers.

    Alternate Form (#): This option changes the conversion behavior based on the type:

    • Integers: For binary (b, B), octal (o), or hex (x, X), it adds the appropriate prefix (0b, 0, 0x, or 0X).
    • Floating-point: Ensures a decimal point is always present, even if no digits follow it. For g/G types, it prevents the removal of trailing zeros.

  12. Create a mouth shape composition for Rhubarb

    master

    Before running the script, you must create a composition in After Effects that contains your mouth shapes. The shapes must be placed on specific frames in a sequence:

    FrameMouth Shape
    00:00{A}
    00:01{B}
    00:02{C}
    00:03{D}
    00:04{E}
    00:05{F}
    00:06{G} (optional)
    00:07{H} (optional)
    00:08{X} (optional)

    For a reference implementation, check the extras/AdobeAfterEffects/demo directory.