Standard Ebooks Toolset

repository·master·Indexed 23 days ago

https://github.com/standardebooks/tools

A collection of Python-based tools used by Standard Ebooks for ebook production, including ebook setup, text processing, and build automation. Features the `se` command-line interface for tasks such as building ebooks (`se build`), creating drafts (`se create-draft`), linting for style errors (`se lint`), applying typography and semantics rules, and performing interactive search and replace.

Tokens
15.7K
Snippets
30
Records
118
Agent score
79%

What's inside standardebooks-tools

  1. Understand the Standard Ebooks source directory structure

    master

    Many tools in this repository operate on Standard Ebooks source directories. These directories follow a consistent minimal structure:

    • ./images/: Contains source images for the cover, titlepages, and illustrations. Source images should be at maximum resolution; resized/compressed/SVG versions are placed in ./src/epub/images/ for distribution.
    • ./src/epub/: Contains the files that constitute the final distributable EPUB.
      • css/: Contains core.css, local.css, and se.css.
      • images/: Contains distribution-ready assets like cover.svg, logo.svg, and titlepage.svg.
      • text/: Contains XHTML files such as colophon.xhtml, imprint.xhtml, titlepage.xhtml, and uncopyright.xhtml.
      • content.opf: The package document.
      • toc.xhtml: The table of contents.
      • META-INF/container.xml: EPUB container information.
      • mimetype: The EPUB mimetype file.
    • LICENSE.md: The project license.
    .
    |__ images/
    |   |__ cover.jpg
    |   |__ cover.source.jpg
    |   |__ cover.svg
    |   |__ titlepage.svg
    |
    |__ src/
    |   |__ META-INF/
    |   |   |__ container.xml
    |   |
    |   |__ epub/
    |   |   |__ css/
    |   |   |   |__ core.css
    |   |   |   |__ local.css
    |   |   |   |__ se.css
    |   |   |
    |   |   |__ images/
    |   |   |   |__ cover.svg
    |   |   |   |__ logo.svg
    |   |   |   |__ titlepage.svg
    |   |   |
    |   |   |__ text/
    |   |   |   |__ colophon.xhtml
    |   |   |   |__ imprint.xhtml
    |   |   |   |__ titlepage.xhtml
    |   |   |   |__ uncopyright.xhtml
    |   |   |
    |   |   |__ content.opf
    |   |   |__ toc.xhtml
    |   |
    |   |__ mimetype
    |
    |__ LICENSE.md
  2. Install the Standard Ebooks toolset on Ubuntu

    master

    To install the toolset on Ubuntu, you must have Python >= 3.10.12. The installation process differs slightly between Ubuntu 24.04 (Noble) and 20.04 (Focal).

    Ubuntu 24.04 (Noble)

    Install pre-flight dependencies and the toolset using pipx:

    # Install some pre-flight dependencies.
    sudo apt install --yes default-jre git python3-dev python3-pip python3-venv pipx
    
    # Install the toolset.
    pipx install standardebooks

    Ubuntu 20.04 (Focal)

    Install pre-flight dependencies, then install and configure pipx before installing the toolset:

    # Install some pre-flight dependencies.
    sudo apt install --yes default-jre git python3-dev python3-pip python3-venv
    
    # Install pipx.
    python3 -m pip install --user pipx
    python3 -m pipx ensurepath
    
    # Install the toolset.
    pipx install --python=3.12 --fetch-missing-python standardebooks
    # Ubuntu 24.04 (Noble)
    sudo apt install --yes default-jre git python3-dev python3-pip python3-venv pipx
    pipx install standardebooks
  3. Install the Standard Ebooks toolset on Fedora 43

    master

    Install the required dependencies via dnf, ensure your pipx path is configured, and then install the toolset.

    # Install some pre-flight dependencies.
    sudo dnf install pipx python3-devel gcc libxslt-devel git java-25-openjdk-headless
    
    # Ensure $PATH environment variable is correctly set up for pipx.
    pipx ensurepath
    
    # Install the toolset.
    pipx install standardebooks
  4. Run the Nu Html Checker as a standalone web server

    master

    You can run the checker as a self-contained HTTP service using a built-in Jetty server. This allows for browser-based checking of HTML, CSS, and SVG files.

    To start the server, invoke the nu.validator.servlet.Main class with a port number. By default, it listens on all interfaces (0.0.0.0), but future releases may default to 127.0.0.1. To ensure accessibility on a specific network interface, use the nu.validator.servlet.bind-address system property.

    If you encounter a StackOverflowError, increase the thread stack size using the -Xss Java option.

  5. Install the Standard Ebooks toolset on Windows 11

    master

    Install dependencies via WinGet in PowerShell, then use pipx to install the toolset.

    1. Install dependencies:
      winget install --exact --id Python.Python.3.12
      winget install --exact --id Git.Git
      winget install --exact --id Microsoft.OpenJDK.21
      winget install --exact --id tschoonj.GTKForWindows
    2. Restart PowerShell, then install and configure pipx:
      py -3.12 -m pip install --user pipx
      py -3.12 -m pipx ensurepath
    3. Restart PowerShell and install the toolset:
      pipx install --python 3.12 standardebooks
    winget install --exact --id Python.Python.3.12
    winget install --exact --id Git.Git
    winget install --exact --id Microsoft.OpenJDK.21
    winget install --exact --id tschoonj.GTKForWindows
  6. Build and run the Nu Html Checker Web UI

    master

    To build, test, and run the Nu Html Checker so that the Web UI is accessible at http://0.0.0.0:8888/, follow these steps:

    1. Ensure git, python, and JDK 8 or above are installed.
    2. Set the JAVA_HOME environment variable:
      • Ubuntu/Linux: export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 (adjust path as needed).
      • MacOS: export JAVA_HOME=$(/usr/libexec/java_home)
    3. Clone the repository: git clone https://github.com/validator/validator.git
    4. Enter the directory: cd validator
    5. Run the checker script: python ./checker.py all

    Note: The first run requires an internet connection to download several megabytes of dependencies.

    python ./checker.py all
  7. Install the Standard Ebooks toolset on macOS

    master

    The recommended method is using Homebrew. Alternatively, you can use pipx for a more granular installation.

    Using Homebrew

    1. Update Homebrew:
      brew update
    2. Install the tools:
      brew install standardebooks

    Using pipx (Alternative)

    Install dependencies manually and then use pipx:

    # Install some pre-flight dependencies.
    brew install cairo git openjdk pipx python@3.12
    pipx ensurepath
    sudo ln -sfn $(brew --prefix)/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk
    
    # Install the toolset.
    pipx install --python "$(brew --prefix)"/bin/python3.12 standardebooks

    Note on SVG rendering: If you are building projects with SVG images and do not have Chrome or Firefox installed, you must manually enable Safari:

    safaridriver --enable
    brew update
    brew install standardebooks
  8. Install the toolset in editable mode for development

    master

    If you are developing the toolset, install it in "editable" mode using pipx. This allows changes to the source code to take effect immediately without re-installation.

    git clone https://github.com/standardebooks/tools.git
    pipx install --editable ./tools

    After installation, the se binary will be available in your path, reflecting any edits made to the files in the tools/ directory.

  9. Install shell completions for the `se` command

    master

    You can install shell completions for ZSH, Bash, and Fish. These commands assume you have installed the toolset via pipx.

    ZSH Completions

    sudo ln --symbolic $(pipx environment --value PIPX_LOCAL_VENVS)/standardebooks/lib/python3.*/site-packages/se/completions/zsh/_se /usr/share/zsh/vendor-completions/_se && hash -rf && compinit

    Bash Completions

    mkdir --parents $HOME/.local/share/bash-completion/completions/
    ln --symbolic $(pipx environment --value PIPX_LOCAL_VENVS)/standardebooks/lib/python3.*/site-packages/se/completions/bash/se $HOME/.local/share/bash-completion/completions/se

    Fish Completions

    ln --symbolic $(pipx environment --value PIPX_LOCAL_VENVS)/standardebooks/lib/python3.*/site-packages/se/completions/fish/se.fish $HOME/.config/fish/completions/
    # Example: Bash completions
    mkdir --parents $HOME/.local/share/bash-completion/completions/
    ln --symbolic $(pipx environment --value PIPX_LOCAL_VENVS)/standardebooks/lib/python3.*/site-packages/se/completions/bash/se $HOME/.local/share/bash-completion/completions/se
  10. Deploy the Nu Html Checker to a servlet container

    master

    To run the checker within an existing servlet container like Apache Tomcat, deploy the vnu.war file. You can do this via the Tomcat Manager application or by copying the file into the webapps directory.

    Once deployed, the application is typically accessible at http://localhost/[context-root]/, where the context root is usually the filename (e.g., http://localhost/vnu/).

    To customize servlet filter configuration (such as disabling the inbound-size-limit-filter), you can modify the /WEB-INF/web.xml file inside the WAR archive using a ZIP-handling program.

  11. Run the Nu Html Checker via Docker

    master

    The checker is available on Docker Hub as validator/validator. You can run it as a container, specifying ports, environment variables for timeouts, or Java system properties via JAVA_TOOL_OPTIONS.

    To use docker compose, define a service using the validator/validator image and map the necessary ports.

    # Run the latest version
    docker run -it --rm -p 8888:8888 validator/validator:latest
    
    # Run a specific version
    docker run -it --rm -p 8888:8888 validator/validator:17.11.1
    
    # Bind to a specific host address
    docker run -it --rm -p 128.30.52.73:8888:8888 validator/validator:latest
    
    # Set connection and socket timeouts via environment variables
    docker run -it --rm \
       -e CONNECTION_TIMEOUT_SECONDS=15 \
       -e SOCKET_TIMEOUT_SECONDS=15 \
       -p 8888:8888 \
       validator/validator
    
    # Set Java system properties via JAVA_TOOL_OPTIONS
    docker run -it --rm \
       -e JAVA_TOOL_OPTIONS="-Dnu.validator.client.asciiquotes=yes" \
       -p 8888:8888 \
       validator/validator

    Docker Compose Example:

    version: '2'
    services:
      vnu:
        image: validator/validator
        ports:
          - "8888:8888"
        network_mode: "host"