Siegfried

repository·main·Indexed 42 days ago

https://github.com/richardlehane/siegfried

A signature-based file format identification tool that implements PRONOM, MIME-info, FDD, and Wikidata signatures. It provides a CLI tool (sf) for scanning files and directories, as well as a WASM API for integration into web projects.

Tokens: 2.6K
Snippets: 6
Records: 9
Agent score: 5%

What's inside Siegfried

Configure Logging and Output
main
Siegfried provides granular control over logging and output formats.
Output Formats
- -csv: Output results in CSV format.
- -json: Output results in JSON format.
- -droid: Output results in DROID CSV format.
Logging Control
Use the -log flag to control what is logged and where:
- -log e,w: Log errors (e) and warnings (w) to stderr.
- -log u,o: Log unknowns (u) to stdout (o).
- -log d,s: Log debugging (d) and slow messages (s) to stderr.
- -log p,t: Log progress (p) and time (t) while redirecting results to a file.
- -log fmt/1,c: Log instances of fmt/1 and chart results.
https://github.com/richardlehane/siegfried/blob/main/README.md

Install Siegfried

main

Siegfried can be installed using several methods depending on your operating system and whether you have Go installed.

Using Go

If you have Go installed and your go/bin folder is in your PATH:

go install github.com/richardlehane/siegfried/cmd/sf@latest
sf -update

Without Go

Windows: Download a pre-built binary from the releases page and add it to your system path.

macOS (Homebrew):

brew install mistydemeo/digipres/siegfried

Or for the most recent updates:

brew install richardlehane/digipres/siegfried

Ubuntu/Debian (64 bit):

curl -sL "http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x20F802FE798E6857" | gpg --dearmor | sudo tee /usr/share/keyrings/siegfried-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/siegfried-archive-keyring.gpg] https://www.itforarchivists.com/ buster main" | sudo tee -a /etc/apt/sources.list.d/siegfried.list
sudo apt-get update && sudo apt-get install siegfried

FreeBSD:
```
pkg install siegfried
```

Arch Linux:

git clone https://aur.archlinux.org/siegfried.git
cd siegfried
makepkg -si

go install github.com/richardlehane/siegfried/cmd/sf@latest

Install Siegfried WASM in a web project

main

To use Siegfried in a web application, you must include both wasm_exec.js and sf.wasm in your project. Use the following pattern to initialize the Go runtime and instantiate the WebAssembly module:

<script src="wasm_exec.js"></script>
<script>
    const go = new window.Go();

    WebAssembly.instantiateStreaming(
        fetch("sf.wasm"),
        go.importObject
    ).then(
        (obj) => {
            go.run(obj.instance);
        }
    );
</script>

Once the module is loaded, the identify method becomes available on the global scope.

<script src="wasm_exec.js"></script>
<script>
    const go = new window.Go();

    WebAssembly.instantiateStreaming(
        fetch("sf.wasm"),
        go.importObject
    ).then(
        (obj) => {
            go.run(obj.instance);
        }
    );
</script>

Use the Siegfried CLI
main
Siegfried is a signature-based file format identification tool. You can scan individual files, glob patterns, or entire directories.
Basic Usage
```
sf file.ext      # Scan a single file
sf *.ext        # Scan files matching a pattern
sf DIR           # Scan a directory
```
Common CLI Options
- -csv, -json, -droid: Change output format (CSV, JSON, or DROID CSV instead of the default YAML).
- -nr: Do not scan subdirectories.
- -z: Decompress and scan archives (zip, tar, gzip, warc, arc).
- -zs <types>: Selectively decompress and scan specific types (e.g., sf -zs gzip,tar file.tar.gz).
- -hash <type>: Calculate a hash (md5, sha1, sha256, sha512, or crc).
- -sig <file>: Use a custom signature file.
- -f <file>: Scan a list of files and directories provided in a text file.
- -multi <n>: Scan n files in parallel.
- -serve <host:port>: Run in server mode.
- -update: Download the latest signatures.
```
sf -csv file.ext
sf -z file.zip
sf -hash sha256 file.ext
```
https://github.com/richardlehane/siegfried/blob/main/README.md

Run the Siegfried WASM example

main

To run the provided example, download the latest sf.wasm from the releases page and place it in the example folder.

Note: You cannot run the example by opening index.html directly in a browser. It must be served via a web server. You can use the following Go script to host the directory:

package main

import (
    "flag"
    "log"
    "net/http"
)

func main() {
    port := flag.String("p", "8100", "port to serve on")
    directory := flag.String("d", ".", "the directory of static file to host")
    flag.Parse()
    http.Handle("/", http.FileServer(http.Dir(*directory)))
    log.Printf("Serving %s on HTTP port: %s\n", *directory, *port)
    log.Fatal(http.ListenAndServe(":"+*port, nil))
}

To run this server:

Save the code to main.go.
Run go build main.go.
Execute the resulting binary.

package main

import (
    "flag"
    "log"
    "net/http"
)

func main() {
    port := flag.String("p", "8100", "port to serve on")
    directory := flag.String("d", ".", "the directory of static file to host")
    flag.Parse()
    http.Handle("/", http.FileServer(http.Dir(*directory)))
    log.Printf("Serving %s on HTTP port: %s\n", *directory, *port)
    log.Fatal(http.ListenAndServe(":"+*port, nil))
}

Build the Siegfried WASM file
main
To compile the WebAssembly module yourself, use the following Go build command:
```
GOOS=js GOARCH=wasm go build -o sf.wasm github.com/richardlehane/siegfried/wasm
```
Signatures are embedded in the sf.wasm file using the static package. To customize these signatures, edit the gen.go file in the pkg/static package (ensuring the path in the first line of the main function is correct) and then run go generate.
```
GOOS=js GOARCH=wasm go build -o sf.wasm github.com/richardlehane/siegfried/wasm
```
https://github.com/richardlehane/siegfried
Configure Siegfried WASM options
main
When calling identify(FileSystemHandle, ...options), you can pass specific string flags to modify the behavior of the identification process:
- Output Format: Use `
https://github.com/richardlehane/siegfried

Reference: Siegfried CLI Options

main

The following flags are available for the sf command line tool:

sf -csv file.ext | *.ext | DIR             // Output CSV rather than YAML
sf -json file.ext | *.ext | DIR            // Output JSON rather than YAML
sf -droid file.ext | *.ext | DIR           // Output DROID CSV rather than YAML
sf -nr DIR                                 // Don't scan subdirectories
sf -z file.zip | *.ext | DIR               // Decompress and scan zip, tar, gzip, warc, arc
sf -zs gzip,tar file.tar.gz | *.ext | DIR  // Selectively decompress and scan 
sf -hash md5 file.ext | *.ext | DIR        // Calculate md5, sha1, sha256, sha512, or crc hash
sf -sig custom.sig *.ext | DIR             // Use a custom signature file
sf -                                       // Scan stream piped to stdin
sf -name file.ext -                        // Provide filename when scanning stream 
sf -f myfiles.txt                          // Scan list of files and directories
sf -v | -version                           // Display version information
sf -home c:\junk -sig custom.sig file.ext  // Use a custom home directory
sf -serve hostname:port                    // Server mode
sf -throttle 10ms DIR                       // Pause for duration (e.g. 1s) between file scans
sf -multi 256 DIR                          // Scan multiple (e.g. 256) files in parallel 
sf -log [comma-sep opts] file.ext          // Log errors etc. to stderr (default) or stdout
sf -log e,w file.ext | *.ext | DIR         // Log errors and warnings to stderr
sf -log u,o file.ext | *.ext | DIR         // Log unknowns to stdout
sf -log d,s file.ext | *.ext | DIR         // Log debugging and slow messages to stderr
sf -log p,t DIR > results.yaml             // Log progress and time while redirecting results
sf -log fmt/1,c DIR > results.yaml         // Log instances of fmt/1 and chart results
sf -replay -log u -csv results.yaml        // Replay results file, convert to csv, log unknowns
sf -setconf -multi 32 -hash sha1           // Save flag defaults in a config file
sf -setconf -serve :5138 -conf srv.conf    // Save/load named config file with '-conf filename'

Use the Siegfried WASM API
main
The identify function allows you to identify file types using a FileSystemHandle. You can pass either a file handle or a directory handle; if a directory is provided, Siegfried will recursively scan all files and subdirectories within it.
Function Signature
identify(FileSystemHandle, ...options) returns Promise(String)
Options
Options are passed as strings in any order:
- Output Format: `
https://github.com/richardlehane/siegfried

Siegfried

What's inside Siegfried

Output Formats

Logging Control

Using Go

Without Go

Basic Usage

Common CLI Options

Function Signature

Options