gosax

repository·main·Indexed 41 days ago

https://github.com/orisano/gosax

A high-performance, memory-efficient Go library for read-only XML SAX (Simple API for XML) parsing. It utilizes SWAR (SIMD Within A Register) optimizations to stream XML documents without loading them entirely into memory. The library includes the xmlb extension for high-performance decoding and provides compatibility utilities to migrate from the standard encoding/xml package.

Tokens
2.4K
Snippets
12
Records
14
Agent score
51%

What's inside gosax

  1. Migrate from encoding/xml to gosax

    main

    When moving from the standard library encoding/xml to gosax, consider the following adjustments:

    Handling Self-Closing Tags

    To mimic the behavior of encoding/xml, set gosax.Reader.EmitSelfClosingTag to true. This ensures self-closing tags are recognized and processed correctly.

    Using TokenE for Interface Compatibility

    If your existing code relies on the xml.Token interface, you can use gosax.TokenE to wrap the events returned by dec.Event(). Note that using TokenE and Token involves memory allocation due to the use of interfaces.

    Using xmlb for High Performance

    For a more performant bridge that simplifies rewriting code, use the xmlb extension. xmlb.Decoder provides a higher-performance way to process XML tokens compared to the standard library or the interface-based TokenE approach.

  2. Initialize a gosax Reader

    main

    To parse XML, create a Reader using one of the following constructors:

    • NewReader(r io.Reader): Creates a reader with a default buffer size of 2MB.
    • NewReaderSize(r io.Reader, bufSize int): Creates a reader with a custom buffer size.
    • NewReaderBuf(r io.Reader, buf []byte): Creates a reader using a pre-allocated byte slice to minimize allocations.

    You can also reuse an existing Reader by calling Reset(reader io.Reader).

    import "github.com/orisano/orisano/gosax"
    import "os"
    
    file, _ := os.Open("data.xml")
    reader := gosax.NewReader(file)
  3. Use xmlb for high-performance XML decoding

    main

    The xmlb extension provides a high-performance bridge for XML parsing. Instead of using type assertions on interfaces, you use tok.Type() to switch between event types like xmlb.StartElement, xmlb.CharData, and xmlb.EndElement, and then use specific methods to retrieve the data.

    var dec *xmlb.Decoder
    for {
    	tok, err := dec.Token()
    	if err == io.EOF {
    		break
    	}
    	switch tok.Type() {
    	case xmlb.StartElement:
    		t, _ := tok.StartElement()
    		// ...
    	case xmlb.CharData:
    		t, _ := tok.CharData()
    		// ...
    	case xmlb.EndElement:
    		t := tok.EndElement()
    		// ...
    	}
    }
  4. Basic usage of gosax Reader

    main

    To parse an XML document, create a new reader using gosax.NewReader(reader). You can then iterate through the XML stream by calling r.Event() in a loop. Check for gosax.EventEOF to terminate the loop. Each event provides access to the underlying data via the Bytes field.

    package main
    
    import (
    	"fmt"
    	"log"
    	"strings"
    
    	"github.com/orisano/gosax"
    )
    
    func main() {
    	xmlData := `<root><element>Value</element></root>`
    	reader := strings.NewReader(xmlData)
    
    	r := gosax.NewReader(reader)
    	for {
    		e, err := r.Event()
    		if err != nil {
    			log.Fatal(err)
    		}
    		if e.Type() == gosax.EventEOF {
    			break
    		}
    		fmt.Println(string(e.Bytes))
    	}
    }
  5. Configure Self-Closing Tag Emission

    main

    By default, gosax treats self-closing tags (e.g., <tag/>) as a EventStart followed by an EventEnd. If you enable EmitSelfClosingTag on the Reader, the parser will emit an EventStart and then immediately an EventEnd for that same tag, effectively simulating the self-closing behavior in a way that might be easier for certain state machines to handle.

    reader := gosax.NewReader(file)
    reader.EmitSelfClosingTag = true
  6. Convert gosax bytes to specific encoding/xml types

    main

    If you need fine-grained control over memory or specific XML types, use these direct conversion utilities to transform []byte slices into standard encoding/xml types:

    FunctionReturnsDescription
    StartElement(b []byte)xml.StartElement, errorConverts bytes to a start element, including attributes. Returns syntax error if an attribute has no value.
    EndElement(b []byte)xml.EndElementConverts bytes to an end element.
    CharData(b []byte)xml.CharData, errorConverts bytes to character data (unescapes the content).
    Comment(b []byte)xml.CommentConverts bytes to an XML comment (removes <!-- and -->).
    ProcInst(b []byte)xml.ProcInstConverts bytes to a processing instruction.
    Directive(b []byte)xml.DirectiveConverts bytes to an XML directive (removes <! and >).
  7. Decode XML entities with Unescape()

    main

    The Unescape(b []byte) function decodes XML entity references (like &lt;, &amp;, or numeric references like &#x3c;) within a byte slice. It performs the unescaping in-place where possible to be efficient.

    input := []byte("Hello &lt;World&gt;")
    unescaped, err := gosax.Unescape(input)
    // unescaped is []byte("Hello <World>")
  8. Extract Tag Names with Name()

    main

    The Name(b []byte) function extracts the tag name from a raw XML tag byte slice. It handles stripping characters like <, /, and > and splits the name from any subsequent attributes or whitespace.

    // If event.Bytes is "<tag attr='val'>"
    name, remaining := gosax.Name(event.Bytes)
    // name is "tag"
    // remaining is " attr='val'>"
  9. Parse XML using the Event() method

    main

    The primary way to consume the XML stream is by calling Event(). This method returns the next Event in the stream and an error.

    CRITICAL: Memory Management The Event.Bytes slice is only valid until the next call to Event(). The underlying memory may be overwritten by the parser to maintain high performance. If you need to use the data outside the immediate loop iteration, you must make a copy of the bytes.

    for {
        event, err := reader.Event()
        if err == io.EOF {
            break
        }
        if err != nil {
            log.Fatal(err)
        }
    
        switch event.Type() {
        case gosax.EventStart:
            // Process start tag
            // IMPORTANT: Copy event.Bytes if you need it later
            tagBytes := make([]byte, len(event.Bytes))
            copy(tagBytes, event.Bytes)
        case gosax.EventText:
            // Process text content
        }
    }
  10. Skip nested XML scopes with Skip()

    main

    The Skip function advances a *gosax.Reader to the end of the current nested scope. It tracks the depth of EventStart and EventEnd events and returns when the nesting level returns to zero. This is useful for jumping over entire sub-trees of XML content.

    err := gosax.Skip(reader)
    if err != nil {
        // Handle error (e.g., EOF or syntax error)
    }
  11. Extract Attributes with NextAttribute()

    main

    The NextAttribute(b []byte) function parses attributes from a byte slice (typically the remaining slice returned by Name()). It returns an Attribute struct containing the Key and Value as byte slices, and the remaining bytes for further parsing.

    // Assuming 'remaining' is the slice after the tag name
    attr, rest, err := gosax.NextAttribute(remaining)
    if err == nil {
        fmt.Printf("Key: %s, Value: %s\n", attr.Key, attr.Value)
    }