Install xmlquery
masterInstall the xmlquery package using go get.
$ go get github.com/antchfx/xmlqueryrepository·master·Indexed 19 days ago
https://github.com/antchfx/xmlqueryA Go package for querying XML documents using XPath expressions. It supports parsing XML from URLs, strings, and io.Readers, and provides a StreamParser for memory-efficient processing of large files. The library allows for node manipulation, attribute management, and serialization of XML trees back to strings or writers.
Install the xmlquery package using go get.
$ go get github.com/antchfx/xmlqueryThe xmlquery library represents XML documents as a tree of Node objects. Each node has a NodeType that defines its role in the document structure. Common node types include:
DocumentNode: The root of the document tree.ElementNode: A standard XML element (e.g., <item>).TextNode: The text content within an element.CharDataNode: Content wrapped in <![CDATA[...]]>.CommentNode: XML comments (e.g., <!-- comment -->).AttributeNode: An attribute of an element.DeclarationNode: Document type declarations (e.g., <!DOCTYPE...>).ProcessingInstruction: XML processing instructions (e.g., <?target instruction?>).NotationNode: A directive in the document (e.g., <!text...>).The StreamParser allows you to process large XML documents in a streaming fashion, which prevents high memory consumption by only keeping the target nodes in memory.
To use it, you must provide a streamElementXPath that points to the elements you want to extract. You can optionally provide a streamElementFilter for more complex logic (e.g., filtering elements based on their content or attributes).
Key behaviors:
Read() returns the next *Node that matches your criteria.Read() is called, the previous target node is automatically removed from the document tree to free up memory.io.EOF is returned, indicating no more matching nodes were found.// Scenario 1: Simple streaming
// xml := `<AAA><BBB>b1</BBB><BBB>b2</BBB></AAA>`
sp, err := xmlquery.CreateStreamParser(strings.NewReader(xml), "/AAA/BBB")
for {
n, err := sp.Read()
if err == io.EOF {
break
}
if err != nil {
panic(err)
}
fmt.Println(n.OutputXML(true))
}
// Scenario 2: Advanced filtering
// xml := `<AAA><BBB>b1</BBB><BBB>b2</BBB></AAA>`
// We only want BBB elements where the text is NOT 'b1'
sp, err := xmlquery.CreateStreamParser(strings.NewReader(xml), "/AAA/BBB", "/AAA/BBB[. != 'b1']")
for {
n, err := sp.Read()
if err == io.EOF {
break
}
// ... handle n
}Once you have a node, you can extract its content or attributes:
n.InnerText(): Returns the text content of the node.n.SelectElement(name): Finds a child element by name.//book/@id), then access the InnerText property of the resulting node.// Get text from a child element
if n := channel.SelectElement("title"); n != nil {
fmt.Printf("title: %s\n", n.InnerText())
}
// Get attribute value via XPath
list := xmlquery.Find(doc, "//book/@id")
fmt.Println(list[0].InnerText) // outputs the @id valueTo parse UTF-16 XML files, use xmlquery.ParseWithOptions() and provide a ParserOptions object containing a CharsetReader in the DecoderOptions.
// Example: Converting UTF-16 to UTF-8 for parsing
// (Requires unicode and transform packages)
options := xmlquery.ParserOptions{
Decoder: &xmlquery.DecoderOptions{
CharsetReader: func(charset string, input io.Reader) (io.Reader, error) {
return input, nil
},
},
}
doc, err := xmlquery.ParseWithOptions(utf8Reader, options)Use the following functions to locate nodes in an XML document:
xmlquery.Find(doc, xpath): Returns a slice of matching nodes. Note: This function panics if the XPath query is invalid.xmlquery.FindOne(doc, xpath): Returns the first matching node.xmlquery.QueryAll(doc, xpath): Returns a slice of matching nodes. Note: This returns an error instead of panicking if the XPath query is invalid.xmlquery.QuerySelector(doc, expr): Uses a pre-compiled xpath.Expr to find a node (useful for performance).// Find all matching nodes
list, err := xmlquery.QueryAll(doc, "a")
// Find the first matching node
book := xmlquery.FindOne(doc, "//book[2]")
// Find all matching nodes (panics on invalid XPath)
list := xmlquery.Find(doc, "//author")Use the following methods to parse XML depending on your source:
xmlquery.LoadURL(url).xmlquery.Parse(strings.NewReader(s)).io.Reader: Use xmlquery.Parse(reader).os.Open and pass it to xmlquery.Parse(f).// Parse from URL
doc, err := xmlquery.LoadURL("http://www.example.com/sitemap.xml")
// Parse from string
s := `<?xml version="1.0" encoding="utf-8"?><rss version="2.0"></rss>`
doc, err := xmlquery.Parse(strings.NewReader(s))
// Parse from io.Reader (e.g., a file)
f, err := os.Open("../books.xml")
doc, err := xmlquery.Parse(f)To evaluate XPath expressions that return values (like sum() or count()) rather than nodes, you must use the xpath package to compile the expression and then evaluate it using an XPathNavigator created from your xmlquery document.
import (
"github.com/antchfx/xmlquery"
"github.com/antchfx/xpath"
)
// Evaluate total price of all books
expr, err := xpath.Compile("sum(//book/price)")
price := expr.Evaluate(xmlquery.CreateXPathNavigator(doc)).(float64)
fmt.Printf("total price: %f\n", price)
// Count the number of books
expr, err := xpath.Compile("count(//book)")
count := expr.Evaluate(xmlquery.CreateXPathNavigator(doc)).(float64)To save memory when processing large XML files, use xmlquery.CreateStreamParser. This allows you to iterate through specific elements one by one rather than loading the entire document into memory.
CreateStreamParser accepts the source reader and one or more XPath expressions to filter the elements you want to stream.
// Simple stream parsing
f, _ := os.Open("../books.xml")
p, err := xmlquery.CreateStreamParser(f, "/bookstore/book")
for {
n, err := p.Read()
if err == io.EOF {
break
}
if err != nil {
panic(err)
}
fmt.Println(n)
}
// Advanced stream parsing with element filtering
p, err := xmlquery.CreateStreamParser(f, "/bookstore/book", "/bookstore/book[price>=10]")When working with XML that uses namespaces, you can compile XPath expressions with a namespace map to allow querying with custom prefixes.
nsMap := map[string]string{
"q": "http://xmlns.xyz.com/process/2003",
"r": "http://www.w3.org/1999/XSL/Transform",
"s": "http://www.w3.org/2001/XMLSchema",
}
expr, _ := xpath.CompileWithNS("//q:activity", nsMap)
node := xmlquery.QuerySelector(doc, expr)When using OutputXMLWithOptions or WriteWithOptions, you can pass OutputOption functions to customize the serialization:
WithOutputSelf(): Configures the output to include the root node itself.WithEmptyTagSupport(): Writes empty elements as <empty/> instead of <empty></empty>.WithoutComments(): Skips CommentNode elements in the output.WithPreserveSpace(): Preserves whitespace in the output.WithoutPreserveSpace(): Trims whitespace in the output.WithIndentation(indentation string): Sets the string used for formatting the output (e.g., " " or "\t").// Example: Write indented XML with empty tag support and no comments
err := node.WriteWithOptions(os.Stdout,
WithIndentation(" "),
WithEmptyTagSupport(),
WithoutComments(),
)Use ParseWithOptions(r io.Reader, options ParserOptions) to customize the parsing behavior. One common use case is enabling line number tracking by setting WithLineNumbers: true in the ParserOptions struct. This is useful for error reporting or debugging specific locations in the source XML.
// Example of parsing with line numbers enabled
n, err := xmlquery.ParseWithOptions(r, xmlquery.ParserOptions{
WithLineNumbers: true,
})