repository·main·Indexed 22 days ago
https://github.com/projectdiscovery/wappalyzergoA high-performance Go implementation of the Wappalyzer technology detection library. It identifies web technologies used by websites through HTML and HTTP header analysis using normalized regexes. The library includes a client for fingerprinting, support for loading custom fingerprint JSON files, and an `update-fingerprints` CLI tool to synchronize fingerprint data from the official Wappalyzer and webappanalyzer repositories.
webappanalyzer and the official wappalyzer repository.To ensure you have the latest Wappalyzer fingerprints, use the update-fingerprints command via go install. This tool manages the auto-updating database of fingerprints used by the library.
go install -v github.com/projectdiscovery/wappalyzergo/cmd/update-fingerprints@latestThe library uses two primary structures to represent technology detection rules:
Fingerprints: A raw representation of technology fingerprints, typically used for unmarshaling JSON data. It contains maps for Cookies, Dom, JS, Headers, HTML, Script, ScriptSrc, and Meta tags.CompiledFingerprints: An optimized, internal representation where raw patterns are compiled into ParsedPattern objects for efficient matching. This is what the engine uses during runtime to perform detection.When a match is found, the engine also considers implies (technologies that are implicitly present if the primary technology is detected) and returns a matchPartResult containing the application name, version, and confidence level.
You can initialize a new Wappalyze client using New() to use the embedded fingerprints, or NewFromFile() to load fingerprints from a specific JSON file. Using NewFromFile allows you to use updated fingerprint definitions without recompiling your application.
When using NewFromFile, you can control how embedded fingerprints interact with the file-based ones using the loadEmbedded and supersede parameters:
loadEmbedded: If true, the client first loads the built-in fingerprints.supersede: If true (and loadEmbedded is also true), fingerprints in the provided file will overwrite embedded fingerprints if they share the same application name.// Using embedded fingerprints
client, err := wappalyzer.New()
// Using fingerprints from a specific file
// loadEmbedded: true, supersede: true
client, err := wappalyzer.NewFromFile("path/to/fingerprints.json", true, true)To use the library, initialize a new client using wappalyzer.New() and then call the Fingerprint method. The Fingerprint method requires the HTTP response headers and the response body (as a byte slice) to identify technologies via normalized regexes.
package main
import (
"fmt"
"io"
"log"
"net/http"
wappalyzer "github.com/projectdiscovery/wappalyzergo"
)
func main() {
resp, err := http.DefaultClient.Get("https://www.hackerone.com")
if err != nil {
log.Fatal(err)
}
data, _ := io.ReadAll(resp.Body) // Ignoring error for example
wappalyzerClient, err := wappalyzer.New()
fingerprints := wappalyzerClient.Fingerprint(resp.Header, data)
fmt.Printf("%v\n", fingerprints)
// Output: map[Acquia Cloud Platform:{} Amazon EC2:{} Apache:{} Cloudflare:{} Drupal:{} PHP:{} Percona:{} React:{} Varnish:{}]
}Use GetRawFingerprints() to access the raw JSON string containing all technology fingerprints embedded within the package. This is useful if you need to perform custom parsing or inspect the underlying fingerprint definitions directly.
package main
import (
"fmt"
"github.com/projectdiscovery/wappalyzergo"
)
func main() {
rawFingerprints := wappalyzer.GetRawFingerprints()
fmt.Println(rawFingerprints)
}If you are working with a CompiledFingerprint object, you can access the compiled rules for JavaScript and DOM detection using the following methods:
GetJSRules(): Returns a map[string]*ParsedPattern containing the compiled JS rules.GetDOMRules(): Returns a map[string]map[string]*ParsedPattern containing the compiled DOM rules.jsRules := compiledFingerprint.GetJSRules()
domRules := compiledFingerprint.GetDOMRules()Use FormatAppVersion to create a standardized string representation of a detected technology. If a version is provided, it returns app:version; otherwise, it returns just the app name.
// Returns "nginx:1.18.0"
str := wappalyzer.FormatAppVersion("nginx", "1.18.0")
// Returns "nginx"
str := wappalyzer.FormatAppVersion("nginx", "")Use ParsePattern to convert a raw pattern string into a ParsedPattern object. The function supports complex patterns that include a regular expression followed by metadata separated by \;.
Supported metadata keys:
confidence:<int>: Sets the detection confidence level (defaults to 100).version:<string>: Defines how the version should be extracted or formatted. The version string can use placeholders like \1, \2, etc., to refer to regex capture groups, or ternary expressions (e.g., \1?exists:not_exists) to handle conditional versioning.If the pattern starts with an empty string before the first \;, SkipRegex is set to true, meaning the pattern is treated as a simple existence check without regex evaluation.
pattern := "some-regex-pattern\;confidence:80\;version=\\1"
parsed, err := wappalyzer.ParsePattern(pattern)
if err != nil {
// handle error
}Use GetCategoriesMapping() to get a map of technology categories. The map keys are integers (parsed from the original JSON category IDs) and the values are categoryItem structs containing the category's Name and Priority.
package main
import (
"fmt"
"github.com/projectdiscovery/wappalyzergo"
)
func main() {
mapping := wappalyzer.GetCategoriesMapping()
for id, item := range mapping {
fmt.Printf("ID: %d, Name: %s, Priority: %d\n", id, item.Name, item.Priority)
}
}The Wappalyze client allows you to inspect the underlying fingerprint data used for detection via two methods:
GetFingerprints(): Returns the original *Fingerprints structure (the raw data).GetCompiledFingerprints(): Returns the *CompiledFingerprints structure (the optimized version used for matching).The Wappalyze client provides several methods to identify technologies based on HTTP response headers and the response body.
Important: Do not mutate the body byte slice while calling these functions, as it may lead to unexpected behavior.
Use Fingerprint(headers, body) to get a map of detected technologies. The keys in the returned map are formatted as application:version (if a version was detected).
If you need more than just the names, use these specialized methods:
FingerprintWithTitle(headers, body): Returns the detected technologies and the HTML <title> of the page.FingerprintWithInfo(headers, body): Returns a map of AppInfo containing descriptions, website URLs, icons, CPEs, and categories.FingerprintWithCats(headers, body): Returns a map of CatsInfo containing the raw category identifiers for the detected technologies.// Basic fingerprinting
results, err := wappalyzer.New()
techs := results.Fingerprint(headers, body)
// Fingerprinting with page title
techs, title := results.FingerprintWithTitle(headers, body)
// Fingerprinting with detailed AppInfo
infoMap := results.FingerprintWithInfo(headers, body)
// infoMap[techName] returns AppInfo { Description, Website, Icon, CPE, Categories }