surf

Surf allows you to mimic specific browser fingerprints to evade detection. You can use the .Impersonate() builder method followed by specific browser targets like .Chrome() or .Firefox(). You can also combine these with platform-specific methods like .IOS(), .Android(), or .RandomOS() to vary the device profile.

// Chrome Impersonation
client := surf.NewClient().
    Builder().
    Impersonate().
    Chrome().        // Latest Chrome v150
    Build().
    Unwrap()

// Firefox with Random OS
client := surf.NewClient().
    Builder().
    Impersonate().
    RandomOS().      // Randomly selects Windows, macOS, Linux, Android, or iOS
    Firefox().       // Latest Firefox v148
    Build().
    Unwrap()

// iOS Chrome
client := surf.NewClient().
    Builder().
    Impersonate().
    IOS().
    Chrome().
    Build().
    Unwrap()

Surf features an extensible middleware system. You can register middleware using the .With() method during the client building process. Middleware can target the *surf.Request, *surf.Response, or the *surf.Client itself.

// Request Middleware
client := surf.NewClient().
    Builder().
    With(func(req *surf.Request) error {
        req.AddHeaders("X-Custom-Header", "value")
        return nil
    }).
    Build().
    Unwrap()

// Response Middleware
client := surf.NewClient().
    Builder().
    With(func(resp *surf.Response) error {
        fmt.Printf("Response status: %d\n", resp.StatusCode)
        return nil
    }).
    Build().
    Unwrap()

// Client Middleware
client := surf.NewClient().
    Builder().
    With(func(client *surf.Client) error {
        client.GetClient().Timeout = 30 * time.Second
        return nil
    }).
    Build().
    Unwrap()

Use surf.NewClient() to initiate a request. Surf uses a Result type pattern; check for errors using .IsErr() and access the successful response via .Ok().

package main

import (
    "fmt"
    "log"
    "github.com/enetx/surf"
)

func main() {
    resp := surf.NewClient().Get("https://api.github.com/users/github").Do()
    if resp.IsErr() {
        log.Fatal(resp.Err())
    }

    fmt.Println(resp.Ok().Body.String().Unwrap())
}

Install the Surf HTTP client library using go get.

Required Go version: 1.27+

go get -u github.com/enetx/surf

To improve performance:

1. Reuse Clients: Create a client once and reuse it for multiple requests. Use defer client.CloseIdleConnections() to clean up.
2. Enable Caching: Use .Builder().CacheBody() to enable body caching. Subsequent accesses to the body will use the cache instead of performing new network I/O.

The surf.Client builder allows for advanced network configurations:

• H2C: Enable HTTP/2 without TLS using .H2C().
• Custom Header Order: Use .SetHeaders(headers) with a g.NewMapOrd to control exact header order for fingerprinting evasion.
• DNS: Set a custom DNS server with .DNS("IP:PORT") or use DNS-over-TLS with .DNSOverTLS().Cloudflare().
• Unix Sockets: Connect via Unix domain sockets using .UnixSocket("path").
• Interface Binding: Bind to a specific local IP using .InterfaceAddr("IP").
• Retries: Configure automatic retries with .Retry(count, duration).

// Example: Custom DNS and Retries
client := surf.NewClient().
    Builder().
    DNS("8.8.8.8:53").
    Retry(3, 2*time.Second).
    Build().
    Unwrap()

For large files, use resp.Ok().Body.Stream() to get an io.ReadCloser. For Server-Sent Events, use resp.Ok().Body.SSE(callback) where the callback returns true to continue reading or false to stop.

// Streaming
resp := surf.NewClient().Get("https://example.com/large-file").Do()
if resp.IsOk() {
    stream := resp.Ok().Body.Stream()
    defer stream.Close()
    // ... use scanner on stream
}

// SSE
resp := surf.NewClient().Get("https://example.com/events").Do()
if resp.IsOk() {
    resp.Ok().Body.SSE(func(event *sse.Event) bool {
        fmt.Printf("Event: %s, Data: %s\n", event.Event, event.Data)
        return true
    })
}

To maintain a session across multiple requests (e.g., for login flows), use the .Builder() to enable a cookie jar via .Session(). For manual control, use .AddCookies() to send specific cookies or iterate over resp.Ok().Cookies to retrieve them from a response.

// Enable persistent sessions
client := surf.NewClient().
    Builder().
    Session().        // Enable cookie jar
    Build().
    Unwrap()

// Login
client.Post("https://example.com/login").Body(credentials).Do()

// Subsequent requests will include session cookies
resp := client.Get("https://example.com/dashboard").Do()

// Manual Cookie Management
resp := surf.NewClient().
    Get("https://example.com").
    AddCookies(cookies...).
    Do()

To inspect the TLS handshake and server identity, use resp.Ok().TLSGrabber(). This returns a structure containing TLSVersion, ExtensionServerName, FingerprintSHA256, CommonName, and Organization.

if resp.IsOk() {
    if tlsInfo := resp.Ok().TLSGrabber(); tlsInfo != nil {
        fmt.Printf("TLS Version: %s\n", tlsInfo.TLSVersion)
        fmt.Printf("Fingerprint: %s\n", tlsInfo.FingerprintSHA256)
    }
}

Check if a request was successful using resp.IsOk(). If successful, access the response via resp.Ok().

Status Codes: Use resp.Ok().StatusCode with methods like .IsSuccess(), .IsRedirection(), .IsClientError(), or .IsServerError().

Body Processing: Access the body via resp.Ok().Body and use methods like .String(), .Bytes(), .UTF8(), .Contains(string), or .Dump(filename) to process content.

resp := surf.NewClient().Get("https://example.com/data").Do()
if resp.IsOk() {
    body := resp.Ok().Body

    // As string
    if content := body.String(); content.IsOk() {
        fmt.Println(content.Ok())
    }

    // As bytes
    if data := body.Bytes(); data.IsOk() {
        fmt.Println(len(data.Ok()))
    }

    // Save to file
    err := body.Dump("response.html")
}