polyfill-rs

repository·main·Indexed 42 days ago

https://github.com/floor-licker/polyfill-rs

A high-performance Rust client for Polymarket version 0.4.3, optimized for low latency and minimal memory allocation in critical trading paths. It features a zero-allocation WebSocket hot path using simd-json for book updates, a ClobClient for market interaction, and comprehensive support for L1/L2 authentication headers and EIP-712 order signing.

Tokens
23.7K
Snippets
94
Records
123
Agent score
53%

What's inside polyfill-rs

  1. How zero-allocation hot paths work in polyfill-rs

    main

    The library is designed for high performance by minimizing heap traffic in critical paths.

    Zero-allocation coverage: Strict zero-allocation claims apply to tested, warmed hot paths, specifically:

    • Existing-level book updates
    • Selected read-side calculations

    Paths that DO touch the allocator: Certain operations are designed to use the allocator when necessary:

    • First time seeing a token/book (HashMap insert + key clone)
    • New price levels when a sorted side needs to grow
    • Book removal/drop paths that release owned buffers
  2. WebSocket snapshot ordering logic

    main

    Since Polymarket book messages lack a monotonic server sequence/version, polyfill-rs implements the following ordering logic for the book applier:

    • Newer timestamps: Treated as newer data.
    • Older timestamps: Rejected.
    • Duplicates: Exact same-timestamp and same-hash messages are suppressed.
    • Same-timestamp/Different-hash: Accepted in the order they arrive via WebSocket.
  3. Understanding the two parsing layers

    main

    The library maintains two distinct parsing strategies to balance performance and ergonomics:

    1. High-performance WS path: Uses WsBookUpdateProcessor (found in src/ws_hot_path.rs). It walks a reusable simd-json tape and applies fixed-point book levels directly. This is the zero-allocation path for WebSocket book updates.
    2. Ergonomic compatibility path: Uses a generic stream parser (found in src/decode.rs) that parses through serde_json::Value. This path is more flexible, allowing it to tolerate batches, unknown event types, and mixed message shapes, but it is not zero-allocation.
  4. Quick Start with ClobClient

    main

    You can initialize a new ClobClient by providing the Polymarket CLOB URL. This client allows you to interact with markets, such as fetching sampling markets via get_sampling_markets.

    use polyfill_rs::{ClobClient, Side, OrderType};
    
    #[tokio::main]
    async fn main() -> Result<(), Box<dyn std::error::Error>> {
        let client = ClobClient::new("https://clob.polymarket.com");
        let markets = client.get_sampling_markets(None).await?;
        println!("Found {} markets", markets.data.len());
        Ok(())
    }
  5. Template for Adding New Integration Tests

    main

    When adding new integration tests, follow this pattern to ensure proper configuration handling and graceful skipping when credentials are missing.

    #[tokio::test]
    async fn test_new_feature() -> Result<()> {
        let config = TestConfig::from_env();
        
        // Skip if requirements not met
        if !config.has_auth() {
            TestReporter::skip("test_new_feature", "no private key");
            return Ok(());
        }
        
        // Test implementation
        let client = config.create_auth_client()?;
        let result = client.some_new_method().await?;
        
        // Assertions
        assert!(result.is_valid());
        
        TestReporter::success("test_new_feature");
        Ok(())
    }

    Best Practices

    1. Use TestConfig: Always use the shared test configuration.
    2. Handle Missing Credentials: Skip tests gracefully using TestReporter::skip when credentials aren't available.
    3. Measure Performance: Include timing measurements for performance-critical operations.
    4. Provide Context: Use descriptive test names and error messages.
    5. Clean Up: Don't leave test data in the system.
  6. Configure Environment Variables for Integration Testing

    main

    Integration tests that communicate with the real Polymarket API require specific environment variables. You can set these in a .env file or export them directly to your shell.

    Required

    • POLYMARKET_PRIVATE_KEY: Your private key.

    Optional API Credentials

    • POLYMARKET_API_KEY
    • POLYMARKET_API_SECRET
    • POLYMARKET_API_PASSPHRASE
    • Note: POLYMARKET_SECRET and POLYMARKET_PASSPHRASE are accepted as backward-compatible aliases.

    Optional Configuration

    • POLYMARKET_HOST: Defaults to https://clob.polymarket.com.
    • POLYMARKET_CHAIN_ID: Defaults to 137.
    • POLYMARKET_SIGNATURE_TYPE: Defines wallet type: 0 (EOA), 1 (Proxy), 2 (Gnosis Safe/browser wallet), 3 (Poly1271).
    • POLYMARKET_FUNDER_ADDRESS: Override the derived funder/proxy wallet address.
    • POLYMARKET_RESOLVE_IP: Pin clob.polymarket.com to a specific A record if local DNS is blocked.
    export POLYMARKET_PRIVATE_KEY="your_private_key_here"
    export POLYMARKET_API_KEY="your_api_key"
    export POLYMARKET_API_SECRET="your_api_secret"
    export POLYMARKET_API_PASSPHRASE="your_passphrase"
  7. Quick Start: Running Basic Tests

    main

    To run the standard test suite, including unit tests and documentation tests, use the following commands. Note that real-API integration tests are marked as #[ignore] by default and will not run unless explicitly requested.

    # Run unit tests + doc tests
    cargo test --all-features
    
    # Run the "no-alloc hot paths" regression tests
    cargo test --all-features --test no_alloc_hot_paths
    
    # Compile-check all examples
    cargo build --examples
    cargo test --all-features
  8. Debug Tests with Detailed Logging

    main

    If tests are failing and you need more visibility into the execution flow, use the RUST_LOG environment variable to increase verbosity.

    # Enable debug logging
    RUST_LOG=debug cargo test --all-features --test integration_tests -- --ignored --nocapture --test-threads=1
    
    # Enable trace logging for maximum detail
    RUST_LOG=trace cargo test --all-features --test integration_tests -- --ignored --nocapture --test-threads=1
    RUST_LOG=debug cargo test --all-features --test integration_tests -- --ignored --nocapture --test-threads=1
  9. Run Full Integration Tests

    main

    To execute tests that interact with the live Polymarket API (which are ignored by default), use either the provided helper script or run cargo test with specific flags.

    Using the helper script

    ./scripts/run_integration_tests.sh

    Using Cargo directly

    # --ignored: runs the #[ignore] tests
    # --nocapture: shows stdout/stderr
    # --test-threads=1: runs sequentially to avoid rate limits/conflicts
    cargo test --all-features --test integration_tests -- --ignored --nocapture --test-threads=1
  10. Use WebSocketBookApplier for high-performance order book updates

    main

    For latency-sensitive applications that primarily need order book updates, use WebSocketBookApplier. This mode bypasses standard StreamMessage decoding to apply updates directly into an OrderBookManager using a WsBookUpdateProcessor.

    Workflow:

    1. Create an OrderBookManager and 'warm it up' by creating books for all target asset IDs.
    2. Create a WebSocketStream.
    3. Convert the stream using .into_book_applier(books, processor).
    4. The resulting applier implements Stream, yielding Result<WsBookApplyStats> instead of generic messages.

    Optimization Note: For maximum performance, use apply_bytes_message(&mut [u8]) with a reusable mutable buffer to avoid allocations.

    let books = OrderBookManager::new(64);
    // Warm up books
    books.get_or_create_book("asset_id").unwrap();
    
    let processor = WsBookUpdateProcessor::new(1024);
    let stream = WebSocketStream::new("wss://...");
    let mut applier = stream.into_book_applier(&books, processor);
    
    // The applier can now be used as a Stream
    while let Some(result) = applier.next().await {
        let stats = result?;
        // stats contains book_messages and book_levels_applied
    }
  11. Optimize repeated order creation with PreparedOrderPath

    main

    For latency-sensitive applications that create many orders for the same market or token, use PreparedOrderPath. This object caches expensive computations and state, such as:

    • Tick-size rounding configurations
    • Exchange address parsing
    • Token ID parsing (to U256)
    • EIP-712 domain preparation
    • Normalized bytes32 fields for builder_code and metadata

    Once prepared, you can call create_limit_order or create_market_order directly on the PreparedOrderPath instance.

    // 1. Initialize builder
    let builder = OrderBuilder::new(signer, None, None);
    
    // 2. Prepare the path (do this once per market/token)
    let prepared_path = builder.prepare_order_path(
        137, 
        "123456", 
        Decimal::from_str("0.01").unwrap(), 
        false, 
        None, 
        None
    ).unwrap();
    
    // 3. Create orders rapidly
    let limit_order = prepared_path.create_limit_order(
        Side::BUY, 
        Decimal::from_str("0.45").unwrap(), 
        Decimal::from_str("12.34").unwrap(), 
        Some(1_900_000_000)
    ).unwrap();