DOCSEARCHbyAlgolia
Submit Docs

Spector.js

repository·master·Indexed 23 days ago

https://github.com/babylonjs/spector.js

A debugging tool for WebGL and WebGL2 developers to explore and troubleshoot scenes by capturing frames to inspect command lists, visual states, and context information. It is compatible with all engines and vanilla scripts, supporting both main thread and OffscreenCanvas in Web Workers. The library includes a Model Context Protocol (MCP) server that allows AI assistants to navigate websites, capture frames, and inspect deep WebGL state using a headless Chromium browser via Playwright.

Tokens
20.2K
Snippets
38
Records
132
Agent score
81%

What's inside Spector.js

  1. Capture OffscreenCanvas in Workers

    master

    SpectorJS supports OffscreenCanvas both on the main thread and inside Web Workers. Use the appropriate bundle for your environment:

    BundleUse
    dist/spector.bundle.jsMain thread (includes UI)
    dist/spector.worker.bundle.jsInside Workers (headless, no UI)

    Manual API (Recommended for Workers)

    To capture from a Worker, you must bridge the Worker on the main thread and load the worker bundle inside the worker script.

    Main thread implementation:

    var spector = new SPECTOR.Spector();
var worker = new Worker('myWorker.js');

// Bridge the Worker for capture
spector.spyWorker(worker);

// Capture from the Worker
spector.onCapture.add(function(capture) {
    console.log('Captured from Worker:', capture.commands.length, 'commands');
    spector.getResultUI().display();
    spector.getResultUI().addCapture(capture);
});

spector.captureWorker(worker);

    Inside myWorker.js:

    importScripts('spector.worker.bundle.js');

var canvas = new OffscreenCanvas(800, 600);
var gl = canvas.getContext('webgl2');

function render() {
    // ... draw calls ...
    setTimeout(render, 16);
}
render();

    Auto-injection (Best-effort)

    Use spyWorkers() to monkey-patch the global Worker constructor. This automatically injects the Spector bundle into every new Worker created.

    var spector = new SPECTOR.Spector();
spector.spyWorkers('spector.worker.bundle.js');

// To stop intercepting:
// spector.stopSpyingWorkers();

    Note: Auto-injection may fail with cross-origin Workers, strict CSP policies, or ES module Workers. Use the manual API for reliability.

  3. Use SPECTOR.CaptureMenu and SPECTOR.ResultView as standalone components

    master

    SpectorJS provides two UI components that can be used independently of the browser extension:

    1. SPECTOR.CaptureMenu: An embedded menu used to select canvases, trigger captures, and play/pause rendering. It can be used as a standalone component.
    2. SPECTOR.ResultView: An embedded panel used to display and inspect the results of captured scenes. It can also be used as a standalone component.

  4. Access adapters via React Context

    master

    Adapters are passed down to the React tree using React Context. Components can access the adapter instance to read state or trigger events:

    • For the Capture Menu: CaptureMenuContext $\rightarrow$ useCaptureMenu() $\rightarrow$ adapter instance.
    • For the Result View: ResultViewContext $\rightarrow$ useResultView() $\rightarrow$ adapter instance.

    Usage Pattern:

    const adapter = useCaptureMenu();
// Read state from the adapter's store
const state = useStore(adapter.store);
// Trigger events via the adapter
adapter.handleXxx();

  5. Integrate live shader editing into a WebGL engine

    master

    SpectorJS supports live shader editing for engines that implement a specific rebuild interface. Instead of SpectorJS managing the complex state of re-binding uniforms and attributes, the engine is responsible for recompiling the program and managing its state.

    To enable this, you must attach a rebuildProgram function to your linked WebGLProgram objects.

    The rebuildProgram Signature

    Your engine must provide a function with this signature:

    rebuildProgram(
    vertexSourceCode: string, 
    fragmentSourceCode: string, 
    onCompiled: (program: WebGLProgram) => void, 
    onError: (message: string) => void
): void;
    • vertexSourceCode / fragmentSourceCode: The new shader source code from the editor.
    • onCompiled: A callback triggered when compilation succeeds. It must receive the new linked WebGLProgram.
    • onError: A callback triggered on failure. It must pass the WebGL error message so SpectorJS can display it in the editor gutter.

    Implementation Pattern

    When linking a program in your engine, append the rebuild function to the WebGLProgram instance:

    // 'program' is the linked WebglProgram
// 'this._rebuildProgram' is your engine's implementation
program.__SPECTOR_rebuildProgram = this._rebuildProgram.bind(this);
    rebuildProgram(vertexSourceCode: string, // The new vertex shader source
            fragmentSourceCode: string, // The new fragment shader source
            onCompiled: (program: WebGLProgram) => void, // Callback triggered by your engine when the compilation is successful. It needs to send back the new linked program.
            onError: (message: string) => void): void; // Callback triggered by your engine in case of error. It needs to send the WebGL error to allow the editor to display the error in the gutter.

// program is the linked WebglProgram that Babylon is expanding
// with a custom rebuild function.
// Noticed we bind the context to ensure it runs as part of your engine and not the program itself
program.__SPECTOR_rebuildProgram = this._rebuildProgram.bind(this);

  6. Manage state with ExternalStore

    master

    State management is handled by ExternalStore.ts using a bridge between imperative code and React via useSyncExternalStore.

    Key Rules for State Updates:

    • Mount Once: Never call root.render() repeatedly. Mount the root once and update state via store.setState().
    • Immutable Updates: setState takes an updater function: (prev) => newState. You must always return a new object reference (e.g., using the spread operator ...) to trigger React re-renders.
    • Subscription: Components subscribe to the store using the useStore(store) hook.
    • Binding: getSnapshot and subscribe are implemented as arrow functions to ensure they are bound correctly, as React calls them without a this context.

  8. How the Spector.js Frontend Architecture works

    master

    Spector.js is a WebGL debugger that intercepts WebGL calls and displays them in an embedded overlay. The frontend is built with React 18 and follows a layered architecture:

    1. Backend (src/backend/): Handles WebGL interception, context spying, and command recording.
    2. Shared (src/shared/): Contains domain types like ICapture, ICommandCapture, Observable, and Logger.
    3. Frontend (src/embeddedFrontend/react/): The React 18 UI layer.

    The entry point is src/spector.ts, which creates the Spector class and instantiates the frontend via ReactCaptureMenu and ReactResultView.

  9. How the Spector.js MCP Server works

    master

    The Spector.js MCP server acts as a bridge between an AI Assistant and a WebGL website. It uses a headless Chromium browser (via Playwright) to navigate to a target URL and injects the Spector.js debugger into the page at runtime.

    Architecture Flow: AI Assistant $\leftrightarrow$ MCP Server (stdio) $\leftrightarrow$ Playwright (headless Chromium) $\leftrightarrow$ Any WebGL Website + Spector.js

    When running from within the Spector.js repository, the server prioritarily uses the locally built dist/spector.bundle.js. This allows developers to test MCP tooling against local changes to Spector.js without needing to publish to npm.

  11. Use the Adapter Pattern to bridge React and the Public API

    master

    The frontend uses an adapter pattern to connect the imperative public API (called by spector.ts) to the React UI. There are two main adapters:

    • ReactCaptureMenu (react/CaptureMenu/ReactCaptureMenu.ts)
    • ReactResultView (react/ResultView/ReactResultView.ts)

    Adapter Responsibilities:

    • Expose the exact same public methods and Observable properties as the Spector class (e.g., display(), hide(), setFPS(), addCapture()).
    • Own an ExternalStore<State>.
    • Mount the React tree once via createRoot() in the constructor.
    • Translate imperative calls into store.setState() to trigger React re-renders.
    • Translate React event callbacks into Observable.trigger() calls back to spector.ts.

    Note: When modifying the public API, you must update the adapter class. React components should remain purely presentational.

  12. Add a new sample to Spector.js

    master

    To add a new sample for testing:

    1. Add a new script in the sample/js folder.
    2. Launch it by replacing fileName in the following URL pattern (omit the .js extension):

    http://localhost:1337/sample/index.html?sample=fileName

    http://localhost:1337/sample/index.html?sample=fileName