PyGraphistry

PyGraphistry is an open-source Python library designed for graph visualization, analytics, and AI, featuring native GPU acceleration. Key capabilities include:

• Dataframe-native graph processing: Ingest and prepare data using Pandas, Spark, RAPIDS (GPU), and Apache Arrow.
• Integrations: Connect to various data platforms (Databricks, Splunk, PostgreSQL, etc.), graph databases (Neo4j, Amazon Neptune, TigerGraph, etc.), and Python tools (NetworkX, Graphviz).
• GFQL (Graph Query Language): A vectorized, dataframe-native graph query language with an open-source GPU runtime. It supports Cypher-like syntax via g.gfql("MATCH ...") and remote execution via g.gfql_remote([...]).
• graphistry[ai]: Access streamlined graph ML and AI methods including clustering, UMAP embeddings, and graph neural networks.
• Large-scale visualization: Create interactive visualizations with millions of edges and built-in features like drilldowns and filtering.
• Performance: Uses Apache Arrow for fast CPU ingestion and optional RAPIDS-based GPU mode for significant speedups.

Use Kepler.gl integration when you need rich interactive maps, fine-grained styling control, geographic region visualization (countries/states), or multiple map layers.

Graphistry provides:

• Full Kepler.gl passthrough: Direct access to native Kepler layers and configurations.
• Graphistry dataset shortcuts: Simplified dataset creation from nodes, edges, and geographic data.
• Type-safe configuration: Using KeplerDataset, KeplerLayer, and KeplerEncoding classes.

To use it, you typically follow this workflow:

1. Define your data (nodes/edges).
2. Use .encode_kepler_dataset() to register datasets.
3. Use .encode_kepler_layer() with a KeplerLayer object to define how data is visualized (points, arcs, hexagons, etc.).
4. Use .encode_kepler() with a KeplerEncoding object for complete configuration.

import graphistry
from graphistry import KeplerLayer
import pandas as pd

cities = pd.DataFrame({
    "id": ["NYC", "LA", "London"],
    "latitude": [40.7128, 34.0522, 51.5074],
    "longitude": [-74.0060, -118.2437, -0.1278]
})

g = (graphistry
     .nodes(cities, "id")
     .bind(point_latitude="latitude", point_longitude="longitude")
     .encode_kepler_dataset(id="cities", type="nodes")
     .encode_kepler_layer(KeplerLayer({
         "id": "city-points",
         "type": "point",
         "config": {
             "dataId": "cities",
             "columns": {"lat": "latitude", "lng": "longitude"},
             "visConfig": {"radius": 10, "opacity": 0.8}
         }
     }))
     .layout_settings(play=0))

g.plot()

Collections are overlays that define subsets of nodes, edges, or subgraphs using GFQL operations. They are applied in priority order (earlier collections override later ones).

• Collection Set: Wraps a gfql_chain in a set type. Requires an id, name, node_color, and an expr containing the gfql_chain.
• Collection Intersection: Creates a new collection by intersecting existing sets. Requires a name, node_color, and an expr of type intersection containing an array of sets (IDs).

// Collection Set
{
  "type": "set",
  "id": "purchasers",
  "name": "Purchasers",
  "node_color": "#00BFFF",
  "expr": {
    "type": "gfql_chain",
    "gfql": [
      {"type": "Node", "filter_dict": {"status": "purchased"}}
    ]
  }
}

// Collection Intersection
{
  "type": "intersection",
  "name": "High Value Purchasers",
  "node_color": "#AA00AA",
  "expr": {
    "type": "intersection",
    "sets": ["purchasers", "vip"]
  }
}

GFQL supports a subset of Cypher syntax for pattern matching.

Supported Pattern Forms:

• Single-pattern MATCH: Supports node aliases, relationship aliases, inline property maps, and top-level params=... binding.
• Node Patterns: Supports single labels (p:Person) and multi-label patterns (p:Person:Admin).
• Relationship Direction: Supports directed ->, <-, and undirected -[]- patterns.
• Relationship Types: Supports alternation like [r:KNOWS|HATES].
• Variable-length Relationships: Supports [*n], [*m..n], [*], and typed forms like [:R*2..4].
• Connected Patterns: Supports mixing variable-length and fixed-length relationships (e.g., MATCH (a)-[:R*2]->()-[:S]->(c)) and comma-separated patterns (e.g., MATCH (a)-[:A]->(b), (b)-[:B]->(c)).
• Path Binding: Supports MATCH p = (n)-[r]->(b) as long as the path variable itself is not the projected output.

The Let operation implements a DAG pattern with named bindings. It allows for complex query construction by binding intermediate results to names.

Python Example:

let({
    'persons': n({'type': 'Person'}),
    'adults': ref('persons', [n({'age': ge(18)})])
})

Wire Format:

{
  "type": "Let",
  "bindings": {
    "persons": {
      "type": "Node",
      "filter_dict": {"type": "Person"}
    },
    "adults": {
      "type": "Ref",
      "ref": "persons",
      "chain": [{
        "type": "Node",
        "filter_dict": {
          "age": {"type": "GE", "val": 18}
        }
      }]
    }
  }
}

Scoping Rules (Lexical Scoping)

• Nested Let: A Let binding can contain another Let. The inner Let is an opaque unit; its internal bindings are not visible to the outer scope.
• Visibility:
  • Outer bindings are visible to inner bindings.
  • Inner bindings are not visible to outer bindings (they do not leak upward).
  • Sibling Let blocks are isolated; they can reuse names without collision.
• Shadowing: If an inner binding has the same name as an outer binding, the inner one shadows the outer one within its scope.
• Result: The result of a Let block is the last executed binding in its scope.

GFQL's let bindings allow you to define a Directed Acyclic Graph (DAG) of named operations. This enables you to create reusable named subgraphs and reference them later in the same query, which is more efficient and cleaner than manually managing intermediate Python variables.

• let({...}): Defines a dictionary of named operations.
• ref(name, operations): References a previously defined named operation in the let block to perform further operations on it.

from graphistry import let, ref, n, e_forward, ge

# GFQL Let: Define a DAG of named operations
result = g.gfql(let({
    'persons': n({'type': 'person'}),
    'adults': ref('persons', [n({'age': ge(18)})]),  # Reference and filter persons
    'connections': [
        n({'type': 'person', 'age': ge(18)}),
        e_forward({'type': 'knows'}),
        n()  # Find connections from adults
    ]
}))

# Access any named result from the DAG
adults = result._nodes[result._nodes['adults']]
connections = result._edges[result._edges['connections']]

When developing for PyGraphistry, adhere to functional programming principles to ensure compatibility with both CPU (Pandas) and GPU (cuDF) engines.

Core Rules

• Immutability: Always return new objects; never modify DataFrames in-place.
• Avoid copy(): Most DataFrame operations already return new objects.
• Use df.assign(): Do not use the df[col] = val syntax for adding columns.

Examples

DataFrame Operations

# ✅ Good - Functional style
df = df.assign(new_col=values)
df = df[df['col'] > 0]  # Returns new DataFrame

# ❌ Bad - In-place modification  
df['new_col'] = values
df.drop('col', inplace=True)

Engine Abstraction

Always use engine-agnostic abstractions instead of checking for specific DataFrame types (like pd.DataFrame or cudf.DataFrame) to ensure code works across both CPU and GPU.

# ✅ Good - Engine agnostic
from graphistry.compute.typing import DataFrameLike
engine = resolve_engine(df)
result = engine.df_concat([df1, df2])

# ❌ Bad - Engine specific
if isinstance(df, pd.DataFrame):
    result = pd.concat([df1, df2])