DOCSEARCHbyAlgolia
Submit Docs

Neo4j Graph Data Science (GDS) Python Client

repository·main·Indexed 42 days ago

https://github.com/neo4j/graph-data-science-client

A Python client for the Neo4j Graph Data Science (GDS) library that allows users to project graphs, run algorithms, and manage machine learning pipelines using pure Python code. It provides a Pythonic interface for GDS tasks, supporting execution modes such as mutate, write, and stream (returning pandas DataFrames), and integrates with Aura Graph Analytics via GdsSessions.

Tokens
77K
Snippets
281
Records
342
Agent score
53%

What's inside Neo4j GDS Python Client

  1. Overview of the Neo4j Graph Data Science Python Client

    main

    The Neo4j Graph Data Science Python client is the official library for executing graph algorithms and machine learning pipelines on Neo4j graphs. It acts as a wrapper around the [Neo4j Python driver], providing a high-level Pythonic interface to the Neo4j Graph Data Science (GDS) Cypher API.

    With the client, you can use specialized graph, model, and pipeline objects to:

    • Create and manage graph projections: Define the subgraph on which algorithms will run.
    • Run graph algorithms: Execute GDS algorithms (like PageRank, Louvain, etc.) directly from Python.
    • Train and use machine learning pipelines: Manage the lifecycle of GDS machine learning models.

  2. Use Machine Learning Pipelines in GDS

    main

    The graphdatascience client provides a high-level API for managing Machine Learning Pipelines. These pipelines allow you to orchestrate complex workflows including training models, managing model catalogs, and performing inference for different ML tasks.

    Supported ML tasks include:

    • Node Classification: Predicting labels for nodes.
    • Link Prediction: Predicting the likelihood of relationships between nodes.
    • Node Regression: Predicting continuous values for nodes.

  3. Inspect OGBL (Link Property) graphs

    main

    OGBL datasets are used for link property prediction.

    Homogeneous OGBL graphs:

    • Node label: A single label "N".
    • Relationship types: Up to six types representing the split: "TRAIN_POS", "TRAIN_NEG", "VALID_POS", "VALID_NEG", "TEST_POS", "TEST_NEG".
    • Properties: An optional "features" property on nodes.

    Heterogeneous OGBL graphs:

    • Node labels: Multiple labels based on the dataset.
    • Relationship types: Names from the original dataset, suffixed with the split type (e.g., "_TRAIN", "_VALID", or "_TEST").
    • Properties: A "classLabel" property on all relationships (a single non-negative integer mapping to the original relationship types) and an optional "features" property on nodes.
    # Example: Loading 'ogbl-ddi' (Homogeneous)
G = gds.graph.datasets.ogbl.load("ogbl-ddi")

assert G.name() == "ogbl-ddi"
assert G.node_count() == 4_267
assert G.node_labels() == ["N"]
assert G.relationship_count() == 1_334_889 + 197_481  # Positive + negative counts
assert G.relationship_types() == ["TRAIN_POS", "VALID_POS", "VALID_NEG", "TEST_POS", "TEST_NEG"]

  4. Inspect OGBN (Node Property) graphs

    main

    OGBN datasets are used for node property prediction. They can be homogeneous or heterogeneous.

    Homogeneous OGBN graphs:

    • Node labels: Up to three disjoint labels representing the split: "Train", "Valid", and "Test".
    • Relationship type: A single type "R".
    • Properties: A "classLabel" property (single number) and an optional "features" property (array of numbers).

    Heterogeneous OGBN graphs:

    • Node labels: Multiple labels based on the dataset, plus the split labels ("Train", "Valid", "Test"). Nodes may have multiple labels.
    • Properties: A "classLabel" property on target nodes and an optional "features" property.
    # Example: Loading 'ogbn-arxiv' (Homogeneous)
G = gds.graph.datasets.ogbn.load("ogbn-arxiv")

assert G.name() == "ogbn-arxiv"
assert G.node_count() == 169_343
assert G.node_labels() == ["Train", "Valid", "Test"]
assert G.node_properties()["Train"] == ["features", "classLabel"]
assert G.relationship_count() == 1_166_243
assert G.relationship_types() == ["R"]
assert G.relationship_properties()["R"] == []

  5. Use Apache Arrow Flight for faster graph construction

    main

    If the Apache Arrow Flight Server is enabled in GDS (it is always enabled for Aura Graph Analytics and AuraDS), gds.graph.construct can leverage it to:

    • Significantly speed up graph construction.
    • Support multiple DataFrames for both nodes and relationships.

    Requirements for multiple DataFrames:

    • If providing multiple node DataFrames, they must contain distinct node IDs across all provided DataFrames.
    • You must call GraphDataScience.set_database to explicitly specify the target Neo4j database before calling construct.

  6. Create a Graph object

    main

    The Graph object is the primary entry point for all graph operations in the GDS Python client, including running algorithms, training models, and inspecting projected graphs. You can create a Graph object using several methods:

    • Native Projection: Projecting a graph directly from a Neo4j database using labels and types.
    • Cypher Projection: Using custom Cypher queries for complex projection logic.
    • From existing graphs: Retrieving a graph already in the catalog using gds.graph.get(name) or generating one from a distribution using gds.graph.generate.
    • From external data: Using construct with Pandas DataFrames or loading from NetworkX graphs.
    • Sampling/Filtering: Using gds.graph.filter, gds.graph.sample.rwr, or gds.graph.sample.cnarw.

    Most projection methods return a tuple containing the Graph object and a result object (like ProjectionResult or GraphCypherProjectResult) containing metadata.

    # Example: Getting an existing graph
G = gds.graph.get("offices")

  7. How machine learning pipelines work in the GDS Python Client

    main

    Machine learning pipelines (for link prediction, node property prediction, and node regression) are represented as specialized pipeline objects in the GDS Python Client. Instead of using Cypher procedures directly, you interact with these objects using a Pythonic API.

    Key differences from the Cypher API:

    • Method Chaining: Methods are called directly on the pipeline object, so you don't need to pass the pipeline name as a parameter to every call.
    • Configuration: Configuration parameters are passed as named keyword arguments in Python methods.
    • Training: The train method accepts a graph object instead of a graph name and returns a model object and a train result object containing metadata.
    • Predictions: Prediction methods on the resulting model objects also accept graph objects and use keyword arguments for configuration.

  8. Understand return types for Cypher procedure method calls

    main

    When calling methods that correspond to Cypher procedures (those preceded by CALL in the documentation), the return type depends on the execution mode:

    • Stream mode: If the procedure uses stream mode, the method returns a Pandas DataFrame representing the result table.
    • Other modes: For algorithms running in stats or mutate mode, the method returns a specific type defined for that method.
    • Server version: The gds.server_version() method specifically returns a ServerVersion object.

  9. How documentation snippet testing works

    main

    The documentation testing harness uses Asciidoctor to extract Python code blocks.

    Selection Criteria: Only AsciiDoc blocks with style: source, language: python, and without the no-test role are tested.

    Deployment Attributes:

    • enterprise: Only runs in test_plugin_enterprise.
    • networkx: Requires graphdatascience[networkx] extra; runs in all deployments (can be skipped via DOC_TEST_NETWORKX=no).
    • session: Only runs in the AGA deployment (test_aga).
    • plugin: Only runs in plugin deployments (test_docs-plugin).
    • min-server-version: Only runs if the GDS version is $\ge$ the specified version.

    Execution Lifecycle: Before a snippet runs, the harness:

    1. Prepends setup code (e.g., creating a GraphDataScience object named gds).
    2. Appends cleanup code (e.g., dropping projected graphs and resetting the database).

    Grouping: If a block has a group attribute, it is concatenated with all other snippets in the same group into a single script execution.

  10. Manage GDS jobs using JobHandle

    main

    When running algorithms or projections in Neo4j Graph Data Science, the client returns job handle objects that allow you to monitor and manage the lifecycle of asynchronous tasks.

    There are three specialized types of job handles depending on the operation performed:

    1. JobHandle: The base handle for general GDS procedures.
    2. ProjectionJobHandle: Specifically for jobs related to creating graph projections.
    3. WriteJobHandle: For jobs that write results back to the database. This handle provides access to WriteBackResult objects.

    You can use these handles to check the status of a job, retrieve results, or manage the job's execution state.

  11. Use asynchronous execution with handles

    main

    Most long-running operations in the V2 API provide non-blocking variants that return a handle instead of waiting for completion. Handles allow you to initiate work, perform other tasks, and retrieve results later. This is particularly useful for long-running graph projections or algorithm computations where you might want to recover the job from a different process using its ID.

    There are three specific handle types returned by the API:

    • ProjectionJobHandle: Returned by async endpoints that produce new graphs. It yields the projected graph and a summary.
    • JobHandle: Returned by algorithm compute(...) methods. Methods like stream(), mutate(...), and write(...) consume the result and will block on completion if necessary.
    • WriteJobHandle: Returned by write-back operations started from a JobHandle. It yields a WriteBackResult.

    All handles implement a common interface for monitoring and retrieving jobs.

    # Example of the async workflow
projection_handle = gds.graph.project.native_async("people", ["*"], ["*"])
projection_handle.wait()  # blocks until projection is done
G, _ = projection_handle.result()

compute_handle = gds.page_rank.compute(G, damping_factor=0.85)
# ... do other work here ...

scores = compute_handle.stream()         # blocks on completion, returns a DataFrame
write_handle = compute_handle.write(write_properties="pagerank")
write_result = write_handle.result()     # returns WriteBackResult

  12. Understand algorithm execution modes

    main

    The format of the data returned by an algorithm depends on its execution mode:

    mutate and write modes

    These modes modify the graph in place (adding properties or embeddings). They return a typed Pydantic object (e.g., PageRankMutateResult) containing metadata about the operation, such as the number of properties written.

    stream mode

    This mode does not persist results on the server. Instead, it streams the results back to the Python client as a pandas DataFrame. Warning: Streaming can be resource-intensive because the entire result is materialized in client memory. This is particularly relevant for large graphs or algorithms producing large outputs like node embeddings.

    train mode

    Used for algorithms that produce machine learning models. The Python client provides specialized support for interacting with these models via a dedicated model object.