From CSV to PageRank: Leveraging Neo4j GDS Graph Projection in Production

Problem: Running Graph Algorithms on a Live Store

When you keep your Neo4j database as the source of truth for an application, any heavy analytics workload can slow down or even lock the database. Engineers often ask: Can I run PageRank or community detection without touching the live data? The answer is yes – by using the Graph Data Science (GDS) library’s graph projection feature.

Thesis: Projection = Fast, Isolated Analytics

A graph projection creates an in‑memory copy of selected nodes and relationships. The copy is a snapshot, so the original database stays untouched while GDS algorithms run on the projected graph. Because the data lives in the JVM heap, queries are lightning‑fast, but you must manage memory and keep the projection in sync with the source.

Step‑by‑Step: From CSV to PageRank

1. Load the raw data

   Assume a CSV file people.csv with columns id, name, and friendId that represents a friendship network.

   USING PERIODIC COMMIT 1000
   LOAD CSV WITH HEADERS FROM 'file:///people.csv' AS row
   MERGE (p:Person {id: row.id})
   SET p.name = row.name
   WITH p, row
   MATCH (f:Person {id: row.friendId})
   MERGE (p)-[:KNOWS]->(f);
   

   Run this in the Neo4j Browser or Neo4j Desktop. You need write permissions on the database.

2. Project the graph to GDS

   The projection is one call that pulls the chosen nodes and relationships into the GDS in‑memory store. The call returns a summary you can inspect.

   CALL gds.graph.project(
     'socialGraph',
     ['Person'],
     {
       KNOWS: {
         orientation: 'UNDIRECTED'
       }
     }
   ) YIELD nodeCount, relationshipCount, nodeProperties, relationshipProperties;
   

   Key points:

   • socialGraph is the name you’ll use when calling algorithms.
   • Only nodes with label Person are included.
   • Only KNOWS relationships are projected.
   • Orientation can be UNDIRECTED for undirected algorithms or DIRECTED if direction matters.
3. Run PageRank on the projected graph

   Now call the algorithm. The result is streamed back; you can limit or order it as needed.

   CALL gds.pageRank.stream('socialGraph')
   YIELD nodeId, score
   RETURN gds.util.asNode(nodeId).name AS name, score
   ORDER BY score DESC
   LIMIT 5;
   

   Because the algorithm runs on the in‑memory graph, it finishes in a fraction of a second, even for hundreds of thousands of nodes.

Trade‑Offs to Keep in Mind

• Memory pressure: The entire projected graph resides in the JVM heap. A 1‑million‑node graph may require 3–4 GB of heap. Monitor metrics.graph.projectedHeapBytes via CALL gds.metrics.stream() to ensure you stay within limits.
• Data staleness: Projections are static snapshots. If you add, delete, or modify nodes/relationships in the source database, the projected graph does not automatically update. You must re‑project to keep analytics current.
• Operational overhead: Re‑projecting large graphs can be expensive. Consider incremental projection or schedule re‑projection during low‑traffic windows.

Actionable Checklist for Production

• Allocate sufficient JVM heap for the projected graph size. A rule of thumb: 4 bytes per node + 8 bytes per relationship, plus overhead.
• Use CALL gds.graph.project with property filters (nodeFilter, relationshipFilter) to shrink the graph.
• Schedule re‑projection using CALL gds.graph.project('name', …) in a nightly job or trigger it after bulk loads.
• Expose projection metrics via Prometheus or Neo4j’s internal metrics to alert on memory spikes.
• When the projected graph is no longer needed, drop it with CALL gds.graph.drop('socialGraph'); to free heap.

By following this pattern you isolate heavy analytics from the live database, keep performance predictable, and maintain a clear path to keep the analytics up to date.