# Week 6: GraphRAG and global context

Applied AI Course — Sawan Ruparel

Public self-study edition. See https://applied-ai.sawanruparel.com/updates/ for version notes.

# GraphRAG

## When structure beats similarity

**Week 6 — Applied AI Course**

Knowledge graphs meet retrieval: index *relationships*, not just chunks.

---

Sources

- [Edge et al., "From Local to Global: A Graph RAG Approach to Query-Focused Summarization" (Microsoft, 2024)](https://arxiv.org/abs/2404.16130)
- [Guo et al., "LightRAG: Simple and Fast Retrieval-Augmented Generation" (2024)](https://arxiv.org/abs/2410.05779)
- [microsoft/graphrag](https://github.com/microsoft/graphrag)

---

# Where We Came From

```mermaid
flowchart LR
    Docs[Documents] --> Chunk[Chunk]
    Chunk --> Embed[Embed]
    Embed --> Index[(Vector index)]
    Q[User question] --> Eq[Embed query]
    Eq --> ANN[ANN search]
    Index --> ANN
    ANN --> TopK[Top-k chunks]
    TopK --> LLM
    Q --> LLM
    LLM --> Ans[Answer]
```

Vector RAG retrieves text chunks whose **embeddings are closest** to the query. This works well when:

- The answer lives in a few specific passages
- The user's wording is close to the document's wording
- The corpus is large enough that semantic similarity is meaningful

## What it ignores

A vector index treats the corpus as a **bag of chunks**. It throws away:

- **Relationships between entities** ("Acme Corp is owned by ConsolidatedCo") that span chunks
- **The corpus structure** — folder hierarchy, citation graph, version history, threaded conversation
- **Aggregate properties** that require reading many chunks together ("how many engineers left in Q3?")

For point lookups, this is fine. For questions that require *connecting* facts or *summarizing* a slice of the corpus, it falls down. That's where GraphRAG fits.

## The benchmark numbers behind the hype

Microsoft's GraphRAG paper [Edge et al. 2024](https://arxiv.org/abs/2404.16130) reports **+20–80% comprehensiveness** vs naive RAG on query-focused summarization tasks (using LLM-as-judge). LightRAG ([Guo et al. 2024](https://arxiv.org/abs/2410.05779)) shows similar gains at lower cost. Both compare against well-tuned RAG baselines, not strawmen.

Sources

- [Lewis et al. — original RAG (2020)](https://arxiv.org/abs/2005.11401)
- [Edge et al. — GraphRAG paper](https://arxiv.org/abs/2404.16130)
- [Anthropic — Contextual Retrieval blog (2024)](https://www.anthropic.com/news/contextual-retrieval)

---

# Three Question Shapes Vector RAG Can't Answer

```mermaid
flowchart LR
    Q1["'Who reports to whom<br/>at Acme?'"] -.->|"needs<br/>relationships"| RAG[Vector RAG]
    Q2["'Summarize all of<br/>2025's product launches'"] -.->|"needs<br/>aggregation"| RAG
    Q3["'How did Alice's role<br/>change after the merger?'"] -.->|"needs<br/>multi-hop"| RAG
    RAG -.-> Fail[fragile or<br/>incomplete answers]
```

## Multi-hop questions

> *"Which of our customers run on AWS, and how many of them have open Tier-1 tickets?"*

Vector RAG retrieves passages about customers, separately about AWS, separately about tickets. None of the top-k chunks contains the *join*. The model is left to fabricate it from partial information.

## Global summarization

> *"What were the recurring themes in our customer interviews last quarter?"*

The answer requires reading **every** interview transcript, then synthesizing. No top-k retrieval works; the question is global, not local.

## Entity-centric queries with traversal

> *"What are all the products that depend on the auth service?"*

The dependency graph spans many docs. A bag of chunks loses the graph structure. Even if every chunk is retrieved, the model has to reconstruct edges from prose.

## What this looks like in eval

Naive RAG on a 1,000-document corpus, scored by LLM-judge on a benchmark of 100 questions split 80/20 between local and global:

| Question type | Naive RAG | GraphRAG (global search) |
|---------------|-----------|--------------------------|
| Local fact lookup | **84%** correct | 81% correct |
| Multi-hop | 31% | **62%** |
| Global summarization | 22% | **78%** |

(Numbers approximate, from [Edge et al. 2024 §4.3](https://arxiv.org/abs/2404.16130). Local lookup parity is the expected result — GraphRAG isn't trying to win at point lookups.)

## Implication

Vector RAG is not "wrong" — it's optimized for the *local lookup* shape. If your traffic is mostly local lookups, you don't need GraphRAG. If you have a mix, you need both, plus a router that picks the right one per query.

Sources

- [Edge et al. — GraphRAG paper](https://arxiv.org/abs/2404.16130)
- [LangChain — RAG anti-patterns](https://blog.langchain.dev/)

---

# Index the Graph, Not Just the Chunks

GraphRAG augments — or replaces — the vector index with a **structured knowledge graph**. Entities become nodes; the relationships between them become edges. Retrieval can now traverse the graph, not just look up nearest neighbors in an embedding space.

```mermaid
flowchart LR
    Docs[Documents] --> Extract["LLM:<br/>entities + relations"]
    Extract --> Graph["Knowledge graph<br/>(nodes + edges)"]
    Extract --> Embed[Embeddings of nodes]
    Graph --> Community[Community detection]
    Community --> Summaries["LLM:<br/>community summaries"]
    Embed --> Local["Local search<br/>(entity-centered)"]
    Summaries --> Global["Global search<br/>(community-centered)"]
    Q[Query] --> Router{Local or global?}
    Router --> Local --> Ans1[Answer]
    Router --> Global --> Ans2[Answer]
```

## What the LLM does at index time

A GraphRAG pipeline uses an LLM **at indexing**, not just at retrieval. For each chunk:

1. **Extract entities** — people, orgs, products, files, dates, anything domain-specific (a configurable taxonomy)
2. **Extract relations** — "Alice manages Bob", "Postgres is a dependency of OrderService", "Q3 launch postponed Q4 launch"
3. **Generate descriptions** — for each entity and relation, a short summary the system can search against later

This is expensive: many LLM calls during indexing. The payoff is that retrieval is **cheap and structured** — a graph traversal rather than a re-embedding cascade.

## What the structure gives you

| Capability | Vector RAG | GraphRAG |
|------------|------------|----------|
| Point lookup | ✅ | ✅ (via node embedding) |
| Multi-hop | ❌ | ✅ (graph traversal) |
| Global summary | ❌ | ✅ (community summaries) |
| Entity-centric drill-down | ❌ | ✅ (node + neighbors) |
| Schema-free | ✅ | ✅ (LLM-extracted) |
| Cheap to index | ✅ | ❌ (LLM-heavy) |
| Cheap to update | ✅ | ❌ (re-extract affected entities) |

## What's *not* new

The idea of indexing entities and relations predates LLMs by decades (ontology-based retrieval, semantic networks). What's new is that **LLMs can do the extraction** for free-form text — no schema, no NER training, no hand-coded rules.

Sources

- [Edge et al. — GraphRAG paper](https://arxiv.org/abs/2404.16130)
- [microsoft/graphrag — reference implementation](https://github.com/microsoft/graphrag)

---

# Turning Chunks Into Nodes

The first stage of the indexing pipeline. For each chunk, an LLM extracts named entities and their (optionally) typed attributes.

```mermaid
flowchart LR
    Chunk["Chunk: 'Alice, the VP of Engineering at Acme<br/>since 2023, oversees the platform team.'"]
    Chunk --> LLM
    LLM --> Out["[<br/>{name: 'Alice', type: 'Person', desc: 'VP Engineering at Acme since 2023'},<br/>{name: 'Acme', type: 'Organization', desc: 'company that employs Alice'},<br/>{name: 'platform team', type: 'Team', desc: 'engineering team Alice oversees'}<br/>]"]
```

## The extraction prompt (simplified)

```python
EXTRACTION_PROMPT = """
Extract entities and their relationships from the text below.

Entity types to look for: {entity_types}

For each entity, output:
- name: the canonical name
- type: from the list above
- description: one sentence summarizing what the text says about this entity

For each relationship between two entities in this chunk, output:
- source, target: entity names
- description: how they are related (from the text)
- weight: 1-10 strength

Output JSON. Do not invent entities not mentioned in the text.

TEXT:
{chunk}
"""
```

A real pipeline ([microsoft/graphrag](https://github.com/microsoft/graphrag)) adds:

- A **gleaning loop** — after the first pass, ask the model "did you miss any?" up to N times to catch entities mentioned in passing
- **Consistency checks** — entities and relations must be grounded in the text, not invented
- **A configurable taxonomy** — entity types specific to the domain (`Patent`, `Compound`, `Trial` for a pharma corpus; `Repo`, `PR`, `Author` for an engineering corpus)

## Cost anatomy

Extracting from a 1M-token corpus with Claude Haiku at $1/M input + $5/M output:
- Input: 1M tokens × $1 = **$1**
- Output: ~30k tokens (the extracted JSON) × $5 = **$0.15**
- With one round of gleaning: roughly 1.5×

So **~$1.50–$2 per 1M tokens** with Haiku — cheap enough that re-indexing a corpus weekly is not unreasonable.

## Pitfalls

- **Hallucinated entities** — the model "remembers" entities from training data. Mitigate by requiring the model to quote the source span
- **Coreference** — "Alice" in chunk 12 and "she" in chunk 14 are the same person. A naive extractor produces two entities; a deduplication pass fixes it (next slide)
- **Granularity drift** — different chunks extract the same entity at different levels of detail. Resolved at the merging stage

Sources

- [microsoft/graphrag — extraction prompts & source](https://github.com/microsoft/graphrag)
- [Edge et al. — §3.1 Source Documents → Text Chunks → Element Instances](https://arxiv.org/abs/2404.16130)

---

# Merging Local Extractions Into One Graph

Each chunk produces its own little entity-and-relation dump. The graph-construction stage merges all of those into a single global graph.

```mermaid
flowchart LR
    subgraph "Per-chunk extractions"
      E1["Chunk 1:<br/>(Alice, Acme)"]
      E2["Chunk 7:<br/>(Alice Chen, Acme Corp)"]
      E3["Chunk 22:<br/>(Acme, Postgres)"]
    end
    subgraph "Deduplication"
      D[Resolve aliases<br/>via LLM + embedding]
    end
    E1 & E2 & E3 --> D
    D --> Graph[("Merged graph:<br/>3 entities, 2 edges,<br/>aliases collapsed")]
```

## The merging steps

### 1. Normalize names

`Alice` ≈ `Alice Chen` ≈ `A. Chen` — resolve to one canonical entity. Two signals:

- **String similarity** (Levenshtein, Jaro-Winkler) for obvious typo and abbreviation matches
- **Embedding similarity** on the entity descriptions for semantic matches ("VP of Eng" vs "VP Engineering")
- **LLM judge** as the tiebreaker for ambiguous cases ("Is 'Apple Inc' in chunk 12 the same as 'Apple' in chunk 88?")

### 2. Merge descriptions

When two chunks describe the same entity, the system **summarizes both descriptions into one** via an LLM call. The merged description preserves the union of facts, drops duplication.

### 3. Aggregate edge weights

If chunks 1, 5, and 22 all assert "Alice manages Bob", the edge gets `weight = 3` (or whatever weighting scheme — sum, max, average).

## What the merged graph looks like

```python
# Pseudocode of the resulting structure
graph = {
  "nodes": {
    "Alice": {"type": "Person", "description": "VP Engineering at Acme since 2023, oversees platform team",
              "source_chunks": [1, 7, 14, 22]},
    "Acme":  {"type": "Org",    "description": "Software company; employs ~500; uses Postgres",
              "source_chunks": [1, 7, 22, 31]},
    ...
  },
  "edges": [
    {"source": "Alice", "target": "Acme", "rel": "works_at", "weight": 3, "source_chunks": [1, 7, 14]},
    {"source": "Acme",  "target": "Postgres", "rel": "uses",  "weight": 1, "source_chunks": [22]},
    ...
  ]
}
```

## Why the deduplication step is load-bearing

Skip it and you get **graph fragmentation**: 18 versions of "Alice", each with one edge — none of which can answer "how many people does Alice manage?". Most of the lift in GraphRAG comes from this stage being well-tuned.

Sources

- [Edge et al. — §3.2 Element Summaries](https://arxiv.org/abs/2404.16130)
- [LightRAG — entity merging via embedding similarity](https://arxiv.org/abs/2410.05779)

---

# Finding Clusters in the Graph

A merged knowledge graph for a real corpus has thousands of nodes. To make global queries tractable, GraphRAG runs **community detection** — partitioning the graph into densely-connected clusters that an LLM can summarize one at a time.

```mermaid
flowchart LR
    Graph["Merged graph<br/>(5k nodes, 12k edges)"] --> Leiden["Leiden algorithm<br/>(hierarchical)"]
    Leiden --> L0["Level 0:<br/>2 mega-communities<br/>(broad)"]
    Leiden --> L1["Level 1:<br/>12 communities"]
    Leiden --> L2["Level 2:<br/>50 communities<br/>(narrow)"]
    L0 & L1 & L2 --> Sum["LLM summarizes<br/>each community"]
    Sum --> Store[(Community summaries)]
```

## Why Leiden, not spectral or k-means

The reference GraphRAG pipeline uses the [**Leiden algorithm**](https://www.nature.com/articles/s41598-019-41695-z) ([Traag, Waltman, van Eck 2019](https://www.nature.com/articles/s41598-019-41695-z)). Three reasons:

- **Hierarchical output** — Leiden naturally produces a tree of communities at multiple granularities. GraphRAG uses this directly to let queries pick the "level of zoom" they need
- **Resolution parameter** — tunable to taste; lower resolution = bigger communities, higher = smaller
- **Quality and speed** — strict improvement over the older Louvain algorithm in both modularity and runtime

A typical run on a 5k-node graph: hundreds of milliseconds, producing 3 levels with 2 / 12 / 50 communities.

## Summarizing each community

For every community at every level, an LLM is asked:

> *Here is a community of N entities and the relations among them. Produce a 1–2 paragraph summary describing what this community is about, the main entities, and notable relationships.*

The output is stored alongside the graph. At query time, **global search retrieves over these summaries**, not over the original chunks.

## Cost

Community summarization runs roughly **once per (community × level)**. For 64 total communities across 3 levels, ~64 LLM calls totaling a few thousand input tokens each. With Haiku, **single-digit dollars for the whole pass**.

## Tunable knobs

- **Resolution** — controls cluster size; defaults work for most corpora
- **Levels to summarize** — usually 2–3 is enough; deeper levels are diminishing returns
- **Summary length** — longer summaries cost more at retrieval; shorter risk losing nuance
- **Min community size** — too-small communities are noise; drop those below ~5 nodes

Sources

- [Traag et al. — Leiden algorithm (2019)](https://www.nature.com/articles/s41598-019-41695-z)
- [Edge et al. — §3.3–3.4 Graph Communities → Community Summaries](https://arxiv.org/abs/2404.16130)

---

# Entity-Centered Retrieval

**Local search** answers questions about a specific entity by gathering everything the graph knows about that entity and its immediate neighbors.

```mermaid
flowchart LR
    Q["'Tell me about Alice'"] --> Embed[Embed query]
    Embed --> Match[Find seed entity]
    Match --> Alice((Alice))
    Alice --> N1((Acme))
    Alice --> N2((Platform Team))
    Alice --> N3((Bob))
    Alice --> N4((Project Atlas))
    Alice -.-> Chunks["Source chunks<br/>that mention Alice"]
    N1 & N2 & N3 & N4 -.-> Chunks2["Chunks for<br/>1-hop neighbors"]
    Chunks & Chunks2 --> Context[Assembled context]
    Context --> LLM
    LLM --> Answer
```

## What goes into the context window

For a local query, the retriever assembles roughly:

1. **Description of the seed entity** itself
2. **Descriptions of 1-hop neighbors** (and optionally 2-hop, weighted by edge strength)
3. **Edge descriptions** along the path between seed and neighbor
4. **A handful of source chunks** that mention any of the above (back-references)

All ranked by relevance, packed into the available context budget.

## Code sketch

```python
def local_search(query: str, graph, embedder, llm) -> str:
    # 1. Find seed entity by embedding similarity
    seed = embedder.nearest_node(query, top_k=1)

    # 2. Expand to neighbors
    neighbors = graph.neighbors(seed.id, max_hops=2, max_count=20)

    # 3. Gather supporting chunks
    chunk_ids = set(seed.source_chunks)
    for n in neighbors:
        chunk_ids.update(n.source_chunks)
    chunks = fetch_chunks(chunk_ids, limit=10)

    # 4. Assemble + answer
    context = format_local_context(seed, neighbors, chunks)
    return llm.generate(query=query, context=context)
```

## When local search shines

- "Who is X?" or "What does X do?"
- "What are the dependencies of Y?"
- "What's wrong with project Z?" (recent issues, blockers, etc. that link to Z)

## When it falls short

- "What are the recurring themes across all the X?" — this is global, not local
- Questions where the model doesn't know the right seed entity from the wording (e.g., "the issue we hit last sprint" — fuzzy reference)

For the second case, hybrid retrieval (next section) helps: combine an embedding-based fallback with the graph traversal.

Sources

- [Edge et al. — §4.2 Local Search](https://arxiv.org/abs/2404.16130)
- [microsoft/graphrag — local search implementation](https://github.com/microsoft/graphrag)

---

# Map-Reduce Over Community Summaries

**Global search** answers corpus-wide questions ("what are the main themes?") by asking the LLM to answer the query *against each community summary*, then combining the partial answers.

```mermaid
flowchart TB
    Q[Query] --> Pick[Pick level<br/>of community<br/>granularity]
    Pick --> Comm1[Community 1 summary]
    Pick --> Comm2[Community 2 summary]
    Pick --> Comm3[Community 3 summary]
    Pick --> CommN[... Community N]
    Comm1 -->|map| P1[Partial answer + score]
    Comm2 -->|map| P2[Partial answer + score]
    Comm3 -->|map| P3[Partial answer + score]
    CommN -->|map| PN[Partial answer + score]
    P1 & P2 & P3 & PN -->|reduce| Final[Combined answer]
```

## The two-pass pattern

### Map pass

For each community summary at the chosen level, run the LLM:

```
You are answering the user's question using only the community summary below.
If the summary is not relevant, say so. If it is, produce a partial answer
and rate the partial answer's helpfulness on a scale of 1-100.

QUESTION: {query}
COMMUNITY SUMMARY: {summary}
```

This is parallelizable — N communities, N independent LLM calls.

### Reduce pass

Filter to the top-K partial answers by self-rating, then ask the LLM to synthesize:

```
You will combine the partial answers below into one final answer.
Resolve contradictions in favor of more recent / more specific information.
Cite which community each fact came from.

QUESTION: {query}
PARTIAL ANSWERS:
- [Community 3, score 87]: ...
- [Community 12, score 76]: ...
```

## Cost characterization

For a corpus that produced 50 communities at the chosen level, a global query costs **~50 small LLM calls** in the map pass plus one larger call in the reduce pass.

With Claude Haiku and a few-hundred-token average summary, a global query costs roughly **$0.05–$0.20** depending on community count. Not free; not nothing. Reserve it for queries that actually need it.

## Picking the level

| Question shape | Use level |
|----------------|-----------|
| "What are the broad themes?" | Top (Level 0) — handful of mega-communities |
| "What did each team focus on?" | Middle (Level 1) |
| "What did the platform team focus on?" | Bottom (Level 2) — narrower, more specific |

A query router or the user's choice picks the level. Some implementations let the model choose by retrieving summaries at multiple levels and picking the most useful.

Sources

- [Edge et al. — §4.1 Global Search](https://arxiv.org/abs/2404.16130)
- [microsoft/graphrag — global search implementation](https://github.com/microsoft/graphrag)

---

# The Reference Implementation

[microsoft/graphrag](https://github.com/microsoft/graphrag) is the canonical implementation of the [Edge et al. 2024](https://arxiv.org/abs/2404.16130) paper — open source, Python, batteries-included.

```mermaid
flowchart TB
    subgraph Index["Indexing pipeline (offline)"]
      direction LR
      Docs --> Chunk --> Extract --> Merge --> Comm --> Sum
    end
    subgraph Query["Query pipeline (online)"]
      direction LR
      Q --> Router{Local or<br/>global?}
      Router -->|local| LS[Local search]
      Router -->|global| GS[Global search]
      LS --> Ans
      GS --> Ans
    end
    Index -.->|parquet files,<br/>vector DB,<br/>nx graph| Query
```

## What you get out of the box

- **Indexing CLI** — `graphrag index --root ./project` runs the whole extraction → merge → community → summarization pipeline, configurable via YAML
- **Query CLI** — `graphrag query --method local "..."` or `--method global "..."`
- **Configurable LLMs** — OpenAI, Azure OpenAI, Anthropic, local Ollama via standard adapters
- **Configurable storage** — Parquet on disk by default; LanceDB, Azure Cosmos DB, or your own backend
- **Observability hooks** — every LLM call is logged and replayable

## Typical configuration

```yaml
# settings.yaml
llm:
  type: openai_chat
  model: gpt-4o-mini      # or claude-sonnet-4-6 via adapter
  api_key: ${OPENAI_API_KEY}

embeddings:
  llm:
    type: openai_embedding
    model: text-embedding-3-large

entity_extraction:
  entity_types: [Person, Organization, Product, Location]
  max_gleanings: 1

community_reports:
  max_length: 2000

cluster_graph:
  max_cluster_size: 10
```

## What it does well

- **Reproducibility** — outputs of every pipeline stage are versioned parquet files; you can re-run only the stage you changed
- **Eval baseline** — the repo ships with the eval harness from the paper, so you can compare your changes against the published numbers
- **Drop-in answer quality** — for global queries, "just run microsoft/graphrag with defaults" is already a strong baseline

## What it does poorly

- **Cost** — the full pipeline on a non-trivial corpus is dollars-to-tens-of-dollars, not cents. You'll feel it
- **Latency** — indexing a Wikipedia-sized corpus is hours, not minutes
- **Hot updates** — adding a new document means re-running affected entity extraction, then re-merging; no first-class "incremental insert"

These limitations are what spawned the next wave: LightRAG, LazyGraphRAG, LinearRAG, and others.

Sources

- [microsoft/graphrag](https://github.com/microsoft/graphrag)
- [Edge et al. — GraphRAG paper](https://arxiv.org/abs/2404.16130)

---

# Defer the Expensive Work

LazyGraphRAG ([Microsoft Research, Nov 2024](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/)) attacks the cost problem head-on: skip the up-front LLM extraction, and only do graph-style work for queries that need it.

```mermaid
flowchart LR
    subgraph Standard["GraphRAG (eager)"]
      D1[Docs] --> Extract1[LLM extract<br/>$$$]
      Extract1 --> Graph1[Full graph]
      Graph1 --> Q1[Query: cheap]
    end
    subgraph Lazy["LazyGraphRAG"]
      D2[Docs] --> Cluster[Stat-based<br/>concept clusters]
      Cluster --> Index2[Lightweight index]
      Q2[Query] --> ID[Detect relevant clusters]
      ID --> JIT[LLM extract<br/>only those clusters]
      JIT --> Ans2[Answer]
    end
```

## The trade

| | GraphRAG (eager) | LazyGraphRAG |
|---|---|---|
| Index cost | High (LLM per chunk) | Low (statistical only) |
| First query latency | Low | Higher (extracts at query time) |
| Subsequent queries on same area | Low | Low (cached extraction) |
| Quality on global queries | Strong baseline | Comparable or better in Microsoft's eval |
| Best for | Corpora you'll query many times | Corpora where most data is never queried |

Microsoft reports LazyGraphRAG matching GraphRAG quality at **0.1% of the indexing cost** on their benchmarks — and *exceeding* it on some categories of global queries.

## Why it works

Two insights:

1. **The graph is over-extracted in eager GraphRAG.** Most chunks contribute entities that no query ever asks about. The LLM work was wasted.
2. **Statistical concept clusters are good enough as a routing index.** You don't need a *correct* graph to know which slice of the corpus is relevant — embedding clustering plus a small bag-of-concepts works.

The actual graph extraction happens *at query time*, only over the chunks the router pulled in. Result: cheaper overall, especially on long-tail corpora.

## When eager still wins

- **You will query the corpus thousands of times.** Amortizing the index cost is fine
- **You need offline analytical workloads** ("show me the dependency graph") that benefit from the full graph existing
- **Latency-sensitive global queries.** Lazy approaches pay an LLM-extraction cost at query time; that's hard to hide

## Implementation notes

As of mid-2026 there is no official open-source LazyGraphRAG. The technique is described well enough in the [Microsoft blog post](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/) to reproduce; several community implementations exist with varying maturity.

Sources

- [Microsoft Research — LazyGraphRAG blog](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/)
- [Edge et al. — original GraphRAG (for comparison)](https://arxiv.org/abs/2404.16130)

---

# Dual-Level Retrieval

LightRAG ([Guo et al. 2024](https://arxiv.org/abs/2410.05779)) is a simpler, cheaper alternative to microsoft/graphrag with explicit dual-level retrieval and easier incremental updates.

```mermaid
flowchart LR
    Docs --> Extract[LLM:<br/>entities + relations]
    Extract --> Graph[Graph]
    Graph --> LowEmbed["Embed each entity<br/>(low-level)"]
    Graph --> HighEmbed["Embed each<br/>relation cluster<br/>(high-level)"]
    Q[Query] --> Low["Low-level keys<br/>(specific entities)"]
    Q --> High["High-level keys<br/>(broad concepts)"]
    Low --> LowEmbed --> Match1[Match]
    High --> HighEmbed --> Match2[Match]
    Match1 & Match2 --> Combined[Combined retrieval]
    Combined --> LLM --> Ans
```

## The dual-level idea

Most queries have **two retrieval needs at once**:

- **Specific** — "the Postgres outage on March 14" (low-level: specific entities)
- **Broad** — "outages in Q1" (high-level: temporal/category clusters)

LightRAG indexes both. The LLM rewrites the query into *low-level* and *high-level* keys; the retriever does **two** searches and merges the results.

## What's different vs microsoft/graphrag

| | microsoft/graphrag | LightRAG |
|---|---|---|
| Community detection | Yes (Leiden) | No |
| Hierarchical | Multiple levels | Two fixed levels |
| Incremental insert | Painful | First-class |
| Indexing cost | High | ~50% of microsoft/graphrag |
| Code complexity | Higher | Lower |

LightRAG's main contribution isn't a new algorithm — it's a **cheaper, simpler pipeline** that hits similar accuracy with less infrastructure.

## Incremental updates

LightRAG was designed with continuous corpora in mind. Adding a document:

1. Extract entities and relations from the new doc only
2. Find existing entities that match (string + embedding)
3. Merge; emit updated descriptions for affected nodes
4. Re-embed only the affected nodes

No global re-clustering, no community re-summarization. This is the main reason teams pick LightRAG over GraphRAG for live systems.

## Reference implementation

```python
from lightrag import LightRAG, QueryParam

rag = LightRAG(
    working_dir="./graph_storage",
    llm_model_func=claude_complete,        # any OpenAI-compatible / custom
    embedding_func=openai_embed_3_large,
)

# Indexing
await rag.ainsert(open("docs/2026-q1-report.md").read())

# Query — pick mode per call
ans = await rag.aquery("Tell me about Alice's role change",
                        param=QueryParam(mode="local"))
ans = await rag.aquery("Recurring themes from Q1 incidents",
                        param=QueryParam(mode="global"))
ans = await rag.aquery("Both",  # uses dual-level
                        param=QueryParam(mode="hybrid"))
```

Sources

- [Guo et al. — LightRAG paper](https://arxiv.org/abs/2410.05779)
- [HKUDS/LightRAG — official implementation](https://github.com/HKUDS/LightRAG)

---

# Memory Inspired by the Hippocampus

HippoRAG ([Gutiérrez et al. 2024](https://arxiv.org/abs/2405.14831)) borrows from neuroscience: the brain's hippocampus does pattern separation and pattern completion to retrieve memories. HippoRAG implements an analog using **Personalized PageRank** over an entity graph.

```mermaid
flowchart LR
    Q[Query] --> NER[Extract query entities]
    NER --> Seeds["Seed nodes<br/>(weighted by query)"]
    Seeds --> PPR[Personalized PageRank<br/>over entity graph]
    Graph[(Entity graph)] --> PPR
    PPR --> Ranked["Nodes ranked by<br/>PPR score"]
    Ranked --> Chunks["Source chunks<br/>of top-ranked nodes"]
    Chunks --> LLM --> Ans
```

## How PageRank fits

[Personalized PageRank](https://nlp.stanford.edu/IR-book/html/htmledition/topic-specific-pagerank-1.html) is the same algorithm that ranks web pages, but biased toward "seed" nodes. With seeds set to the entities in the user's query, the algorithm gives high scores to **entities reachable from many of the query's seeds** — a graph-aware notion of relevance.

This handles **multi-hop questions naturally**: if you ask about Alice and Q3 launches, entities that connect to both bubble up to the top.

## What's different

| | microsoft/graphrag | HippoRAG |
|---|---|---|
| Index | Communities + summaries | Just the entity graph |
| Retrieval | Map-reduce over communities | PageRank traversal |
| Indexing cost | High | Moderate |
| Multi-hop quality | Good (via global search) | Particularly strong |
| Code complexity | Higher | Lower |

In the paper's eval, HippoRAG **outperforms vector RAG and matches or beats GraphRAG on multi-hop benchmarks** (HotpotQA, MuSiQue, 2WikiMultiHopQA), at a fraction of the indexing cost — because there are no community summaries to compute.

## Mechanism in one paragraph

Extract entities and relations from the corpus as usual. At query time, extract entities from the *query*; use them as Personalized PageRank seeds; run a few iterations; rank nodes by score; fetch source chunks for the top-K. Pass to the LLM.

## When to reach for HippoRAG

- The query mix is heavy on multi-hop reasoning
- The corpus has lots of cross-entity relationships
- You want graph behavior without community-summarization overhead

When *not* to:

- The corpus is small enough that vector RAG isn't a bottleneck
- Queries are mostly point lookups
- You need explicit "themes across the corpus" — HippoRAG doesn't summarize communities

Sources

- [Gutiérrez et al. — HippoRAG paper (2024)](https://arxiv.org/abs/2405.14831)
- [Personalized PageRank explainer](https://nlp.stanford.edu/IR-book/html/htmledition/topic-specific-pagerank-1.html)
- [OSU-NLP-Group/HippoRAG — official repo](https://github.com/OSU-NLP-Group/HippoRAG)

---

# Vector + Graph + Keyword

In production, almost no one runs GraphRAG **alone**. The winning pattern is hybrid: a vector index, a graph index, and (often) a keyword index — combined by a small ranker.

```mermaid
flowchart TB
    Q[Query] --> Router{"Router<br/>(LLM or rules)"}
    Router -->|local| V[Vector ANN]
    Router -->|specific entity| G[Graph traversal]
    Router -->|exact term| K[BM25 keyword]
    V & G & K --> Merge[Reciprocal rank fusion]
    Merge --> Rerank["Cross-encoder rerank<br/>(optional)"]
    Rerank --> LLM
```

## Why hybrid wins

Each index makes a different kind of mistake:

- **Vector** — misses exact-match terms ("error code E_421"); fails on numerical or code-like content
- **Graph** — depends on entity extraction quality; fails on truly novel entities not yet in the graph
- **Keyword** — exact-string fragile; brittle to paraphrase

Combining them recovers most failures. The standard merger is [**reciprocal rank fusion**](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf): take the top-K from each retriever, score documents by `Σ 1 / (k + rank_i)`, sort. No tuning needed.

## Routing — when do you need graph at all?

A simple classifier (or a tiny LLM call) per query is cheap:

```python
ROUTE_PROMPT = """Classify the user's query into ONE of:
- local: asks about a specific entity or document
- global: asks for themes, summaries, or aggregations across the corpus
- factual: needs an exact value, name, code, or identifier

QUERY: {query}
"""

route = classify(query)
match route:
    case "local":   return graph_local_search(query)
    case "global":  return graph_global_search(query)
    case "factual": return hybrid(vector, keyword)(query)
```

In practice, **70% of queries are "factual" or "local" lookups** for which vector or vector+keyword is sufficient. Save graph for the global / multi-hop slice.

## Production reference architecture

```mermaid
flowchart LR
    Q --> Cache{Cache hit?}
    Cache -->|yes| Ans1[Cached]
    Cache -->|no| Route[Route]
    Route --> Retrieve[Hybrid retrieve]
    Retrieve --> Filter["Safety / policy filter"]
    Filter --> LLM
    LLM --> Stream[Stream to user]
    Stream --> Store[Store for cache]
```

Most teams that ran GraphRAG in 2024–25 ended up with this shape by 2026.

Sources

- [Cormack et al. — Reciprocal Rank Fusion (SIGIR 2009)](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf)
- [Anthropic — Contextual Retrieval](https://www.anthropic.com/news/contextual-retrieval)
- [Microsoft GraphRAG — query routing examples](https://github.com/microsoft/graphrag)

---

# What Does GraphRAG Actually Cost?

GraphRAG's LLM-heavy indexing is the headline cost. Concrete numbers for a 10 million-token corpus using Claude Haiku 4.5 (~$1 / $5 per M tokens, May 2026):

```mermaid
flowchart LR
    subgraph Index["Indexing (one-time per re-build)"]
      E["Extraction<br/>~$10–$15"]
      M["Merging<br/>~$2"]
      C["Community summaries<br/>~$1"]
    end
    subgraph Query["Per query"]
      L["Local search<br/>~$0.001"]
      G["Global search<br/>~$0.05–$0.20"]
    end
    Index --> Total["Total index ~$15–$20"]
    Query --> ATotal["Avg ~$0.02 / query<br/>(if 90% local, 10% global)"]
```

## Indexing breakdown

| Stage | Why it costs that |
|-------|-------------------|
| Entity + relation extraction | 1 LLM call per chunk × 10k chunks |
| Gleaning (1 round) | Another 30% of chunks |
| Description merging | ~1 call per merged entity, fewer than total |
| Community detection | Cheap (algorithmic, no LLM) |
| Community summarization | ~1 call per community per level (~64 calls) |

**Most of the cost is extraction.** Halving the gleaning rounds, dropping low-information chunks, or using a smaller extraction model are the highest-leverage cost cuts.

## Per-query cost

Local: a few thousand tokens of context, one LLM call. **Cents at most.**

Global: scales with number of communities at the chosen level. **5–20¢ per query** is typical. This is where most of the operational cost lives in production.

## How to cut cost

| Lever | Magnitude |
|-------|-----------|
| Use Haiku for extraction, not Opus | 5–10× |
| Skip gleaning, single-pass extraction | 1.3× |
| Drop chunks below a relevance threshold | 1.5–2× |
| Cache global-query results by semantic hash | depends on traffic |
| Switch to LazyGraphRAG (extract on demand) | ~1000× on index, 2–5× on query |
| Switch to LightRAG (no community summaries) | 1.5–2× on index |

## When the math doesn't work out

If your traffic is < 100 queries/day and your corpus is < 100k tokens, **don't use GraphRAG**. The indexing cost dominates per-query value. A naive vector RAG is fine.

If your traffic is > 10k queries/day with significant global-query mix, the indexing cost amortizes well — and the bottleneck shifts to query-time cost. That's when caching and LazyGraphRAG start mattering.

Sources

- [Anthropic — Claude pricing](https://www.anthropic.com/pricing)
- [Microsoft Research — LazyGraphRAG cost analysis](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/)

---

# Picking the Right Retrieval Strategy

```mermaid
flowchart TB
    Start[New project] --> Q1{Corpus has<br/>strong entity<br/>structure?}
    Q1 -->|no| V[Vector RAG]
    Q1 -->|yes| Q2{Workload includes<br/>multi-hop or<br/>global queries?}
    Q2 -->|no| V
    Q2 -->|yes| Q3{Corpus updates<br/>frequently?}
    Q3 -->|no| Q4{Cost-sensitive?}
    Q3 -->|yes| LR[LightRAG<br/>incremental updates]
    Q4 -->|yes| LZ[LazyGraphRAG<br/>defer extraction]
    Q4 -->|no| MS[microsoft/graphrag<br/>reference, full features]
    V --> Hybrid["+ keyword (BM25)<br/>+ reranker"]
    LR & LZ & MS --> HybridG["+ vector fallback<br/>for fuzzy lookups"]
```

## Decision questions in plain English

### Does your corpus have entity structure?

Yes for: customer records, code repos, knowledge bases, research libraries, project documents.
No (mostly) for: blog posts, marketing copy, fiction, unstructured chat logs.

### Do users ask multi-hop or global questions?

Look at real traffic. Sample 100 queries; classify. If <10% are non-local, GraphRAG's indexing cost is hard to justify.

### How often does the corpus change?

Daily: LightRAG or a custom incremental pipeline.
Weekly+: any GraphRAG works; re-index nightly is fine.
Mostly static: even eager microsoft/graphrag is reasonable.

### Cost-sensitive?

How much is "$10–$50 per index pass + few cents per query" worth to the answer-quality lift? For internal-tool quality bars, often nothing. For customer-facing search with measurable retention impact, often a lot.

## Common combinations seen in production

| Use case | Pattern |
|----------|---------|
| Internal docs Q&A | Vector + keyword, no graph |
| Code understanding agent | Vector + graph (via AST) + keyword |
| Legal/research review | LightRAG hybrid with citations |
| Customer-support routing | Vector RAG + small rule layer; graph for analytics |
| Org-wide knowledge agent | LazyGraphRAG with vector fallback |

## A practical test

Before committing to GraphRAG, do this:

1. Build a vector-RAG baseline. Measure quality with LLM-judge on 50 real queries
2. Hand-annotate the 10 worst failures
3. Could those failures be answered by traversing entities or summarizing communities? If yes → GraphRAG is the right next investment. If no → fix retrieval ranking or chunking instead

Sources

- [Edge et al. — GraphRAG eval methodology](https://arxiv.org/abs/2404.16130)
- [Anthropic — RAG without the cargo cult](https://www.anthropic.com/news/contextual-retrieval)

---

# How to Measure GraphRAG

Standard RAG eval (top-K hit rate, MRR) doesn't capture what GraphRAG promises. Global queries don't have a single "correct passage" to retrieve.

```mermaid
flowchart TB
    Eval["GraphRAG eval"]
    Eval --> Local["Local queries:<br/>standard RAG metrics"]
    Eval --> Global["Global queries:<br/>LLM-as-judge rubric"]
    Local --> M1[Top-K hit rate]
    Local --> M2[MRR]
    Local --> M3[Answer accuracy]
    Global --> M4[Comprehensiveness]
    Global --> M5[Diversity]
    Global --> M6[Empowerment]
    Global --> M7[Directness]
```

## The four global-query rubric dimensions

Microsoft's eval ([Edge et al. §4.3](https://arxiv.org/abs/2404.16130)) uses LLM-as-judge to score answers on four dimensions:

- **Comprehensiveness** — does the answer cover the breadth of the question?
- **Diversity** — does it surface multiple perspectives where applicable?
- **Empowerment** — does it help the user understand and reason further?
- **Directness** — does it answer the question, or wander?

The judge gets *both* answers (baseline vs GraphRAG) and picks a winner per dimension. Aggregate wins gives you the per-dimension lift.

## Building your own eval set

You can't evaluate what you can't measure. To get a real number:

1. **Sample 100–200 real queries** from production logs (anonymized)
2. **Classify each by question type** (local fact, multi-hop, global summary, etc.)
3. **Hand-label "good" answers** for ~30 of them (the expensive step)
4. **Run baselines and variants** through the same questions
5. **Score with LLM-as-judge** using a fixed rubric. Use a different model from the one being evaluated, to avoid bias

For local queries, you can use exact-match or contains-match on the hand-labeled answer. For global queries, use the four-dimension rubric.

## What to watch for

- **Eval contamination** — if your judge model trained on the corpus you're evaluating, scores are inflated
- **Question shape skew** — your eval should match the production query mix, not be all global queries
- **Cost per eval run** — running 200 queries × 4 systems × LLM-judge is real money. Cache aggressively

## A minimal eval harness

```python
@dataclass
class EvalCase:
    query: str
    type: Literal["local", "global", "multi_hop"]
    gold_answer: str | None = None

async def eval_run(systems: dict[str, RAGSystem], cases: list[EvalCase]):
    results = []
    for case in cases:
        answers = {name: await sys.answer(case.query) for name, sys in systems.items()}
        for name, ans in answers.items():
            score = await judge(case, ans)
            results.append({"system": name, "case": case.query, **score})
    return results
```

Sources

- [Edge et al. — §4.3 Eval methodology](https://arxiv.org/abs/2404.16130)
- [Ragas — open-source RAG eval framework](https://github.com/explodinggradients/ragas)
- [Anthropic — evaluating LLM applications](https://docs.claude.com/en/docs/test-and-evaluate/develop-tests)

---

# Common Ways to Waste Money on GraphRAG

## Anti-pattern 1: "Just turn on GraphRAG"

Running microsoft/graphrag with defaults on a corpus your team has never queried, then declaring it "improved retrieval" because anecdotes feel good.

- **What's wrong:** No baseline to compare against; no eval set; cost is now 100×; no one can prove (or disprove) whether it helps
- **Fix:** Always have a vector-RAG baseline + a measured eval set *before* turning on GraphRAG

## Anti-pattern 2: Building a graph for a corpus that doesn't have entities

A folder of well-written technical blog posts. The pipeline runs; the graph is mostly singleton nodes; community detection produces 200 tiny communities; global queries are worse than the vector baseline.

- **Fix:** Look at your extracted graph. If average degree is < 2, your corpus doesn't have the right shape for GraphRAG. Stay on vector + keyword

## Anti-pattern 3: Treating it as a vector-RAG replacement

GraphRAG is at its best on global and multi-hop queries; it's at best parity with vector RAG on local lookups. Replacing vector entirely makes the common case (point lookups) **worse**.

- **Fix:** Hybrid. Always.

## Anti-pattern 4: Re-indexing the whole corpus on every change

A 10M-token corpus with 5 new documents/day. Re-running the full pipeline nightly. Costs scale linearly with corpus size, not change size.

- **Fix:** LightRAG, or roll your own incremental: re-extract only the new chunks, merge into the existing graph, re-summarize only affected communities

## Anti-pattern 5: Believing the eval benchmarks transfer

The Microsoft GraphRAG paper showed +20–80% on **query-focused summarization** over **news/podcast transcripts**. Your corpus is engineering wikis and your queries are mostly "what does this code do" — those benchmarks tell you very little.

- **Fix:** Build your own eval set with your queries and your corpus. Always.

## Anti-pattern 6: Trusting the graph too much

The LLM extracted "Alice manages Bob" from a chunk that actually said "Alice mentioned Bob in passing". The graph now has a fabricated managerial relationship. Global queries about reporting structure surface fiction.

- **Fix:** Require the extractor to quote source spans; sample-audit the extracted graph; weight edges by extraction confidence

## Anti-pattern 7: Ignoring drift

A graph indexed in January is queried in November. Entities have been renamed, restructured, deprecated. Answers reflect history, not present.

- **Fix:** Time-stamp every chunk and entity; weight recent over old; re-index periodically; consider versioned graphs for audit-sensitive domains

Sources

- [Edge et al. — limitations section](https://arxiv.org/abs/2404.16130)
- [Microsoft Research — LazyGraphRAG analysis](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/)

---

# Open Problems and Active Research

GraphRAG circa 2024 was a single technique. By 2026 it's an active research area with several open fronts.

## Incremental and streaming GraphRAG

The biggest practical gap: efficient updates. LightRAG's per-doc incremental is a start; LazyGraphRAG sidesteps the problem differently; nobody has cracked "continuous streaming corpora with sub-second update latency to the graph". Active work in 2026 papers.

## Cross-corpus graphs

Most graphs are per-corpus, per-tenant. A common ask in enterprise: "graph that spans my internal wiki, my customer-support tickets, and my code." Tooling for safe cross-corpus graphs (auth, deduplication, governance) is immature.

## Causal and temporal edges

Today's GraphRAG edges are mostly co-occurrence: "Alice and Bob are mentioned together." Causal edges ("the postponement of Project X caused the reorg of Team Y") and temporal edges ("Alice managed Bob from Jan to Aug; now Alice reports to Carol") are not first-class. Several papers in 2025 propose typed relation extraction with temporal grounding.

## Multi-modal graphs

The text-extraction pipeline ignores images, diagrams, code, tables. A diagram in a slide deck *is* a graph; extracting and merging it with the text graph is open. The 2026 frontier models that handle high-resolution vision ([Claude Opus 4.7](https://www.anthropic.com/news/claude-opus-4-7)) make this newly tractable.

## Learned routing

The "local vs global" router is usually hand-tuned. Several papers propose learning the router from query/answer pairs, with measurable gains on heterogeneous workloads.

## Compact graph representations

A 5M-node graph doesn't fit in a context window. Current approaches summarize communities; future approaches might **compress the graph itself** into a learned representation that the model can attend to directly. Early work on this is promising but very early.

```mermaid
flowchart LR
    Now["GraphRAG circa 2024:<br/>extract → cluster → summarize → query"]
    Now --> Inc["+ Incremental updates"]
    Now --> Cross["+ Cross-corpus + governance"]
    Now --> Temp["+ Temporal + causal edges"]
    Now --> MM["+ Multi-modal"]
    Now --> LR["+ Learned routing"]
    Now --> Comp["+ Compact representations"]
```

## What you can take to industry now

- **microsoft/graphrag** for static, single-corpus, batch-indexed
- **LightRAG** for incremental and cost-sensitive
- **HippoRAG** for multi-hop-heavy workloads
- **Hybrid with vector + keyword** in all cases
- A **measured eval harness** in all cases

Sources

- [Edge et al. — GraphRAG paper](https://arxiv.org/abs/2404.16130)
- [Microsoft Research — LazyGraphRAG](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/)
- [Guo et al. — LightRAG](https://arxiv.org/abs/2410.05779)
- [Gutiérrez et al. — HippoRAG](https://arxiv.org/abs/2405.14831)

---

# What to Remember

## Five claims

1. **GraphRAG indexes relationships, not just chunks.** The LLM does extra work at index time so retrieval can traverse a graph at query time
2. **Local and global search are different code paths.** Local = entity-centered drill-down. Global = map-reduce over community summaries
3. **The biggest cost is indexing**, not querying. Most production trouble is about making that cost amortize
4. **Hybrid wins.** Vector + graph + keyword + reranker is the actual production architecture. Pure GraphRAG is for special-purpose workloads
5. **The benchmarks don't transfer.** Build your own eval set with your queries and corpus before committing — and especially before claiming GraphRAG "works" for your case

## What you should be able to do after this lecture

- Sketch the indexing pipeline (chunks → extract → merge → cluster → summarize)
- Decide whether a query is local or global
- Read a community summary and explain how it would be used in global search
- Pick between microsoft/graphrag, LightRAG, LazyGraphRAG, HippoRAG given constraints
- Identify cost and quality trade-offs of each
- Write a 50-line PageRank-on-entity-graph baseline if asked

## What's next

- **Week 7 — Evaluation & LLM-as-a-Judge.** The eval harness you need to actually measure whether GraphRAG (or any retrieval change) helps
- **Week 11 — HITL & Memory.** Long-term memory is graph-shaped; many of the same patterns apply

Sources

- [Edge et al. — GraphRAG paper](https://arxiv.org/abs/2404.16130)
- [Microsoft Research — LazyGraphRAG](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/)
- [Guo et al. — LightRAG](https://arxiv.org/abs/2410.05779)
- [Gutiérrez et al. — HippoRAG](https://arxiv.org/abs/2405.14831)
- [microsoft/graphrag](https://github.com/microsoft/graphrag)
- [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG)