---
title: "The Self-Correcting RAG: Implementing Agentic and Recursive Retrieval Loops"
description: "Static RAG retrieves once and assumes top-k results contain complete context. Recursive retrieval loops inspect accumulated context, identify missing facts, synthesize targeted follow-up queries, and terminate on explicit criteria or iteration bounds."
date: "November 18, 2025"
url: "https://himanshuat.com/blogs/self-correcting-rag-agentic-recursive-retrieval"
---
# The Self-Correcting RAG: Implementing Agentic and Recursive Retrieval Loops

A single-shot search returns five candidate chunks. The model synthesizes a fluent paragraph, but the output fails in the specific area where those five passages were silent.

Standard RAG architectures execute sequentially: embed query, retrieve top-$k$, construct prompt, generate answer. This assumes the initial search step achieved high recall and precision.

> Single-shot pipelines cannot detect when initial retrieval is incomplete. Reasoning over retrieved context requires evaluating whether current information satisfies query constraints before generating.

Multi-hop queries and complex technical questions require iterative investigation: identify missing evidence, formulate targeted secondary queries, and aggregate evidence across retrieval steps.

### Chapter 0: Anatomy of an iterative retrieval loop

An agentic retrieval loop connects retrieval and generation into a cyclical state machine.

Embed the user query, run an initial vector search, then hand the accumulated context back to an evaluator along with the original query. The evaluator answers one question: does this context already support a complete answer? If not, it names the gap and writes a follow-up query. Synthesis runs once the evaluator says COMPLETE or the iteration cap trips.

```mermaid
flowchart TD
  Q[user query] --> E[embed + search]
  E --> C[accumulate context]
  C --> J{context sufficient?}
  J -->|NEEDS_MORE_INFO| F[write follow-up query]
  F --> E
  J -->|COMPLETE| S[synthesize response]
```

---

### Step 1: Wrap vector retrieval into an invokable primitive

The retriever has to be callable from inside the loop, not wired into a one-shot pipeline.

```typescript
interface VectorDB {
    search: (embedding: number[], k: number) => Promise<Document[]>;
}

interface Document {
    id: string;
    content: string;
    metadata?: Record<string, any>;
    embedding?: number[];
}

const getEmbedding = async (text: string): Promise<number[]> => {
    const response = await fetch('https://api.openai.com/v1/embeddings', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
        },
        body: JSON.stringify({
            input: text,
            model: 'text-embedding-3-small',
        }),
    });

    if (!response.ok) {
        throw new Error(`Embedding API error: ${response.statusText}`);
    }
    const data = await response.json();
    return data.data[0].embedding;
};
```

Parameterized $k$ values allow broad candidate exploration on the initial pass ($k=5$) and targeted precision retrieval on follow-ups ($k=2$ or $3$).

---

### Step 2: Enforce structured schema validation on evaluation calls

Loop control needs a deterministic output from the model, not freeform text. Pin it to a JSON schema:

```typescript
type LLMCallOptions = {
    model: string;
    temperature?: number;
    max_tokens?: number;
};

const callLLM = async (
    messages: Array<{ role: 'system' | 'user' | 'assistant', content: string }>, 
    options: LLMCallOptions
): Promise<string> => {
    const response = await fetch('https://api.openai.com/v1/chat/completions', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
        },
        body: JSON.stringify({
            model: options.model,
            messages: messages,
            temperature: options.temperature ?? 0.0,
            max_tokens: options.max_tokens,
            response_format: { type: "json_object" }
        }),
    });

    if (!response.ok) {
        throw new Error(`LLM API error: ${response.statusText}`);
    }
    const data = await response.json();
    return data.choices[0].message.content;
};
```

---

### Step 3: Extract structured critique and follow-up queries

The evaluation prompt requires three fields: termination status, a critique explaining missing context, and a specific search query designed to resolve the gap. Tracking previous search queries prevents the model from repeating redundant lookups.

```typescript
interface AgentDecision {
    status: 'COMPLETE' | 'NEEDS_MORE_INFO';
    followUpQuery?: string;
    critique?: string;
}

const getAgentDecisionPrompt = (
    originalQuery: string, 
    currentContext: string[], 
    accumulatedSearchHistory: string[], 
    maxTokens: number
): Array<{ role: 'system' | 'user', content: string }> => [
    {
        role: 'system',
        content: `You are an evaluation agent in a retrieval pipeline. Evaluate the supplied context against the user query.
        Determine if current context contains sufficient information to produce a complete, factual answer.
        If sufficient, return status: 'COMPLETE'.
        If insufficient, return status: 'NEEDS_MORE_INFO', provide a concise 'critique' of missing data, and formulate a targeted 'followUpQuery'.
        Output must be a valid JSON object matching the schema: {"status": "COMPLETE" | "NEEDS_MORE_INFO", "critique": string, "followUpQuery": string}.`
    },
    {
        role: 'user',
        content: `Original Query: "${originalQuery}"
        
        ---
        Accumulated Context:
        ${currentContext.length > 0 ? currentContext.map((doc, i) => `Document ${i + 1}:\n${doc}`).join('\n---\n') : 'No context yet.'}
        
        ---
        Search History:
        ${accumulatedSearchHistory.length > 0 ? accumulatedSearchHistory.join('\n') : 'No previous searches.'}
        
        Evaluate context completeness:`
    }
];

const evaluateContext = async (
    originalQuery: string,
    currentContext: string[],
    accumulatedSearchHistory: string[],
    llmOptions: LLMCallOptions,
    maxTokensForFinalAnswer: number
): Promise<AgentDecision> => {
    const prompt = getAgentDecisionPrompt(originalQuery, currentContext, accumulatedSearchHistory, maxTokensForFinalAnswer);
    const responseJson = await callLLM(prompt, llmOptions);
    try {
        return JSON.parse(responseJson) as AgentDecision;
    } catch (error) {
        console.error("Failed to parse agent decision JSON:", responseJson, error);
        return {
            status: 'NEEDS_MORE_INFO',
            followUpQuery: `Targeted search for "${originalQuery}" specifics`,
            critique: "JSON parse fallback triggered."
        };
    }
};
```

---

### Step 4: Bound loop execution with strict termination limits

Unbounded recursive loops risk infinite recursion and runaway API costs. Enforce hard limits on iteration counts, token accumulation, and search width:

```typescript
const MAX_RAG_ITERATIONS = 3;
const FINAL_ANSWER_MAX_TOKENS = 1500;

interface RecursiveRAGResult {
    answer: string;
    iterations: number;
    finalContext: string[];
    searchHistory: string[];
}

const runRecursiveRAG = async (
    userQuery: string,
    vectorDB: VectorDB,
    llmOptions: LLMCallOptions,
    k_initial: number = 5,
    k_followup: number = 3
): Promise<RecursiveRAGResult> => {
    let accumulatedContext: string[] = [];
    let searchHistory: string[] = [];
    let currentIteration = 0;
    let currentSearchQuery = userQuery;

    while (currentIteration < MAX_RAG_ITERATIONS) {
        currentIteration++;
        searchHistory.push(currentSearchQuery);

        const embedding = await getEmbedding(currentSearchQuery);
        const docs = await vectorDB.search(embedding, currentIteration === 1 ? k_initial : k_followup);
        const newContexts = docs.map(d => d.content);

        accumulatedContext = Array.from(new Set([...accumulatedContext, ...newContexts]));

        const decision = await evaluateContext(
            userQuery,
            accumulatedContext,
            searchHistory,
            llmOptions,
            FINAL_ANSWER_MAX_TOKENS
        );

        if (decision.status === 'COMPLETE') {
            break;
        } else {
            currentSearchQuery = decision.followUpQuery || userQuery;
        }
    }

    const finalSynthesisPrompt = [
        {
            role: 'system' as const,
            content: `Synthesize a concise, accurate answer based strictly on the provided context. If facts are incomplete, state the omission explicitly.`
        },
        {
            role: 'user' as const,
            content: `Original Query: "${userQuery}"\n\nContext:\n${accumulatedContext.join('\n---\n')}`
        }
    ];

    const finalAnswer = await callLLM(finalSynthesisPrompt, { ...llmOptions, max_tokens: FINAL_ANSWER_MAX_TOKENS });

    return {
        answer: finalAnswer,
        iterations: currentIteration,
        finalContext: accumulatedContext,
        searchHistory: searchHistory
    };
};
```

---

### Step 5: Route evaluation and synthesis to appropriate model tiers

The evaluation step performs a narrow classification task across iterations, while the synthesis step generates the final response.

Using a fast, lightweight model (such as `gpt-4o-mini` or `claude-3-haiku`) for context evaluation reduces per-iteration latency. Reserve larger parameter models for the final synthesis pass.

---

### Key production failure modes

- **Repetitive search queries:** If the model fails to incorporate previous search history, follow-up iterations retrieve identical chunks. Include search logs in evaluation prompts.
- **Context dilution:** As documents accumulate across iterations, irrelevant text can overwhelm LLM attention. Apply context compression, deduplication, or cross-encoder reranking prior to final synthesis.
- **Latency overhead:** Each iteration adds sequential network calls (embedding generation, vector query, LLM evaluation). Measure end-to-end latency and verify that accuracy gains justify the multi-second execution cost.

---

## When this is the wrong choice

- **The question is single-hop.** If the answer sits in one document, the first search finds it and the evaluator returns `COMPLETE` on iteration one. You paid an extra embedding call, a vector query, and an LLM classification to be told the pipeline was already done.
- **You have a tight latency budget.** With `MAX_RAG_ITERATIONS = 3`, the worst case is three embedding calls, three vector queries, three evaluator calls, and a synthesis call, all sequential. That ceiling is the number to hold against your p99, not the average.
- **Retrieval is broken rather than incomplete.** The loop assumes a follow-up query will surface what the first one missed. If the index is badly chunked or the embedding model is wrong for the corpus, every iteration searches the same broken space and the evaluator just spends money confirming it.

---

Source: https://himanshuat.com/blogs/self-correcting-rag-agentic-recursive-retrieval