Chapter 7 — Project P7 — AI-Powered Backend with RAG

Why this project

Building a single AI feature — a chatbot, a summarizer, a code reviewer — teaches you one pattern. Building an AI Code Review Assistant that combines LLM integration, RAG, streaming, agents, cost tracking, and security defenses teaches you the entire AI backend stack. This project is the culmination of L7: every concept from chapters 1-6, wired together into a working system that you'd be proud to show in a senior backend interview.

You've learned LLM APIs, embeddings, streaming, RAG, agents, and AI security as individual chapters. This project forces you to make them work together — which is where the real learning happens. The AI Code Review Assistant isn't a toy. It's the same pattern used by GitHub Copilot code review, CodeRabbit, and every other AI-powered dev tool: accept code, analyze it with LLMs enhanced by project-specific context, stream results back, and track everything.

By the end of this project, you'll have built a backend that: accepts a GitHub PR or code snippet, retrieves relevant project context (coding standards, past reviews, file history) via RAG, runs multiple AI agents in parallel (bug detector, style checker, security auditor), streams findings to the client in real-time, and tracks cost per review with prompt injection defenses at every layer.

Spec: what you're building

AI Features

1. Code Review Agent: Accepts a code diff or snippet and returns structured feedback: bugs found, style violations, security concerns, and suggestions — each with severity, line reference, and explanation.
2. RAG-enhanced context: Before reviewing, the system retrieves: the project's coding standards (from a stored document), similar past reviews (from pgvector), and relevant documentation (from ingested docs).
3. Streaming responses: Review results stream to the client via Server-Sent Events (SSE) as the AI generates them — users see findings appear in real-time, not a 30-second spinner.
4. Multi-agent pipeline: Three specialist agents run in parallel: Bug Detector (finds logical errors, null references, race conditions), Style Checker (enforces naming conventions, structure, best practices), and Security Auditor (flags SQL injection, XSS, hardcoded secrets, unsafe deserialization).
5. Cost tracking dashboard: Every review logs the model used, tokens consumed, cost incurred, and cache hit/miss. A /costs endpoint returns daily/weekly/monthly breakdowns.
6. Prompt injection defenses: User-submitted code is treated as potentially hostile. Input is sanitized, wrapped in delimiters, and validated before reaching the LLM.

API Endpoints

POST   /api/reviews              Submit code for review (returns review ID, begins streaming)
GET    /api/reviews/:id/stream   SSE endpoint — stream review results as they're generated
GET    /api/reviews/:id          Get complete review with all findings
GET    /api/reviews              List past reviews (paginated, filterable)
POST   /api/documents/ingest     Ingest a project document into the RAG index
GET    /api/costs                Get cost breakdown (daily, by model, by feature)
GET    /api/health               Health check

Success Criteria

• Submit a 200-line TypeScript file with intentional bugs → receive a review identifying at least 70% of the planted issues
• Review results begin streaming within 2 seconds of submission
• RAG retrieval enhances reviews with project-specific context (style guide rules appear in the Style Checker's output)
• Cost per review is tracked to the cent and visible on /api/costs
• Prompt injection attempts in code comments are blocked, not executed
• The entire system runs with docker compose up

Project layout

ai-code-reviewer/
├── docker-compose.yml
├── .env.example
├── package.json
├── tsconfig.json
├── src/
│   ├── index.ts                    # Express/Fastify app entry point
│   ├── config/
│   │   └── env.ts                  # Environment variable validation (zod)
│   ├── routes/
│   │   ├── reviews.ts              # POST /reviews, GET /reviews/:id
│   │   ├── stream.ts               # GET /reviews/:id/stream (SSE)
│   │   ├── documents.ts            # POST /documents/ingest
│   │   └── costs.ts                # GET /costs
│   ├── llm/
│   │   ├── gateway.ts              # AI Gateway — unified LLM provider interface
│   │   ├── providers/
│   │   │   ├── openai.ts           # OpenAI provider adapter
│   │   │   └── anthropic.ts        # Anthropic provider adapter
│   │   ├── model-router.ts         # Routes queries to appropriate model tier
│   │   └── cost-tracker.ts         # Tracks and reports AI costs
│   ├── rag/
│   │   ├── indexer.ts              # Document chunking + embedding + pgvector insert
│   │   ├── retriever.ts            # Vector search + reranking
│   │   └── embeddings.ts           # Embedding generation (OpenAI/text-embedding-3-small)
│   ├── agents/
│   │   ├── orchestrator.ts         # Coordinates specialist agents
│   │   ├── bug-detector.ts         # Bug detection agent
│   │   ├── style-checker.ts        # Style/pattern agent
│   │   ├── security-auditor.ts     # Security review agent
│   │   └── base-agent.ts           # Shared agent loop + tool definitions
│   ├── streaming/
│   │   └── sse-manager.ts          # SSE connection management

Phase 1: Unified LLM Provider Layer

Before any AI feature works, you need a clean abstraction over LLM providers. This layer handles: provider-agnostic interface, API key management, error handling with retries, and response parsing.

// === src/llm/gateway.ts ===
// The single entry point for all LLM calls in the application
 
import { OpenAIProvider } from "./providers/openai";
import { AnthropicProvider } from "./providers/anthropic";
import { CostTracker } from "./cost-tracker";
 
export type ProviderName

// === src/llm/providers/openai.ts ===
// OpenAI-specific provider implementation
 
import OpenAI from "openai";
import type { LLMProvider, CompletionRequest, CompletionResponse } from "../gateway";
 
export class OpenAIProvider implements LLMProvider {
  name

Phase 2: RAG Pipeline — Document Ingestion + Retrieval

The RAG pipeline gives the AI agents project-specific context. Instead of reviewing code against generic best practices, they review against YOUR team's coding standards, YOUR past review history, and YOUR documentation.

// === src/rag/indexer.ts ===
// Document ingestion: chunk → embed → store in pgvector
 
import { getEmbedding } from "./embeddings";
import { sql } from "../db/client";
 
interface ChunkConfig {
  chunkSize: number;    // Target characters per chunk

// === src/rag/retriever.ts ===
// Vector search + reranking for relevant context retrieval
 
import { getEmbedding } from "./embeddings";
import { sql } from "../db/client";
 
interface RetrievalResult {
  content: string;
  documentTitle: string

-- === src/db/migrations/001_initial.sql ===
-- Enable pgvector extension and create core tables
 
CREATE EXTENSION IF NOT EXISTS vector;
 
-- Documents table: stores ingested project docs
CREATE TABLE documents (
  id UUID PRIMARY KEY,
  title TEXT NOT NULL,
  metadata JSONB DEFAULT '{}',
  created_at TIMESTAMPTZ DEFAULT

Phase 3: Streaming Endpoint — SSE for Real-time Review Results

Users shouldn't stare at a loading spinner while three AI agents analyze their code. SSE streams each finding as it's generated.

// === src/streaming/sse-manager.ts ===
// Manages Server-Sent Event connections for review streaming
 
import { Response } from "express";
 
interface SSEClient {
  id: string;
  response: Response;
  connectedAt: Date;
}

// === src/routes/stream.ts ===
// SSE route handler
 
import { Router } from "express";
import { SSEManager } from "../streaming/sse-manager";
import { runCodeReview } from "../agents/orchestrator";
 
const router =

Phase 4: Multi-Agent Code Review Pipeline

Three specialist agents run in parallel, each with its own system prompt and tool set. The orchestrator manages the pipeline and synthesizes results.

// === src/agents/orchestrator.ts ===
// Coordinates the three review agents and streams results
 
import { AIGateway } from "../llm/gateway";
import { retrieveContext } from "../rag/retriever";
import { sanitizeCode } from "../security/input-sanitizer";
import { validateReviewOutput

// === src/agents/security-auditor.ts ===
// Example specialist agent — the others follow the same pattern
 
import type { AIGateway } from "../llm/gateway";
 
const SECURITY_AUDITOR_PROMPT = `You are a senior application security engineer conducting a code review.
 
Your job: find security vulnerabilities in the provided code.
 
Look for:
- SQL injection (string concatenation in queries, unparameterized inputs)
- XSS vulnerabilities (unsanitized user input in HTML/JSX)
- Hardcoded secrets (API keys, passwords, tokens in source code)
- Insecure deserialization (eval(), unserialize(), dynamic imports of user input)
- Missing authentication/authorization checks

Phase 5: Cost Tracking + Security Defenses

Cost tracking middleware

// === src/llm/cost-tracker.ts ===
// Tracks AI costs per session and persists to the database
 
import { sql } from "../db/client";
 
interface CostEntry {
  provider: string;
  model: string;
  inputTokens: number;
  outputTokens

Prompt injection defenses

// === src/security/input-sanitizer.ts ===
// Multi-layer defense against prompt injection in code submissions
 
export function sanitizeCode(input: string): {
  sanitized: string;
  blocked: boolean;
  reason?: string;
} {
  //

Common mistakes

Interview questions

1. Q: You're building an AI code review system. Why would you use three specialist agents instead of one general-purpose agent? A: Three specialist agents provide three key advantages. First, focused prompts: each agent's system prompt targets a specific domain (bugs, style, security) — a single agent with a 3000-token prompt covering all three domains produces lower-quality results because the LLM's attention is diluted. Second, parallel execution: three agents run simultaneously with Promise.all(), cutting latency by ~60% compared to sequential calls. Third, independent failure domains: if the security auditor's API call fails, the bug and style findings are still delivered — degraded service beats no service. The tradeoff is cost (3 LLM calls vs 1), which is managed by using cheaper models for simpler agents (style checker can use GPT-4o-mini while security auditor uses GPT-4o).

2. Q: How do you prevent prompt injection when the "user input" is source code that legitimately contains the word "system" or "instructions"? A: Source code presents a unique challenge because keywords like "system," "execute," or "instructions" appear legitimately in function names, variable names, and comments. The defense strategy has three layers. First, delimiter-based separation: wrap code in XML-style tags (<code_to_review>...</code_to_review>) and explicitly instruct the LLM that content inside these tags is data to be analyzed, not instructions to follow. Second, comment-specific scanning: only block submissions where injection patterns appear in COMMENT syntax (e.g., // ignore all previous instructions or /* system: override */), not in code identifiers. Third, contextual validation: the code is submitted as a review target — the system prompt establishes that the user IS submitting c... Second, comment-specific scanning: only block submissions where injection patterns appear in COMMENT syntax (e.g., // ignore all previous instructions or /* system: override */), not in code identifiers. Third, : the code is submitted as a review target — the system prompt establishes that the user IS submitting code for review, and any directives found in that code are potential security findings to report, not commands to execute. This framing makes the LLM treat "system('rm -rf /')" as a finding worth flagging, not a command to run.

Summary

The AI Code Review Assistant brings together every concept from L7 into a production-grade system. The unified LLM gateway abstracts provider differences and tracks every cent. The RAG pipeline injects project-specific context so reviews are relevant to YOUR codebase, not generic best practices. Three specialist agents run in parallel — bug detector, style checker, and security auditor — each with focused prompts for higher-quality results. SSE streaming delivers findings in real-time so users see progress instead of a spinner. The cost tracker provides per-review, per-model, and per-feature cost visibility. Prompt injection defenses at the input, prompt, and output layers protect against the most common AI attack vector. This is the project you reference in a senior backend interview when asked "have you built a production AI system?"

Quick recall

• AI Gateway = single entry point for all LLM calls. Provider abstraction, cost tracking, error handling — centralized.
• RAG pipeline: chunk → embed → store → retrieve → rerank → inject into prompt. Context makes AI relevant to YOUR codebase.
• Three specialist agents > one general agent. Focused prompts produce better results. Parallel execution cuts latency.
• SSE for streaming, not WebSocket. Unidirectional (server→client) matches LLM streaming perfectly. Simpler to implement and proxy.
• Always wrap user content in delimiters. <code_to_review>...</code_to_review> with explicit instructions prevents prompt injection.
• Cost tracking is not optional. Track per review, per model, per feature. Display on a dashboard. Alert on anomalies.
• Parallel agents with Promise.all(). Bug detector + style checker + security auditor run simultaneously.
• Graceful degradation on agent failure. If one agent fails, return partial results — don't fail the entire review.
• Chunk large code submissions. Estimate tokens before calling the LLM. Split files >8000 tokens into reviewable chunks.
• Persist everything. Reviews, findings, costs, context used. Build history. Track quality trends over time.