Most RAG tutorials show the happy path: load document, chunk, embed, store, query, done. Real pipelines are messier. This is the version I wish I'd found first — including the parts that broke, the design decisions that mattered, and how I'd approach it again.
The stack: n8n for orchestration, Hugging Face sentence-transformers for embeddings, Supabase with pgvector for storage and retrieval, and an LLM call to synthesise the answer.
What RAG actually solves (and what it doesn't)
RAG grounds an LLM's answer in your documents instead of relying on what the model happened to memorise. Compared to fine-tuning, it's cheaper to update (add a document, re-embed, done — no retraining) and it gives you a citation trail back to the source. That's the win: current, grounded, attributable answers.
What it doesn't fix: it can't reconcile contradictory sources for you, it struggles when the answer is spread across a very long document that doesn't chunk cleanly, and it won't read images or complex tables without extra handling. Being honest about that scope up front saves you from blaming the LLM for a retrieval problem.
The ingestion pipeline: from raw document to stored vector
Ingestion is where quality is won or lost. The flow: extract text → chunk → embed → store.
- Chunking — fixed-size character chunking was my first mistake; it splits mid-sentence and destroys the meaning a chunk is supposed to carry. Moving to chunks that respect sentence/paragraph boundaries, with a modest overlap so context isn't severed at the seam, improved retrieval noticeably.
- Embeddings — a compact sentence-transformer model (the MiniLM family) is a sensible default: small, fast, and good enough for most semantic search, with far lower cost than calling a hosted embedding API per chunk.
- Storage — Supabase with the pgvector extension holds each chunk alongside its embedding and metadata (source, position), so retrieval can cite where an answer came from.
- Orchestration — n8n ties the steps together: trigger on a new document, run extraction and chunking, call the embedding step, upsert into Supabase.
The query pipeline: from user question to grounded answer
At query time: embed the user's question with the same model used at ingestion, run a similarity search in Supabase (pgvector's <=> distance operator), take the top-k most similar chunks, assemble them into a context block, and pass that plus the question to the LLM with an instruction to answer only from the provided context and cite sources.
The "same model both sides" rule is non-negotiable — query and document embeddings have to live in the same vector space or similarity is meaningless.
What broke and how I fixed it
- Chunking artifacts — mid-sentence splits produced chunks that retrieved well by keyword but made no sense to the LLM. Fixed by boundary-aware chunking with overlap.
- Embedding drift — changing the embedding model invalidates every stored vector. If you switch models, you must re-embed the whole corpus; mixing vectors from two models silently degrades results.
- pgvector performance at scale — naive similarity scans slow down as the table grows; an appropriate vector index is what keeps query latency flat.
- n8n timeouts on large documents — big files blew past webhook/execution limits. The fix was to break ingestion into smaller batched steps rather than one giant workflow execution.
Evaluating retrieval quality without a labeled dataset
You rarely have a golden test set early on. Cheap evaluation that still works: keep a list of real questions with known correct answers and spot-check that the right chunks are being retrieved (retrieval quality), separately from whether the LLM phrased the answer well (generation quality) — they fail differently. The "obvious miss" test — does it retrieve the clearly relevant passage for a question you know the document answers? — catches most chunking and threshold problems. Tune top-k and the similarity threshold against these by hand until retrieval is reliably surfacing the right context.
Cost and latency breakdown
The dominant cost is usually the LLM synthesis call, not retrieval — local sentence-transformer embeddings are nearly free, and a pgvector query is fast. Latency splits across embedding the query, the vector search, and the generation, with generation almost always the largest slice. Caching answers to repeated or near-identical queries is the simplest meaningful win.
When to use this stack vs. something else
- n8n is great when you want visual orchestration and to wire RAG into other automations; it's overkill if you just need a single embedded function in an app.
- LangChain / LlamaIndex give you more RAG-specific abstractions in code if you'd rather not orchestrate visually.
- Managed vector DBs (Pinecone, Weaviate) earn their cost at large scale or when you need features pgvector doesn't have — but for most projects, Supabase + pgvector keeps everything in one database you already run.
FAQ
Do query and document embeddings have to use the same model? Yes. They must share a vector space, so use the identical model for both — and re-embed everything if you ever switch.
Why are my RAG answers irrelevant even though the document contains the answer? Usually chunking. Mid-sentence or oversized chunks retrieve poorly; use boundary-aware chunks with overlap and check what's actually being retrieved.
Is pgvector enough, or do I need Pinecone? For most small-to-mid projects, pgvector in Supabase is plenty. Reach for a managed vector DB at scale or for specific features you can't get otherwise.