InsightVault
Ask questions of your own PDFs and get answers grounded in the passages that support them.
AI product (RAG), Nov 2025 – Sep 2026
A retrieval-augmented question-answering app. PDFs are chunked, embedded into Postgres with pgvector, and searched by cosine similarity before an LLM answers from the retrieved context only.
Problem
Keyword search over a stack of PDFs misses answers phrased differently from the query, and a general chatbot answers confidently without reading your documents. The goal was answers that come from the uploaded document and show which passages they came from.
Why it was worth building
Retrieval-augmented generation is the core pattern behind most applied LLM products. Building it end to end, from ingestion and chunking to vector indexing, retrieval tuning and grounded prompting, means owning every decision instead of hiding them behind a framework.
Architecture
- PDF uploadPOST /api/upload
- Validate25 MB cap, %PDF- magic bytes, rate limit
- Extract textpdf-parse
- Chunksentence-aware, 800 chars, 120 overlap
- EmbedHF → Gemini → OpenAI fallback
- Storedocument_chunks.embedding vector(1536)
- QuestionPOST /api/query, ≤2,000 chars
- Embed querysame provider chain
- Vector searchmatch_chunks(): cosine, top 6, threshold 0.2
- Build contexttop chunks, capped at 8,000 chars
- LLMGroq or GPT-4o mini, 800 max tokens
- Answer + sourcessimilarity scores returned to UI
Technical decisions
pgvector inside Postgres instead of a separate vector database
Documents, chunks, queries and responses live in one Supabase Postgres database. Similarity search is a SQL function (match_chunks) over an HNSW index with cosine distance, so metadata filters like a single document_id sit in the same query as the vector search.
A provider chain for embeddings
Embeddings try Hugging Face (bge-small), then Gemini (gemini-embedding-001 at 1536 dimensions), then OpenAI (text-embedding-3-small). Free-tier quota failures degrade to the next provider instead of failing the upload.
Retrieval that degrades instead of returning nothing
If no chunk clears the 0.2 similarity threshold, the query retries at 0.0, and failing that falls back to the document’s first chunks. The prompt tells the model to say the document does not contain the answer rather than guess.
Sentence-aware chunking with overlap
Text is split on sentence boundaries and packed to about 800 characters, with a 120-character tail carried into the next chunk, so an answer that spans a boundary is still retrievable.
Challenges
Keeping one vector column across providers
Providers return different dimensions. Gemini can be asked for 1536 directly; the 384-dimension Hugging Face vectors are zero-padded to fit the vector(1536) column. The trade-off: vectors from different providers are not comparable, so a document and its queries need to be embedded by the same provider.
Protecting a paid API from abuse
Upload and query routes are rate-limited per IP (10 uploads per 10 minutes, 40 queries per minute), inputs are length-capped, document IDs are validated as UUIDs, and uploads are checked by magic bytes rather than trusting the file extension.
What I built
- Next.js 16 App Router with route handlers for upload, query, documents, chunks and history.
- Supabase schema with pgvector, an HNSW index (vector_cosine_ops) and a match_chunks() RPC scoped by document_id.
- Per-query latency recorded in a responses table, so response times can be measured over time.
- LLM client switches to Groq when a Groq key is configured and falls back to OpenAI GPT-4o mini, with typed rate-limit detection for clearer errors.
- Answers are returned with the matched chunks and their similarity scores so the UI can show sources.
Results
- Working end-to-end RAG over uploaded PDFs with cited source passages.
- Uploads keep working when one embedding provider’s free quota runs out.
Takeaways
- Most RAG quality problems are retrieval problems: threshold, chunk size and fallback behaviour mattered more than the choice of chat model.
- Mixing embedding providers in one index is convenient but quietly breaks similarity. Recording the provider per chunk is the fix.
What I'd do next
- Store the embedding provider per chunk and re-embed queries with the matching one.
- Turn on the per-user Row Level Security policies already drafted in supabase-schema.sql for multi-tenant vaults.
- Move the in-memory rate limiter to a shared store so limits hold across serverless instances.
- Add a small evaluation set of question/answer pairs to measure retrieval changes instead of eyeballing them.