A retrieval-augmented generation system that answers questions over PDFs, images, charts and text in one place. It runs fully offline on Intel Lunar Lake hardware, so documents never leave the machine, and it is designed to move from a local vector store to a distributed one without code changes.
How it works
Local inference through Ollama with Llama 3.2 Vision, and Nomic embeddings for text and images.
LlamaIndex handles document processing and retrieval.
Milvus Lite locally, with the same code able to target a Milvus cluster.
Uses the Intel Arc GPU and NPU, with 4-bit quantised models to fit in 32 GB of RAM.
FastAPI backend and a Next.js 14 frontend in TypeScript; answers show which documents they came from.
