RocketRAG: Performance-First Retrieval-Augmented Generation

Overview

RocketRAG is a high-performance Retrieval-Augmented Generation system designed with a focus on speed, simplicity, and extensibility. It packages document ingestion, semantic chunking, vector storage, and LLM inference into a pluggable toolkit that runs as both a CLI utility and a FastAPI server. All components are swappable, so the same pipeline can power notebooks, cron jobs, or production APIs.

Architecture Highlights

• Load — Kreuzberg-based document loaders stream PDFs, Markdown, TXT, and more into a unified format.
• Chunk — Chonkie semantic chunking (model2vec) preserves context while staying small enough for laptop use.
• Vectorize — Sentence Transformers (or custom encoders) generate embeddings with batched throughput.
• Store — Milvus Lite keeps the vector DB local, delivering sub-millisecond retrieval without extra dependencies.
• Generate — llama-cpp-python serves quantized GGUF models for low-latency inference on commodity GPUs or CPUs.

The plugin architecture defines BaseLoader, BaseChunker, BaseVectorizer, BaseLLM, and BaseVectorDB interfaces, so adding a new chunker or embedding model is just a small class.

Usage

pip install rocketrag (or uvx rocketrag ...) provides the CLI immediately. rocketrag prepare --data-dir ./docs builds the vector index, and rocketrag ask "prompt" queries it. rocketrag server --host 0.0.0.0 --port 8000 exposes OpenAI-compatible endpoints, realtime streaming, vector visualizations, and a document browser.

Why It Matters

RocketRAG is optimized for speed end to end and ships batteries-included with a verbose CLI, full FastAPI server, health checks, visualizers, and a chat UI — while remaining fully extensible for custom loaders, chunkers, vectorizers, or LLM runtimes.