LOMA: Mobile Offline Medical AI Assistant

Motivation

Billions of people need reliable medical answers where connectivity is limited or privacy is paramount. LOMA (Local Offline Medical Assistant) delivers a zero-cloud experience: the entire assistant, from embeddings to language model responses, runs on the user's phone.

System Design

• Model — Gemma 3n converted to a 4.79 GB GGUF checkpoint, served via llama.rn with GPU offload for up to 99 layers.
• Retrieval — A doc2query-enhanced RAG stack indexes 5 million Q&A-style medical documents so answers are grounded and cite exact sources.
• Embeddings — ExecuTorch runs all-MiniLM-L6-v2 locally, generating 384-d vectors in ~70 ms using only 150–190 MB RAM.
• Database — Turso (SQLite + vector extensions) ships as a pre-built bundle synced through Cloudflare R2; cosine search yields results without ballooning storage.
• Frontend — A React Native application with shared abstractions for storage, queue-based inference, and lazy loading keeps both iOS and Android responsive.

Workflow

A user question is normalized into Gemma's conversation format. The query embedding searches both long-form documents and FAQ-style pairs, retrieved passages are assembled with structured citations, and Gemma 3n generates an answer entirely on-device — never sharing data with servers.

Impact and Metrics

Privacy-preserving responses with verifiable citations improve trust for clinical decision support. LOMA works offline after the initial 4.79 GB download, with the model and database fitting comfortably on mid-range phones. Vector search takes 94 ms for 50k vectors, and response latency stays under one minute even on modest hardware.