I just tagged v0.9 of CorpusKit Reader — but the more interesting thing is what’s underneath it: CorpusKit, a reusable on-device RAG platform that turns any PDF or EPUB into a private, queryable corpus. The Reader is one app built on it; you could build your own. Everything runs locally — no account, no API keys, nothing leaves your device. This post is the full technical tour: the architecture, the retrieval-and-answer pipeline, the toolchain that builds the corpora, and the handful of gotchas that cost me real time.
The idea
Retrieval-augmented generation (RAG) is usually a cloud story: embed your documents, stuff them in a vector database, call a hosted LLM. I wanted the opposite — a RAG stack where every stage runs locally: text extraction, embeddings, vector search, and answer generation. Modern Apple hardware makes this viable: Core ML runs embedding models on essentially any recent device, and Apple Intelligence (Foundation Models) provides an on-device LLM for the generation step.
The result is three pieces:
- CorpusKit — a dependency-light Swift package (the engine): the
Chunk/CorpusMetadatamodel, the embedding service, the vector index, and the bundle loader. - CorpusKit Studio — a macOS authoring app that turns a PDF or EPUB into an indexed, authority-ranked corpus and exports a portable
.corpus.zipbundle. - CorpusKit Reader — the universal iOS and macOS consumer app that loads those bundles and lets you query them.
A .corpus.zip is just a folder of plain files: corpus_meta.json, chunks.json, embeddings.npy, per-chapter importance weights, and a content signature. No database, fully inspectable, trivially shippable.
The retrieval-and-answer pipeline
When you ask a question, two independent things happen:
1. Retrieval (runs everywhere). The question is embedded with Snowflake’s arctic-embed-m-v1.5 (768-dim), converted to Core ML. Pooling and L2 normalization happen client-side in Swift — the Core ML graph only emits the last hidden state, because the in-graph reduce ops mis-convert and produce degenerate embeddings. Cosine similarity against the corpus’s stored vectors yields the top passages.
2. Generation (runs where Apple Intelligence is available). The top passages are handed to a LanguageModelSession with strict instructions: answer only from these excerpts, cite them inline, don’t speculate. The answer streams token-by-token into the UI.
These halves are decoupled, which gives the app its best property.
Graceful degradation
I ran v0.9 on an older iPad that has no Apple Intelligence — and it just worked. That’s by design. When SystemLanguageModel.default.availability isn’t .available, the app falls back to an extractive answer (the top passages’ most relevant sentences) and shows the source citations without the AI summary. Search works everywhere because Core ML runs everywhere; only the written summary requires newer silicon. So nobody gets locked out — they get the passages, which are the real evidence anyway.
Flat-context conversation memory
Follow-up questions (“tell me more”, “why?”) need context, but the on-device model has a modest token window. Instead of accumulating a growing transcript, each turn builds a capped digest of the last few Q&A pairs (questions plus short answer summaries, never the passages) and prepends it to the freshly-retrieved excerpts. Context stays roughly flat no matter how long the conversation runs, so you never drift toward the context ceiling. If you do hit exceededContextWindowSize, the app drops memory and falls back extractively.
The four modes
The reader has a single mode bar over a loaded corpus:
- Chat — semantic Q&A (the pipeline above).
- Read — browse the de-overlapped chapter text; tap a passage in Chat to jump here, scrolled into place.
- Find — literal keyword/phrase search. Since Chat already owns semantic retrieval, Find is purely exact text matching with the match highlighted — a clean, non-overlapping division: Chat = ask in your words, Find = locate exact words.
- Saved — bookmarked passages.
CorpusKit is a platform, not just my app
The Reader is only the most visible piece. Underneath it is CorpusKit — a reusable Swift package with an open, inspectable bundle format — and that’s the part I think matters most.
A .corpus.zip is deliberately boring: plain chunks.json, an embeddings.npy array, metadata, and importance weights. No proprietary database, no lock-in. That means:
- Anyone can author corpora from their own documents. Point Studio at your legal contracts, lab notebooks, API docs, a manuscript you’re editing, or a company handbook, and you get a private, offline, queryable index. The source material never has to touch a server — exactly what you want for sensitive or internal content.
- Anyone can generate corpora however they like. My Python toolchain (Gutenberg → clean EPUB → headless build) is just one producer. Because the format is open, you could build corpora from any pipeline and they’ll load all the same.
- Anyone can build their own app on the engine. CorpusKit Reader is a reference client, not the only possible one. Link the package and you get chunking, embedding, vector search, and grounded generation for free — so a study tool, an internal knowledge base, and a field manual that works with no signal can all share the same on-device RAG stack instead of each reinventing it.
In other words: the interesting deliverable isn’t “an app that reads one book.” It’s a portable, on-device RAG foundation that turns any document collection into something you can talk to — privately, offline, on hardware you already own.
Building corpora — and eating my own dog food
A reader is nothing without content. The toolchain is one command:
./build_corpus.sh --search "Pride and Prejudice"That runs three stages: prep_gutenberg.py downloads a public-domain book (via the Gutendex catalog API) and strips the Project Gutenberg boilerplate — the header, the trailing license, and a body-wide pass that removes any paragraph containing the trademark — then emits a clean EPUB. A headless build harness imports it, chunks it, embeds it, and exports the .corpus.zip into a central store that can later feed a download server.
Then I turned the tool on itself: I wrote the app’s user guides in Markdown and built them into corpora, so you can ask the Reader how to use the Reader. Dogfooding immediately paid off — it surfaced that my build harness was chunking EPUBs flat (fixed-size windows) instead of per-chapter like Studio does, scattering topics across chunks. One config flip later, each help section is its own retrievable passage.
The gotchas (the part you actually want)
A few things that don’t show up in the happy-path tutorials:
The double-extension UTType trap. .corpus.zip is a compound extension, and the system resolves UTType by the substring after the final dot — so it sees plain public.zip-archive. A custom corpus.zip UTI won’t reliably match. The pragmatic fix: the file picker filters on zip, and the import code validates the .corpus.zip suffix itself.
The Swift Testing -only-testing silent skip. My headless corpus builder is a macOS test. Running -only-testing:Target/Suite/method matched nothing and passed vacuously — Swift Testing reports the case as Suite/method(), so the suite-level selector -only-testing:Target/Suite is what actually runs it. A test that “passes” in 0.1s instead of 17s is the tell.
Sandbox-bound headless builds. That test host is sandboxed, so it can only read and write the app’s container Documents. The build script stages the EPUB there and copies the result out afterward — the test can’t touch /tmp or my home folder directly.
SwiftUI state loss across a switch. The conversation lived in @State inside the Chat view. But the mode bar renders modes with a switch, so toggling to Read and back tore down the Chat view and wiped the conversation. Fix: hoist the answer service to the parent container that survives mode switches, and pass it in. A subtle regression that only appears once Chat has siblings.
An accessibility footgun. A passage card used .accessibilityElement(children: .combine) with a custom label — which merged its Read/Save/Share buttons into one element, making them unusable by VoiceOver, and hid the passage text behind the label. The fix: combine only the header (with the raw relevance score hidden), leave the body text readable, and keep the action buttons as independent elements. Plus 44×44 touch targets and @ScaledMetric so the UI grows with Dynamic Type.
Don’t block the main actor on load. Unzipping a bundle and parsing chunks.json + embeddings.npy for a 1,200-passage book freezes the UI if it runs on @MainActor. Moving it to a detached task fixed the hitch.
Designing for a future upgrade without building it yet
v0.9 ships entirely free, but I wanted the structure for a later “unlimited corpora” in-app purchase. So there’s a single Entitlements source of truth: the import limit is nil(unlimited) today, the sidebar shows a live “N imported corpora” counter, and the Import button already carries the gating check as a no-op. When StoreKit gets wired, deriving one boolean from Transaction.currentEntitlements flips the cap, the counter display, and the gating — with zero call-site changes. Bundled free corpora never count against the limit.
Privacy
Because the whole pipeline is local, the privacy story is simple and provable: the app makes no network calls, collects no analytics, requires no account, and works offline. Your questions and your reading never leave the device. Even content delivery, for now, is the user’s browser downloading a file and the app importing it — the app itself stays network-free.
Where it stands
v0.9 is feature-complete: Chat / Read / Find / Saved, a handful of bundled public-domain corpora, import-and-remove with confirmation, settings, accessibility, and the IAP-ready scaffolding. Next up is the distribution layer — a static catalog and download page — and eventually the StoreKit upgrade behind that counter.
The throughline: on-device isn’t a limitation, it’s a feature — and CorpusKit makes it something other people can build on, not just me. Private, offline, and it even runs on the old iPad in the drawer.