# Module 04 — Grounding & RAG: Lab Guide These labs are the runnable complement to the two prose lessons at https://mindpool.io/app/learn/04-ground. Each exercise drives the real `mpl` CLI and the underlying retrieval and generation pipeline it reveals, tying terminal output directly back to the mechanisms taught: character-level chunking and 384-dimensional unit-norm embeddings (`mpl embed`, `mpl index`), brute-force flat KNN search and the model-match guard (`mpl search`), and the full RAG assembly pipeline that converts ranked chunks into a cited answer from qwen3:8b (`mpl rag`). --- ## Prerequisites Work through these steps in order. Every gated `mpl` verb (`embed`, `index`, `search`, `rag`) will refuse with an enrollment error if the entitlement is missing. ### 1. Enroll on-chain Purchase enrollment through the mindpool.io app. Enrollment records the paid flag, paid amount, and referral count in the DefiMentor contract on Polygon. `mpl` checks all three on-chain; there is no account on our servers. ### 2. Activate the entitlement ```bash mpl login ``` `mpl login` prompts for your wallet address, prints a SIWE message, asks you to sign it with the Trust Wallet extension or Trust Wallet mobile, and verifies that your paid flag is set and that either your paidAmount is at least 40 DAI or your referredCount is at least 194 on-chain. A time-limited entitlement is cached in `~/.mpl/`. Re-run `mpl login` to refresh. Verify success: ```bash mpl status ``` You should see your address and `enrolled: true`. ### 3. Confirm the local AI stack and retrieval venv are installed `mpl setup` is a **module 01 (`01-foundations`) verb** — it should have been run when you completed that module. If you skipped it, run it now: ```bash mpl setup ``` `mpl setup` installs the pinned llama.cpp build (sha256-verified archives) and the Python tooling via `uv`, provisioning the **retrieval Python virtual environment** (sentence-transformers + LanceDB) that `mpl embed`, `mpl index`, `mpl search`, and `mpl rag` depend on. By default it also pulls the curriculum's pinned GGUFs (including `qwen3:8b`, sha256-verified) into `~/.mpl/models/`; pass `--no-models` to skip that. The retrieval venv itself (installing the Python packages) can take a few minutes on first run; subsequent runs skip already-present components. If `mpl embed` or `mpl index` fails with `error: ... — run \`mpl setup\` to provision the retrieval venv`, the Python environment is missing — re-run `mpl setup`. Verify the stack is healthy: ```bash mpl doctor ``` ### 4. Confirm `qwen3:8b` is available `qwen3:8b` is the chat model used in `lab-02-rag`. It is a *reasoning model* — it emits a `…` internal chain-of-thought block before its answer, which is pedagogically valuable here: you can watch qwen3:8b reason over the retrieved context before citing its sources. **`mpl embed`, `mpl index`, and `mpl search` do not require `qwen3:8b`** — they use the MiniLM embedding model, which is fetched automatically (pinned revision `1110a243fdf4706b3f48f1d95db1a4f5529b4d41`) on the first `embed` or `index` call. You do not need to pull it manually. Confirm the pinned `qwen3:8b` GGUF is on disk (its sha256-verified weights are pulled by `mpl setup`): ```bash ls ~/.mpl/models/qwen3:8b/Q4_K_M.gguf ``` You should see the file. If it is missing (for example you ran `mpl setup --no-models`), pull it explicitly — it is downloaded and sha256-verified against the manifest pin: ```bash mpl setup --model qwen3:8b ``` There is no lazy auto-pull at run time: if the GGUF is absent when `mpl rag` needs it, the command errors with a clear "model not downloaded — run `mpl setup`" message rather than downloading behind your back. Pulling it now makes sure it is complete before starting lab-02 exercises. ### 5. Materialize the lab assets Both labs write a project-local `./.mpl/` store relative to your current working directory, and use the provided corpus. Materialize this module's assets into a dedicated workspace: ```bash mkdir -p ~/mpl-labs cd ~/mpl-labs mpl lab 04-ground cd 04-ground ``` `mpl lab 04-ground` writes the five-file corpus under `corpus/` inside the new `04-ground/` directory. Run every command below from inside that directory. --- ## How to work the labs - **`lab-01-embeddings.md`** — the retrieval side. Drives `mpl embed`, `mpl index`, `mpl search`, and the model-match guard. Covers embedding a string to a 384-dimensional unit-norm vector, indexing the provided corpus (4 single-chunk files + 1 multi-chunk file), searching with ranked JSON results, and deliberately tripping the guard to understand why incompatible embedding bases make results meaningless. - **`lab-02-rag.md`** — the generation side. Drives `mpl rag`. Covers inspecting both the search plan and the assembled `[n] (source) text` prompt with `--dry-run`, running a grounded live query, using `--json` for stdout-pure output, comparing `-k 2` vs `-k 8` to see the budget trim in action, and running an out-of-corpus question to observe the honest "I don't know" response. Each exercise has the form: > **COMMAND** — the exact terminal command to run > **OBSERVE** — what to look for in the output > **EXPLAIN** — what it means, connecting to lesson mechanics Reference answers and expected output shapes are in `solutions/lab-01.md` and `solutions/lab-02.md`. Use them for self-checking, not as a shortcut — the learning is in the observation. ### The provided corpus Both labs use the small corpus materialized into `corpus/` by `mpl lab 04-ground`: | File | Description | Expected chunks | |---|---|---| | `01-pacific-ocean.md` | Pacific Ocean geography facts | 1 | | `02-photosynthesis.md` | Photosynthesis biology facts | 1 | | `03-roman-republic.md` | Roman Republic history facts | 1 | | `04-speed-of-light.md` | Speed of light physics facts | 1 | | `05-nitrogen-cycle.md` | Nitrogen cycle (multi-paragraph, ~1300 chars) | **3** | `05-nitrogen-cycle.md` is the multi-chunk file: at ~1320 characters it produces 3 overlapping chunks (chunk_ids 0, 1, 2) with the default 512/64 sliding window. The 7 corpus chunks total roughly ~3,000 characters — well under the 12,000-char RAG budget. Lab 01 uses this to observe the chunking and overlap mechanics. **Topics NOT in this corpus** (useful for the out-of-corpus test in lab-02): machine learning algorithms, programming languages, ancient Greek history, astronomy, economics. A question about any of these should return an honest "I don't know" from `mpl rag`. --- ## Module reference Module id: `04-ground`; lab command: `mpl lab 04-ground`; status: runnable. --- ## A note on enrollment gating The `mpl` verbs used in these labs — `embed`, `index`, `search`, and `rag` — are all gated. Attempting to run them without a valid entitlement produces an enrollment error and a pointer to `mpl login`. This is intentional: the labs are part of the enrolled curriculum, not a free sampler. `mpl doctor`, `mpl login`, and `mpl status` are always open. (`mpl setup` is gated too but belongs to module 01 — it is a prerequisite for this module, not a verb you call during module 04 exercises.) --- # Lab 01 — Embeddings, Indexing, and Search > **Lesson connection:** lesson-01 taught that `mpl embed` converts any string to a 384-dimensional unit-norm vector whose distance to other vectors encodes semantic relatedness, that `mpl index` chunks documents with a 512/64 character sliding window and stores each chunk's vector in a project-local LanceDB store, and that `mpl search` does an exact flat KNN scan to return ranked results. It also introduced the model-match guard that prevents comparing vectors from incompatible embedding bases. This lab grounds every step in real terminal output. **Prerequisites:** `mpl login` completed, `mpl setup` completed (retrieval venv provisioned), `mpl lab 04-ground` materialized (see the module README) with your working directory set to the resulting `04-ground/` directory. The `corpus/` directory is already there — no copy step needed. --- ## Exercise 1 — Embed a string and inspect the vector **COMMAND** ```bash mpl embed "the cat sat on the mat" --json ``` **OBSERVE** The output is a JSON object with a single key `"vector"` whose value is an array of exactly **384 floating-point numbers**: ```json {"vector": [-0.0234, 0.0817, -0.0451, ..., 0.0312]} ``` Count the values: there are 384 of them. Every number is a coordinate in the 384-dimensional embedding space. The values are typically in the range −1 to 1; many are small (close to zero) with a few larger positive or negative values. To confirm the vector's L2 norm is approximately 1.0, compute it: ```bash mpl embed "the cat sat on the mat" --json | python3 -c " import json, sys, math vec = json.load(sys.stdin)['vector'] norm = math.sqrt(sum(x*x for x in vec)) print(f'dimension: {len(vec)}') print(f'L2 norm: {norm:.6f}') " ``` You should see `dimension: 384` and `L2 norm:` very close to `1.000000` — confirmed within floating-point rounding. **EXPLAIN** The model's pipeline is `Transformer (6 layers) → Pooling (mean across token positions) → Normalize (L2)`. The final `Normalize` module ensures every output vector has unit length regardless of input. The `.encode()` call in `mpl`'s Python scripts does not pass a `normalize_embeddings` flag — the normalization is applied by the model's own architecture, not by a caller option. The practical consequence: for unit-norm vectors, L2 distance and cosine similarity rankings are equivalent. `mpl search` uses LanceDB's default L2 metric, but because all vectors are unit-norm, the ranking it produces is identical to cosine ranking. The 384 numbers are not word frequencies, vocabulary indices, or human-interpretable axes. They are coordinates in a learned geometric space — the byproduct of gradient descent on millions of contrastive sentence pairs. Their meaning is relational: "this point is close to that point" corresponds to "these texts are semantically similar." --- ## Exercise 2 — Index the corpus and inspect the store **COMMAND A — index all five corpus files** ```bash mpl index corpus/ ``` **OBSERVE** `mpl index` walks the directory recursively and processes every file. Look for a summary line such as: ``` indexed 7 chunks from 5 file(s) ``` The count is 7 because four files produce 1 chunk each and `corpus/05-nitrogen-cycle.md` (at ~1320 characters) produces 3 chunks under the 512/64 sliding window. **COMMAND B — inspect the metadata file** ```bash cat ./.mpl/index.json ``` **OBSERVE** The file contains the exact schema that `index.py` writes via `json.dump(..., indent=2)`: ```json { "model": "sentence-transformers/all-MiniLM-L6-v2", "revision": "1110a243fdf4706b3f48f1d95db1a4f5529b4d41", "dim": 384, "table": "documents", "chunk_size": 512, "chunk_overlap": 64 } ``` Confirm each field: `model` is the full HuggingFace model id, `revision` is the pinned commit hash, `dim` is 384 (the length of the first vector indexed), `table` is `"documents"` (the LanceDB table name), and `chunk_size`/`chunk_overlap` record the parameters used for this index. If you re-index with different parameters, these fields update. **EXPLAIN** Two things to notice: 1. **The chunking arithmetic for `05-nitrogen-cycle.md`.** The file is 1320 characters. With `chunk_size=512` and `chunk_overlap=64`, the stride is `512 − 64 = 448` characters per step: ``` chunk 0: chars 0 – 511 (512 chars) chunk 1: chars 448 – 959 (512 chars) chunk 2: chars 896 – 1319 (424 chars) ``` The overlap means the last 64 characters of chunk 0 (characters 448–511) are also the first 64 characters of chunk 1. This mitigates — but does not eliminate — context loss at chunk boundaries. Chunking is purely character-based: it can and does split mid-word or mid-sentence; a sentence longer than 64 characters spanning a boundary may be incomplete in both chunks. 2. **Project-local vs global.** The store lives at `./.mpl/` relative to your working directory, not at `~/.mpl/`. Your entitlement cache, login state, and global configuration live in `~/.mpl/`. The vector index for any given project lives in that project's `./.mpl/`. This means you can have separate independent indexes for different directories. --- ## Exercise 3 — Search the index and read a ranked result **COMMAND** ```bash mpl search "how does nitrogen return to the atmosphere" ``` **OBSERVE** `mpl search` returns the top 5 results (default `-k 5`) as a ranked list. Each result has the shape: ``` rank: 1 score: 0.XXXX source: corpus/05-nitrogen-cycle.md chunk_id: 2 text: orb nitrate from the soil. The cycle closes through denitrification: anaerobic bacteria in wate... ``` The top-ranked results should be chunks from `corpus/05-nitrogen-cycle.md` because that file is most semantically related to the query. The `score` field is `round(1.0 − _distance, 4)` where `_distance` is LanceDB's `l2` metric (squared Euclidean distance `||u − v||²`) — higher score means closer to the query. Note: `text` is truncated to 200 characters in this default display. The full chunk text is available internally (and used by `mpl rag`). To see the results as structured JSON: ```bash mpl search "how does nitrogen return to the atmosphere" --json ``` **OBSERVE** Each result object has exactly these keys: `rank`, `score`, `source`, `chunk_id`, `text`. The `text` is still truncated to 200 characters in the default output. **EXPLAIN** `mpl search` does the following: 1. Embeds the query string with the same `all-MiniLM-L6-v2` model used at index time. 2. Calls `tbl.search(query_vec).limit(5).to_list()` on the LanceDB `"documents"` table. **No ANN index is built.** LanceDB's behaviour when `.create_index()` has not been called is a brute-force flat scan: it computes L2 distance between the query vector and every one of the 7 stored vectors, then returns the 5 closest. This is O(N × d) — linear in chunk count, linear in dimension. At 7 chunks on local hardware it is instantaneous. It also has 100% recall: it cannot miss the true nearest neighbour because it examines all of them. The `score = 1.0 − _distance` formula is a monotonic transformation, where `_distance` is LanceDB's `l2` metric returning the **squared** Euclidean distance `||u − v||²`. On unit-norm MiniLM vectors: `_distance = 1 − score` and `cos θ = (score + 1) / 2`. Higher score = closer match; squared-L2 ranking and cosine ranking are equivalent — the ordering would be the same either way. The absolute score value is not a calibrated similarity percentage; a score of 0.72 does not mean "72% related." Verify the actual score range on your box. --- ## Exercise 4 — Trip the model-match guard This exercise deliberately corrupts the index metadata to observe the guard error and understand why it is mathematically necessary. **COMMAND A — record the current model id** ```bash cat ./.mpl/index.json ``` Note the value of the `"model"` field: `"sentence-transformers/all-MiniLM-L6-v2"`. **COMMAND B — overwrite the model field with a fake id** ```bash python3 -c " import json with open('.mpl/index.json') as f: meta = json.load(f) meta['model'] = 'sentence-transformers/all-mpnet-base-v2' with open('.mpl/index.json', 'w') as f: json.dump(meta, f, indent=2) print('model field set to:', meta['model']) " ``` **COMMAND C — run a search and observe the guard error** ```bash mpl search "how does nitrogen return to the atmosphere" ``` **OBSERVE** The command exits with an error. The exact error message is: ``` index was built with model 'sentence-transformers/all-mpnet-base-v2' but active model is 'sentence-transformers/all-MiniLM-L6-v2'; re-run `mpl index ` to rebuild the store ``` Note: the message says `index was built with` the model id stored in `index.json` (the fake id you just wrote), `but active model is` the model that `mpl` is actually using. The guard is protecting you from querying with the wrong model. **COMMAND D — restore the index** ```bash mpl index corpus/ ``` This rebuilds `.mpl/` from scratch with the correct model id. Verify: ```bash cat ./.mpl/index.json ``` The `"model"` field should be `"sentence-transformers/all-MiniLM-L6-v2"` again. **EXPLAIN** Two different embedding models place text in different geometric spaces. Even if both produce 384-dimensional vectors, the axes are the result of different training runs — different random seeds, different training data, different contrastive pairs. A query vector from model B has no defined geometric relationship to document vectors from model A. The L2 distances computed between them are arbitrary numbers; the ranking they produce is meaningless. The guard keys on the **model id string**, not on the `dim` field. This is correct: two models could both produce 384-dimensional vectors whose geometric spaces are completely incompatible. Matching dimensions is a necessary condition for computation, but it does not guarantee meaningful comparison. The model id is the right guard. If you genuinely want to change embedding models, the only safe path is to discard the existing store and re-index: `rm -rf ./.mpl/ && mpl index `. --- # Lab 02 — Retrieval-Augmented Generation > **Lesson connection:** lesson-02 taught that `mpl rag` is a four-stage pipeline — retrieve with `--full` text, trim to the 12 000-character budget (dropping lowest-ranked first), assemble the verbatim `[n] (source) text` prompt, generate with qwen3:8b — and that the `--dry-run` flag exposes both plans before any inference runs. This lab makes every stage live in your terminal. **Prerequisites:** `mpl login` completed, `mpl setup` completed (retrieval venv provisioned), the pinned `qwen3:8b` GGUF on disk (`ls ~/.mpl/models/qwen3:8b/Q4_K_M.gguf`), corpus indexed from lab-01 (`./.mpl/` exists). If you skipped lab-01, run `mpl index corpus/` in your working directory before starting. --- ## Exercise 1 — Inspect the full pipeline with `--dry-run` **COMMAND** ```bash mpl rag "how does nitrogen return to the atmosphere" --dry-run ``` **OBSERVE** `--dry-run` prints **both plans** before any execution: **Plan 1 — the search plan**, showing what `mpl search` will execute: ``` → python3 search.py --full -k 5 "how does nitrogen return to the atmosphere" ``` **Plan 2 — the generate plan**, showing the backend, model, and the full assembled prompt context. The assembled prompt follows this exact structure: ``` You are answering the question using only the provided context. Cite sources by their bracket number, like [1]. If the context does not contain the answer, say you don't know. Context: [1] (corpus/05-nitrogen-cycle.md) orb nitrate from the soil. The cycle closes through denitrification: anaerobic bacteria in waterlogged soils convert nitrates back into gaseous nitrogen or nitrous oxide, returning nitrogen to the atmosphere. ... [2] (corpus/05-nitrogen-cycle.md) nia (NH3), which plants can absorb through their roots. Once incorporated into organic matter, nitrogen moves through the food web as organisms consume one another. ... [3] (corpus/05-nitrogen-cycle.md) The nitrogen cycle is the biogeochemical process by which nitrogen is converted between its various chemical forms ... [4] (corpus/04-speed-of-light.md) Light travels through a vacuum at exactly 299,792,458 metres per second. ... [5] (corpus/01-pacific-ocean.md) The Pacific Ocean is the largest and deepest ocean on Earth. ... Question: how does nitrogen return to the atmosphere Answer: ``` (The exact chunk text order depends on the retrieval ranking, and the text is shown truncated here for readability. On your box the full chunk text appears in each `[n]` entry.) **EXPLAIN** The `--dry-run` output shows the full pipeline plan without executing inference: 1. **Search plan**: `mpl rag` calls `search.py` internally with `--full`, meaning chunk text is returned untruncated (unlike the default `mpl search` display which truncates to 200 characters). The full text is needed to assemble the context window. 2. **Generate plan / assembled prompt**: the system instruction is verbatim from `assemble_prompt` in `rag.rs`. Every context line follows the format `[n] (source) ` — the bracket number, the file path, and the complete chunk. The model is instructed to cite by bracket number and to say "I don't know" if the context is insufficient. 3. **The "Answer:" slot is empty.** This is deliberate: the prompt ends with `Answer:` and nothing more. qwen3:8b is completing a continuation. Its `` block will reason over the numbered sources before writing the answer. Use `--dry-run` whenever you want to inspect what retrieval found and what prompt the model will receive before committing to inference. --- ## Exercise 2 — Run a grounded query and read the cited answer **COMMAND** ```bash mpl rag "how does nitrogen return to the atmosphere" ``` **OBSERVE** `mpl rag` outputs in sequence: 1. A **retrieved sources** summary — the ranked list of sources used, before inference begins. 2. qwen3:8b's **`` block** — the model reasons over the retrieved context, explicitly connecting its reasoning to the numbered sources. 3. The **cited answer** — the model's answer with inline `[1]`, `[2]` citations (e.g. "Denitrifying bacteria convert nitrates back to gaseous nitrogen [1], closing the nitrogen cycle."). The answer should correctly describe denitrification and cite the relevant chunk(s) from `corpus/05-nitrogen-cycle.md`. **EXPLAIN** qwen3:8b emits a `` reasoning block (see module 03); the rag answer reasons before it cites. Here, the reasoning is over the retrieved context: you can watch the model decide which `[n]` bracket to cite for each claim. The citations the model emits (`[1]`, `[2]`) correspond directly to the numbered lines in the assembled prompt. `[1]` means "I drew this from the chunk labelled `[1]` in the context." This makes the answer **auditable**: you can read the cited chunk (shown in the `--dry-run` output) and evaluate whether the model interpreted it correctly. Citations are not verification. The model could cite the wrong bracket, paraphrase incorrectly, or interpolate from training knowledge. Auditing requires reading the cited chunk. --- ## Exercise 3 — `--json` for stdout-pure output **COMMAND** ```bash mpl rag "what is the deepest point in the Pacific Ocean" --json ``` **OBSERVE** The only output on stdout is a single JSON object: ```json { "answer": "The deepest known point in the Pacific Ocean is located in the Mariana Trench, at roughly 11 kilometres below sea level [1].", "backend": "llama.cpp", "model": "qwen3:8b", "query": "what is the deepest point in the Pacific Ocean", "retrieved": [ { "rank": 1, "score": 0.XXXX, "source": "corpus/01-pacific-ocean.md", "chunk_id": 0, "text": "The Pacific Ocean is the largest and deepest ocean on Earth. ..." }, ... ] } ``` The teaching lines that `mpl rag` normally emits (retrieved sources summary, transparency output) are redirected to stderr. Only the JSON object appears on stdout. Verify with a pipe: ```bash mpl rag "what is the deepest point in the Pacific Ocean" --json | python3 -m json.tool ``` The pipe should receive clean JSON with no non-JSON lines. **EXPLAIN** The `--json` flag enforces a **stdout purity contract**: only the structured JSON object goes to stdout; all other output (teaching lines, transparency lines, progress) goes to stderr. This makes `--json` suitable for scripting and piping — you can `| jq .answer` or `| python3 -c "..."` without needing to filter out teaching output. The `retrieved` array in the JSON response contains the full retrieved chunk objects (with untruncated text), the same data that fed the assembled prompt. This is useful for programmatic auditing: your code can inspect which sources the answer was drawn from and cross-check the claims. --- ## Exercise 4 — Compare `-k 2` vs `-k 8` (budget trim in action) **COMMAND A — narrow retrieval** ```bash mpl rag "what is the nitrogen cycle" -k 2 --dry-run ``` **OBSERVE** The assembled prompt context contains only 2 `[n]` entries. If the two highest-ranked chunks are both from `corpus/05-nitrogen-cycle.md`, the context covers only the parts of the file those chunks contain — the answer may be partial or incomplete. **COMMAND B — wider retrieval** ```bash mpl rag "what is the nitrogen cycle" -k 8 --dry-run ``` **OBSERVE** The assembled prompt context attempts to include up to 8 chunks. Look at the `--dry-run` output: how many `[n]` entries actually appear in the context? At 7 total indexed chunks, `-k 8` retrieves all 7. However, `trim_to_budget` drops the lowest-ranked chunks if their cumulative character count would exceed 12 000 characters. With only 7 short-to-medium chunks totalling roughly ~3,000 characters (well under the 12,000-char budget), all 7 likely survive the budget — but if you index a larger corpus and repeat this exercise, you will see chunks dropped. **COMMAND C — run live with `-k 8`** ```bash mpl rag "what is the nitrogen cycle" -k 8 ``` **OBSERVE** Compare the answer to the `-k 2` case. With more retrieved chunks, the answer may cover the full nitrogen cycle (fixation, ammonification, nitrification, denitrification) rather than just the aspects captured in the top-2 chunks. **EXPLAIN** `RAG_MAX_CONTEXT_CHARS = 12_000` (characters, not tokens). `trim_to_budget` walks the ranked list from rank 1 downward, accumulating character counts. It keeps each chunk while the running total stays within 12 000 characters. When adding the next chunk would exceed the budget, that chunk and all lower-ranked ones are dropped. The drop order is lowest-ranked first: the most semantically relevant chunks survive; the marginal ones are sacrificed. The practical tradeoff: - `-k 2`: fast retrieval, minimal context, risk of missing relevant evidence - `-k 5` (default): balanced — sufficient for most focused queries - `-k 8`: more candidates, but the budget may trim some; useful when the answer is distributed across many chunks or when the query is multi-faceted The budget exists because attention is O(n²) in context length, and because "Lost in the Middle" (Liu et al., 2023) shows that evidence buried in long contexts is systematically underused. More chunks is not always better. --- ## Exercise 5 — Out-of-corpus question: observe the honest "I don't know" **COMMAND** ```bash mpl rag "what is the time complexity of merge sort" ``` **OBSERVE** The retrieved chunks will be whatever is closest in the embedding space to "time complexity of merge sort" — likely chunks about the nitrogen cycle, photosynthesis, or physics, none of which are meaningfully related. The top scores will be noticeably lower than in Exercise 2. The model's answer should state that it does not know, or that the provided context does not contain information about merge sort or time complexity. A typical response: ``` I don't know. The provided context does not contain information about merge sort or its time complexity. ``` (Or similar phrasing — the exact wording varies.) **EXPLAIN** Two mechanisms work together here: 1. **Vector search does not verify relevance.** `mpl search` returns the top-k closest chunks regardless of how close they actually are. When the query is about a topic with no related document in the store, the top-k results are the *least irrelevant* chunks, not relevant ones. The scores will be lower, but low scores do not prevent retrieval — `mpl rag` always passes at least one chunk to the prompt. 2. **The system instruction licenses "I don't know."** The assembled prompt contains: ``` You are answering the question using only the provided context. Cite sources by their bracket number, like [1]. If the context does not contain the answer, say you don't know. ``` The clause "If the context does not contain the answer, say you don't know" is load-bearing. Without it, a model would often confabulate from training knowledge and attach citations anyway. With it, a well-behaved model recognises that the retrieved text about denitrification and speed of light has no bearing on merge sort and declines to answer. This is the honest baseline behaviour of a correctly implemented RAG system: it does not invent answers from out-of-context knowledge; it tells you when its corpus cannot support an answer. --- ## Teardown The `./.mpl/` directory holds your project-local vector index. To clean up: ```bash rm -rf ./.mpl/ ``` To re-index from scratch at any time: ```bash mpl index corpus/ ``` --- ## Solutions # Solutions — Lab 01: Embeddings, Indexing, and Search *Expected output shapes. Run these on your own enrolled box; exact score values will vary by machine and corpus.* > **For self-checking.** Work through the exercises before consulting these answers. The value is in making and then evaluating your own observations. --- ## Exercise 1 — Embed a string and inspect the vector **Expected output — `mpl embed "the cat sat on the mat" --json`** ```json {"vector": [-0.0234, 0.0817, -0.0451, ..., 0.0312]} ``` The `"vector"` key maps to an array of exactly **384 floating-point numbers**. The values are real-valued coordinates in 384-dimensional space, typically in the range −1 to 1. **Expected output — norm computation pipeline** ``` dimension: 384 L2 norm: 1.000000 ``` (Or within floating-point rounding, e.g. `0.999999` or `1.000001`.) Confirm both values on your box. **What the norm confirms:** The all-MiniLM-L6-v2 model's pipeline ends in a `Normalize (L2)` module. Every `.encode()` call returns unit-norm vectors regardless of any flag the caller passes. This means: - For unit-norm vectors **u** and **v**: `||u − v||² = 2 − 2(u · v) = 2(1 − cos θ)`. - L2 distance ranking and cosine similarity ranking produce the same ordering. - LanceDB's default L2 metric and cosine similarity are therefore equivalent for all vectors in the `mpl` store. --- ## Exercise 2 — Index the corpus and inspect the store **Expected output — `mpl index corpus/`** ``` indexed 7 chunks from 5 file(s) ``` The 7 chunks break down as: | File | Char count | Chunks | |---|---|---| | `corpus/01-pacific-ocean.md` | 381 | 1 | | `corpus/02-photosynthesis.md` | 418 | 1 | | `corpus/03-roman-republic.md` | 396 | 1 | | `corpus/04-speed-of-light.md` | 401 | 1 | | `corpus/05-nitrogen-cycle.md` | 1320 | **3** | **Expected output — `cat ./.mpl/index.json`** ```json { "model": "sentence-transformers/all-MiniLM-L6-v2", "revision": "1110a243fdf4706b3f48f1d95db1a4f5529b4d41", "dim": 384, "table": "documents", "chunk_size": 512, "chunk_overlap": 64 } ``` All six fields exactly as above. The `dim` field is derived from `len(vectors[0])` — it is the actual dimension of the first vector computed, not a hardcoded constant. **The chunking arithmetic for `05-nitrogen-cycle.md`:** Stride = `512 − 64 = 448` characters per step: ``` chunk 0: chars 0 – 511 (512 chars) starts: "The nitrogen cycle is..." chunk 1: chars 448 – 959 (512 chars) starts: "nia (NH3), which plants can absorb..." chunk 2: chars 896 – 1319 (424 chars) starts: "orb nitrate from the soil..." ``` The 64-character overlap is visible: the last 64 characters of chunk 0 (roughly `"nia (NH3), which plants can absorb through their roots.\n\nOnce in"`) are identical to the first 64 characters of chunk 1. This is the only mechanism for mitigating context loss at boundaries. **What the store structure looks like:** ``` ./.mpl/ index.json ← metadata (model, revision, dim, table, chunk_size, chunk_overlap) lancedb/ ← LanceDB directory documents.lance/ ← the table holding source, chunk_id, text, vector records ``` Each record in the `"documents"` table has the shape: ```json { "source": "corpus/05-nitrogen-cycle.md", "chunk_id": 1, "text": "<512 characters of chunk text>", "vector": [384 floats] } ``` --- ## Exercise 3 — Search the index and read a ranked result **Expected output shape — `mpl search "how does nitrogen return to the atmosphere"`** The top results should be chunks from `corpus/05-nitrogen-cycle.md` (especially chunk_id 2 which contains the denitrification text). A typical ranked list looks like: ``` rank: 1 score: 0.7XXX source: corpus/05-nitrogen-cycle.md chunk_id: 2 text: orb nitrate from the soil. The cycle closes through denitrification: anaerobic bacteria in waterlogg... rank: 2 score: 0.7XXX source: corpus/05-nitrogen-cycle.md chunk_id: 1 text: nia (NH3), which plants can absorb through their roots. Once incorporated into organic m... ... (ranks 3–5 will be from other corpus files with lower scores) ``` The exact scores are live-verified post-deploy. Expect the nitrogen cycle chunks to score notably higher than the unrelated files (Pacific Ocean, photosynthesis, Roman Republic, speed of light). **Expected output shape — `mpl search ... --json`** Each result object: ```json { "rank": 1, "score": 0.7XXX, "source": "corpus/05-nitrogen-cycle.md", "chunk_id": 2, "text": "orb nitrate from the soil.\n\nThe cycle closes through denitrification..." } ``` **The score formula (exact):** `score = round(1.0 − _distance, 4)` where `_distance` is LanceDB's `l2` metric, which returns the **squared** Euclidean distance `||u − v||²`. On unit-norm MiniLM vectors: - `_distance = 1 − score` (the squared-L2 distance) - `cos θ = (score + 1) / 2` - Ranking by score is identical to ranking by cosine similarity. Higher score = closer to the query = more semantically similar. The score is a **monotonic ranking signal**, not a calibrated [0,1] similarity percentage. A score of 0.72 does not mean "72% semantically similar." **Lesson Exercise 2 — Score-formula reasoning (answer):** For scores 0.8812 and 0.6234: | | Score | Squared-L2 `_distance` (= 1 − score) | cos θ (= (score + 1) / 2) | |---|---|---|---| | Chunk A | 0.8812 | **0.1188** | **0.9406** | | Chunk B | 0.6234 | **0.3766** | **0.8117** | Chunk A (score 0.8812) is the closer match. Neither score is a percentage: you cannot say the first chunk is "88% semantically similar." The score is a monotonic ranking signal derived from standard squared-L2 distance on unit-norm vectors; its absolute value reflects the geometry of the embedding space, not a calibrated percentage. **The flat scan confirmation:** `mpl` calls `tbl.search(query_vec).limit(top_k).to_list()` with no preceding `.create_index()`. LanceDB's default is brute-force flat scan — 100% recall, O(N × d) per query. At 7 chunks this is instantaneous. --- ## Exercise 4 — Trip the model-match guard **Expected output — the Python edit command** ``` model field set to: sentence-transformers/all-mpnet-base-v2 ``` The `index.json` file now contains `"model": "sentence-transformers/all-mpnet-base-v2"` while the `mpl` binary is still using `sentence-transformers/all-MiniLM-L6-v2`. **Expected error — `mpl search "..."`** The exact error string (from `retrieval.rs check_model_match`): ``` index was built with model 'sentence-transformers/all-mpnet-base-v2' but active model is 'sentence-transformers/all-MiniLM-L6-v2'; re-run `mpl index ` to rebuild the store ``` The command exits with status code 1. **Why the guard exists (the mathematical argument):** Two embedding models produce vectors in different learned geometric spaces. Even at the same dimension (384), the axes are incompatible: - Model A's axis 42 encodes something shaped by Model A's training distribution and random seed. - Model B's axis 42 encodes something shaped by Model B's training distribution and random seed. - These are different directions in different spaces. There is no rotation or translation that maps one to the other. A query vector from the active model (MiniLM-L6-v2) compared against document vectors indexed by the fake model id is not an approximation — it is geometrically meaningless. The L2 distances computed are arbitrary numbers; the ranking they produce is random noise. The guard keys on the **model id string**, not on `dim`. Correct: same `dim` does not mean same geometry. **Expected output — after `mpl index corpus/` restoration** `cat ./.mpl/index.json` returns the correct `"model": "sentence-transformers/all-MiniLM-L6-v2"` field, and `mpl search` works again. --- # Solutions — Lab 02: Retrieval-Augmented Generation *Expected output shapes (live-verified post-deploy). Run these on your own enrolled box; values like exact scores and answer wording will vary.* > **For self-checking.** Work through the exercises before consulting these answers. --- ## Exercise 1 — Inspect the full pipeline with `--dry-run` **Expected output — `mpl rag "how does nitrogen return to the atmosphere" --dry-run`** Two plans are printed before any execution. **Plan 1 — search plan:** ``` → python3 search.py --full -k 5 "how does nitrogen return to the atmosphere" ``` The `--full` flag is key: it retrieves untruncated chunk text (overriding the 200-character display truncation). `mpl rag` needs the full text to assemble the context window. **Plan 2 — generate plan / assembled prompt:** The exact structure of `assemble_prompt(query, chunks)` as defined in `rag.rs`: ``` You are answering the question using only the provided context. Cite sources by their bracket number, like [1]. If the context does not contain the answer, say you don't know. Context: [1] () [2] () [3] () [4] () [5] () Question: how does nitrogen return to the atmosphere Answer: ``` For this query, sources 1–3 will be nitrogen cycle chunks (most likely chunk_ids 2, 1, 0 from `corpus/05-nitrogen-cycle.md` in descending relevance order). Sources 4–5 will be from the less-related files (speed of light, Pacific Ocean, etc.). **Four things to confirm from the `--dry-run` output:** 1. The system instruction is verbatim: `"You are answering the question using only the provided context. Cite sources by their bracket number, like [1]. If the context does not contain the answer, say you don't know."` 2. Each context line follows `[n] (source) text` — bracket number, file path in parentheses, full chunk text. 3. The `Answer:` slot at the end is empty — the model completes this continuation. 4. The highest-ranked chunks appear first (rank 1 = `[1]`, rank 2 = `[2]`, …). The model is more likely to attend to evidence at the beginning and end of the context (as shown by Liu et al., 2023 "Lost in the Middle"), so the most relevant chunks being first is the correct placement. --- ## Exercise 2 — Run a grounded query and read the cited answer **Expected output structure — `mpl rag "how does nitrogen return to the atmosphere"`** ``` Retrieved sources: [1] corpus/05-nitrogen-cycle.md (chunk 2, score 0.7XXX) [2] corpus/05-nitrogen-cycle.md (chunk 1, score 0.7XXX) ... The question asks about nitrogen returning to the atmosphere. Looking at [1], it mentions denitrification: anaerobic bacteria convert nitrates back into gaseous nitrogen or nitrous oxide. This directly answers the question. [2] provides additional context about the cycle. Nitrogen returns to the atmosphere through denitrification: anaerobic bacteria in waterlogged soils convert nitrates back into gaseous nitrogen or nitrous oxide [1]. This closes the nitrogen cycle, returning the nitrogen fixed and processed by living organisms back to the atmospheric pool [2]. ``` (The exact wording of the `` block and the answer will vary across runs due to sampling. The key features to confirm: the `` block reasons over the numbered sources; the answer cites `[1]` and/or `[2]`; the answer is factually grounded in the denitrification text from `corpus/05-nitrogen-cycle.md`.) **What the citations confirm:** The `[1]`, `[2]` brackets correspond to the `[n] (source) text` entries in the assembled prompt. When the model writes `[1]`, it is pointing to the first context entry — in this case, the nitrogen cycle chunk ranked highest by `mpl search`. You can verify by comparing the cited claim to the chunk text shown in the `--dry-run` output. **What the citations do NOT confirm:** The model could cite the wrong bracket, interpolate from training knowledge, or paraphrase incorrectly. The `` block shows the model's citation reasoning — useful for auditing — but the reasoning itself can be wrong. Citations make verification possible; they do not perform it. --- ## Exercise 3 — `--json` for stdout-pure output **Expected output — `mpl rag "what is the deepest point in the Pacific Ocean" --json`** ```json { "answer": "The deepest known point in the Pacific Ocean is in the Mariana Trench, at roughly 11 kilometres below sea level [1].", "backend": "llama.cpp", "model": "qwen3:8b", "query": "what is the deepest point in the Pacific Ocean", "retrieved": [ { "rank": 1, "score": 0.XXXX, "source": "corpus/01-pacific-ocean.md", "chunk_id": 0, "text": "The Pacific Ocean is the largest and deepest ocean on Earth. It covers approximately 165 million square kilometres, more than all of Earth's landmass combined. The Mariana Trench, located in the western Pacific, contains the deepest known point on Earth at roughly 11 kilometres below sea level. The ocean is bounded by Asia and Australia to the west and the Americas to the east.\n" }, ... ] } ``` **What confirms stdout purity:** ```bash mpl rag "what is the deepest point in the Pacific Ocean" --json | python3 -m json.tool ``` `python3 -m json.tool` would fail with a parse error if any non-JSON line (teaching output, transparency lines) were present on stdout. A clean parse confirms stdout carries only the JSON object. **The five fields in the JSON response:** | Field | Content | |---|---| | `answer` | The model's answer text (string) | | `backend` | The inference backend used (`"llama.cpp"` or `"mlx"`) | | `model` | The generation model (`"qwen3:8b"` by default) | | `query` | The original query string | | `retrieved` | Array of retrieved chunk objects with rank, score, source, chunk_id, text | The `retrieved` array contains full (untruncated) chunk text — the same data that fed the assembled prompt. Useful for programmatic auditing. **Where the teaching lines went:** With `--json`, all teaching output goes to stderr. Run with stderr visible to confirm: ```bash mpl rag "what is the deepest point in the Pacific Ocean" --json 2>&1 | head -20 ``` You should see the teaching lines on stderr followed by the JSON object (if stderr and stdout are merged). Or separate them: ```bash mpl rag "..." --json > answer.json 2> teaching.txt ``` --- ## Exercise 4 — Compare `-k 2` vs `-k 8` **Expected observations:** | Command | Context entries | Coverage | |---|---|---| | `mpl rag "what is the nitrogen cycle" -k 2 --dry-run` | 2 `[n]` entries | Top-2 chunks by score — likely the two most directly relevant nitrogen cycle chunks, but depending on the ranking (confirm on your box) may miss the nitrogen fixation context if it ranks outside the top 2 | | `mpl rag "what is the nitrogen cycle" -k 8 --dry-run` | Up to 7 `[n]` entries (budget permitting) | All 7 indexed chunks — with this small corpus, all survive `trim_to_budget` since total chars ≈ 3,000 (well within the 12,000-character budget) | **The `trim_to_budget` rule (exact from `rag.rs`):** - Budget: `RAG_MAX_CONTEXT_CHARS = 12_000` characters - Walk rank 1, 2, 3, … accumulating `text.chars().count()` - Drop the chunk when adding it would exceed the budget (drop lowest-ranked first) - Always keep at least rank 1 regardless of size With the provided corpus (7 chunks, ~3,000 total characters), no trimming occurs at any `-k` value ≤ 7. To see trimming in action, index a larger corpus with more files. **Why the drop order matters:** Keeping the highest-ranked chunks (most semantically relevant) and dropping the lowest-ranked (least relevant) ensures the most answer-bearing evidence survives the budget. The "Lost in the Middle" finding (Liu et al., 2023) shows that placing relevant evidence at the beginning of the context (rank 1 = `[1]`) maximises the model's use of it. More chunks is not always better — the 12 000-character budget enforces selection discipline. --- ## Exercise 5 — Out-of-corpus question: the honest "I don't know" **Expected output — `mpl rag "what is the time complexity of merge sort"`** The retrieved sources will be the closest-in-embedding-space chunks to "time complexity of merge sort." With no programming or computer science content in the corpus, these will be marginally relevant at best — possibly something about the nitrogen cycle (scientific/technical vocabulary) or the Roman Republic (historical vocabulary). The expected model response: ``` I don't know. The provided context does not contain any information about merge sort or its time complexity. ``` (Or similar. The exact phrasing varies. The key: the model does not fabricate an answer from training knowledge.) **Two mechanisms at work:** 1. **`mpl search` always returns results.** It cannot return "no results" — it returns the top-k closest chunks from whatever is in the store. When the topic is absent from the corpus, the retrieved chunks have low relevance (lower absolute scores than Exercise 2's results). But low score does not prevent retrieval — at least one chunk always reaches the prompt. 2. **The system instruction licenses refusal.** The assembled prompt contains: ``` If the context does not contain the answer, say you don't know. ``` This clause instructs the model to report insufficient context rather than hallucinate. A well-behaved model recognises that text about denitrification and light speed does not support claims about merge sort and declines to answer. **What to compare:** | Query type | Top scores (indicative) | Model response | |---|---|---| | In-corpus (nitrogen cycle) | ~0.70–0.85 | Grounded answer with citations | | Out-of-corpus (merge sort) | ~0.30–0.55 | "I don't know" | (Exact score ranges are live-verified post-deploy. The important observation is the relative difference — in-corpus queries score noticeably higher than out-of-corpus queries.) The `--dry-run` output for the out-of-corpus query is instructive: look at the assembled context and confirm that the retrieved chunks genuinely contain no information about merge sort. The model's "I don't know" is correct and grounded.

connect to continue