The notebook / 65 articles
Notes on LLMs, machine learning, data engineering, and the systems that make them work.
Showing 1–10 of 65 posts
A cleaner mental model for LLM inference: prompt templating, tokenization, scheduling, prefill, KV cache, decode, and the split between the inference engine and the model.
Separate predictions, policy, and execution with Jev and LangChain. Follow a refund through middleware, inspect review thresholds, and compare recorded decision latency with...
Carry authorization through retrieval, context, caches, and citations. Define trust boundaries, revocation behavior, and operational controls.
Separate retrieval coverage, context quality, and answer support. Calculate ranking metrics and design reproducible regression gates.
Separate schema validity, citation identity, and factual support; design bounded repair, honest abstention, and streaming behavior around those distinct guarantees.
Translate questions into bounded retrieval plans, preserve metadata constraints, and verify SQL result semantics with a worked incident-table example.
Work through reciprocal rank fusion, distinguish full-text search from BM25, and evaluate reranking and corrective retrieval without treating scores as confidence.
Turn retrieved candidates into useful context by preserving qualifiers, tracking expansion provenance, budgeting tokens, and separating image retrieval from visual answering.
Understand vector geometry, pin the full encoding contract, and evaluate approximate search under real filters before migrating an embedding index.
Design document ingestion around immutable representations, idempotent jobs, and atomic publication so failed updates cannot silently replace searchable evidence.