SQLite as a memory engine

Store lab: one SQLite file holds records, float32 vectors and a full-text index — 24 records in 13 pages of 4,096 bytes, 53,248 bytes, of which 9 pages are the vectors, because 384 dimensions of float32 is a fixed 1,536 bytes whatever the text says. One transactional write appends 5 WAL frames (20,480 bytes) and 200 writes reach the 1,000-page checkpoint; 500 rows batched cost 2,001 frames against 2,500, 19.96% fewer. The term index answers 9 of 12 questions with 4 empty lists, the vector answers 11 and can never return nothing, and both lists answer 11 while putting the right row first only 8 times — a union is not a ranker. Delete the vec0 rows for 10 of the 24 records and every path falls to 4 of 12, which is why the drill hands you a reconciliation query rather than a better retriever.

This Memory Engineering lesson is part of a paid plan. Sign in with a plan that includes this course to read it.