notes
Paper notes and implementation writeups.
These notes track my learning process in public: one mechanism at a time, with enough implementation detail, visuals, and reproduction notes to make the code path inspectable.
- Why KV cache papers matter
The opening map of the memory problem behind modern inference serving.
- Reproducing MiniCache in PyTorch
The first implementation report, centered on arXiv:2405.14366.
- Adding a KV-cache object
The second implementation report: applying the pairwise path to stored keys and values.
categories
- KV-cache memory
Cache footprint, decode bottlenecks, and compression strategies.
- Implementation notes
Reproducible code paths with tests, examples, and benchmark commands.
next note
Next up: a decode benchmark note that makes the latency and memory tradeoff visible.