KV-cache and inference papers, studied by building them.
Cache Atlas is my public notebook for learning modern inference systems by implementing the core mechanics. The focus stays specific: KV-cache memory, decode bottlenecks, paper reproductions, and small experiments you can run locally.
- Why KV cache papers matter
A map of the memory problem that motivated the first implementation.
- Reproducing MiniCache in PyTorch
The API, tests, benchmark command, and the exact scope of the current reproduction.
- Adding a KV-cache object
Moving from tensor primitives toward a decode-time cache interface.
- KV-cache memory
Cache growth, compression axes, retention policies, and decode-time memory pressure.
- Paper reproductions
Small implementation tracks that turn one paper mechanism into runnable code.
- Visual demos
Animations and interactive components that make the systems behavior easier to inspect.
- Benchmarks
Minimal scripts that measure the implemented path before scaling up the experiment.
- minicache-pytorch
Standalone PyTorch package with clean-room primitives, tests, examples, and benchmark scripts.
- prefixkv-pytorch
Planned next track for token and layer budget allocation in vision-language generation.
Next up: a decode benchmark. The useful version should measure longer sequence lengths, retained-token fraction, and the cost of applying compression across selected layer pairs.