KV cache calculator
Calculate KV cache memory requirements for any model on supported GPU systems
Checking...
Tested: , ... — type to autocomplete
Gated model? Add your HF token in Settings →
Select a model and GPU system, then click Calculate
Calculate KV cache memory requirements for any model on supported GPU systems