Hugging Face's logo

harshithsaiv
/
kv-cache-compression

English
quantization
kv-cache
llm-inference
cuda
triton
memory-efficient
mitral
llama
inference-optimization
4-bit precision
mixed-precision
Model card Files
xet
 Community
Harshith Sai Veeraiah commited on
Commit
598ef59
·
verified ·
1 Parent(s): 1d884be

Upload 17 files

Browse files
Files changed (18) hide show
  1. .gitattributes +8 -0
  2. compression_bar.png +0 -0
  3. compression_bar_4methods.png +3 -0
  4. compression_bar_both.png +0 -0
  5. llama-3-8b_sensitivity_heatmap.png +0 -0
  6. long_context_4methods.png +3 -0
  7. long_context_both.png +3 -0
  8. memory_32k_4methods.png +0 -0
  9. memory_vs_context.png +3 -0
  10. memory_vs_context_4methods.png +3 -0
  11. memory_vs_context_both.png +3 -0
  12. mistral-7b_sensitivity_heatmap.png +0 -0
  13. oom_story.png +3 -0
  14. prefill_latency_both.png +0 -0
  15. results_table.png +0 -0
  16. results_table_4methods.png +3 -0
  17. results_table_both.png +0 -0
  18. sensitivity_heatmap.png +0 -0
.gitattributes CHANGED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ compression_bar_4methods.png filter=lfs diff=lfs merge=lfs -text
2
+ long_context_4methods.png filter=lfs diff=lfs merge=lfs -text
3
+ long_context_both.png filter=lfs diff=lfs merge=lfs -text
4
+ memory_vs_context_4methods.png filter=lfs diff=lfs merge=lfs -text
5
+ memory_vs_context_both.png filter=lfs diff=lfs merge=lfs -text
6
+ memory_vs_context.png filter=lfs diff=lfs merge=lfs -text
7
+ oom_story.png filter=lfs diff=lfs merge=lfs -text
8
+ results_table_4methods.png filter=lfs diff=lfs merge=lfs -text
compression_bar.png ADDED
compression_bar_4methods.png ADDED

Git LFS Details

  • SHA256: a7b3118188acce9f0e82b43c038677865f89ab4fa151e4708ee8dd164ab529bb
  • Pointer size: 131 Bytes
  • Size of remote file: 111 kB
compression_bar_both.png ADDED
llama-3-8b_sensitivity_heatmap.png ADDED
long_context_4methods.png ADDED

Git LFS Details

  • SHA256: 2a2f50502e36fd284955d4853af24fc1c57c44e22d54b7330d5030f961c746fc
  • Pointer size: 131 Bytes
  • Size of remote file: 202 kB
long_context_both.png ADDED

Git LFS Details

  • SHA256: f2018345e825367b1c7425a4de1dbd7265a82190012f5b308514eed6f5529df7
  • Pointer size: 131 Bytes
  • Size of remote file: 206 kB
memory_32k_4methods.png ADDED
memory_vs_context.png ADDED

Git LFS Details

  • SHA256: 42ae52641b7ed339289c5e80eae20fbb47cd16c84c873a7d0055b4fcc04a982b
  • Pointer size: 131 Bytes
  • Size of remote file: 117 kB
memory_vs_context_4methods.png ADDED

Git LFS Details

  • SHA256: 569d995194a7bf9ac0d5c6b8eada1ec43fc2298af3041cb11b024c3509212790
  • Pointer size: 131 Bytes
  • Size of remote file: 192 kB
memory_vs_context_both.png ADDED

Git LFS Details

  • SHA256: 88d97413bf625645dead8d253ce2f97e3b2dcd944e8c788c7b73e195dda599f8
  • Pointer size: 131 Bytes
  • Size of remote file: 166 kB
mistral-7b_sensitivity_heatmap.png ADDED
oom_story.png ADDED

Git LFS Details

  • SHA256: 4097b1be5c190a6a6089109fa2462e418642803fb57276a22a17a7b78d6c3b2a
  • Pointer size: 131 Bytes
  • Size of remote file: 107 kB
prefill_latency_both.png ADDED
results_table.png ADDED
results_table_4methods.png ADDED

Git LFS Details

  • SHA256: b2ccb92e2101dac9138587f13f6426d5f8d3f81d144626cd6b37890099cdc91a
  • Pointer size: 131 Bytes
  • Size of remote file: 113 kB
results_table_both.png ADDED
sensitivity_heatmap.png ADDED