RealFeedApp KVGuard detected the KV cache leak bug in SGLang
The team built KVGuard, a sanitizer for LLM serving systems that detects a bug where the server reads data from a shared KV cache buffer belonging to other users. Testing on SGLang 0.5.19 revealed 4095 such uninitialized reads.
- Field
- computer security
- What they did
- Built KVGuard, a tool that detects memory reading bugs where server systems accidentally read data from other users' caches.
- Why it matters
- This prevents information leaks when requests return HTTP 200 but contain hidden fragments of other users' messages.
#kv cache#llm#serving engine#memory safety#sglang#bug
