STRUCTSPARSE
Identical when you need proof. Better when you don't.
Deterministic KV-cache eviction for vLLM. More concurrent requests on the hardware you already run.
The cache is the constraint.
Every token a model reads holds cache memory until its request completes. That memory, not compute, is what caps concurrency.
Eviction today is a bet.
Runtime attention scores decide what survives. Unpredictable before the request runs, unauditable after it, silent when it fails.
Decisions fixed at enqueue.
No learned parameters in the decision path. Same request, same decision, every time, knowable before the first token of the response is computed.
The system
Full context was never the gold standard. It was the only option that did not require trusting a heuristic. StructSparse replaces the heuristic with a rule.
Deterministic
No learned parameters, no scores, no thresholds, no neural forward pass. The eviction decision is a pure function of the request: same input, same decision, every time.
Fixed at enqueue
The keep-set is known before the first token of the response is computed. Capacity stops being a runtime surprise and becomes something you can schedule, admit against, and price.
Certification mode
Outputs are byte-identical to stock vLLM. Not comparable, not within noise: diff returns nothing.
Performance mode
Outputs diverge from stock, and on blind evaluation they match or beat full-context baselines. Divergence is not degradation.
Drop-in
No retraining, no model modification, no new hardware. It is vLLM, with the cache under management.
Verification protocol
Don't believe us. Diff us.
We publish no benchmark tables. Benchmark numbers describe our traffic, not yours, and every eviction paper ever written claims minimal degradation. The only number that matters is the one your own workload produces. The protocol takes an afternoon.
Arm 1: certification
Byte-diff every output
Route a sample of production requests through stock vLLM and through StructSparse in certification mode: sampling held deterministic, matched serving configuration, elevated concurrency on the StructSparse side.
Expected result: zero divergence. No attention-score method survives this arm.
Arm 2: performance
Blind against full context
Switch to performance mode. Run your own evaluations blind against full-context outputs on the same requests.
Expected result: parity or better, at higher capacity.
If either arm fails on your traffic, you have spent an afternoon and learned we were wrong. That is the entire downside.
Run it on your trafficWhat we won't say yet
We do not disclose the decision mechanism ahead of patent publication. The verification protocol exists so that, in the meantime, you take nothing on faith.
19/546,901
US non-provisional. Priority February 2026.
Patent pending US 19/546,901Evaluated on production traffic with a design partnerBuilt on vLLMStructSparse Inc. Delaware
For inference platforms and silicon vendors: detailed results, and the mechanism, under NDA.
hello@structsparse.com