Redis has put LangCache into public preview, a managed cache that keeps generated answers rather than the model state that produced them.
Prefix caching, the usual way to save money, reuses computed key-value states for a shared prompt but still sends the request through the model. A prefix hit is a cheaper generation, not an avoided one. LangCache targets the whole call. The service embeds the incoming question, looks for a stored response above a similarity threshold and returns it if it finds one. Redis handles the embedding itself and exposes the system through a REST API with Python and JavaScript SDKs.
Its own demo shows the gap. Direct inference on a paraphrased support question took 2.232 seconds and consumed 514 input tokens plus 250 output tokens. The cached path returned the earlier answer in 0.37 seconds with no tokens spent.
The math needs care. A hit erases output-token charges, but the input-token side is usually eaten by embedding and storage costs, so Redis frames the benefit as monthly output spend times hit rate. Put $200 a month through the system, with 60 percent of that on output tokens and half the queries hitting cache, and the figure comes to roughly $60. Marketing has run ahead of the arithmetic at times: launch copy promised hits up to 15x quicker and token use down as much as 70 percent, while the product page now cites savings up to 90 percent. One customer running a patient-care voice app reports a 70 percent hit rate, with spend down 70 percent.
The hard part is operational. A loose threshold can return a refund policy to someone asking about upgrades, while a tight one sends nearly every paraphrase back to the model. LangCache offers access scopes, filtering, TTLs and eviction controls for that tuning.
