Recently, Google Research presented a promising compression algorithm called TurboQuant, which promises to revolutionize the handling of working memory in artificial intelligence (AI) models, reducing its size by up to six times without sacrificing quality or performance.
This compression technique focuses on the KV cache, a crucial component that stores the tokens processed during long conversations, and whose growth is one of the main bottlenecks in the AI inference phase.
Memory prices drop as TurboQuant redefines resource optimization in AI
In more technical terms, TurboQuant uses a vector quantization method, which could not only make it easier for data centers to operate with fewer hardware resources but could also significantly impact the DRAM and HBM memory manufacturing industry.
Falls have already begun to be noticed in the stocks of companies like Micron and Samsung, which experienced a notable reduction in their value, hinting that the golden age of the memory industry may be coming to an end.
The discovery has evoked comparisons with the so-called “DeepSeek moment” of Google, in which a Chinese startup launched a competitive AI model at a significantly lower development cost.
Before the official presentation of TurboQuant’s results, some analysts believe that this advancement could be a game changer in the sector, similar to what happened with DeepSeek.
Despite its potential, it is important to note that TurboQuant only affects the inference phase. The training phase of AI models still requires large amounts of memory.
The changes in memory module prices have begun to be evident, with a notable drop in the cost of products like Corsair Vengeance DDR5 modules. The race for resource optimization in AI is advancing, and it remains to be seen how companies will adapt to these new techniques and what implications they will have for the future.