As reported by MIT Tech Review AI, Subquadratic’s claims hinge on sparse attention, a technique that reduces computational load by selectively processing token relationships. While the company’s third-party benchmarks are promising, the lack of public access to SubQ raises questions about scalability and real-world performance. Critics argue that sparse attention has been attempted before without matching dense attention’s effectiveness.
If Subquadratic’s breakthrough holds, it could democratize LLM development by lowering costs and energy consumption, but the industry should remain cautious until independent researchers can replicate the results.