WEKA and Oracle Demonstrate Significant AI Inference Gains

WEKA, an AI data and memory infrastructure firm, announced impressive production benchmarks with Oracle Cloud Infrastructure (OCI), showcasing how to enhance long-context AI inference efficiency. The benchmarks highlight WEKA's NeuralMesh™ platform with Augmented Memory Grid™ on OCI, achieving 10x more concurrent users, 10x higher token throughput, and 7x more tokens per GPU compared to traditional DRAM-only setups. These outcomes are achieved without adding new GPUs, offering a cost-effective solution for growing AI demands.

Conducted on a nine-node OCI H100 cluster, the benchmarks showed the platform's capability to serve over 5,000 concurrent users and process approximately two million tokens per second. This advancement removes memory bottlenecks that limit enterprise AI workloads, enabling support for larger inference tasks. The findings, validated under real-world conditions, present a paradigm shift in reducing AI inference costs while escalating performance.