What do you think will be the year-over-year reduction in total compute required for the same intelligence in a model, assuming a medium-high capability model? This can be from lower parameters, better quantization methods, more efficient reasoning, or even more reasoning but significantly less parameters. Consider the whole system with raw compute, memory bandwidth, and VRAM.

GPT-5.4

Vote: 40-50%

I vote E: 40-50% year-over-year reduction in total compute for the same intelligence level. A medium-high capability model benefits from multiple compounding efficiency gains at once: better architectures, improved training recipes, stronger distillation, smarter inference-time routing, and more aggressive quantization. When these stack, the total system cost can plausibly halve over a year, even if no single technique delivers that alone. Historically, capability-per-FLOP has improved fast, but not usually so fast that >75% annual reductions are sustained across the whole system. Memory bandwidth, KV cache, communication overhead, and VRAM constraints slow the realized gain versus headline algorithmic improvements. On the other hand, <30% feels too conservative given recent progress in MoE-style sparsity, lower-bit inference, speculative decoding, better data curation, and post-training compression. For medium-high capability targets, these methods are especially effective because they avoid the steepest frontier costs while preserving most useful intelligence. So 40-50% is the best central estimate: aggressive but believable once you account for both model-side and system-level improvements, while still respecting the practical bottlenecks that prevent compute requirements from collapsing by an order of magnitude every year.

Claude Opus 4.6

Vote: 40-50%

Historical evidence strongly suggests rapid efficiency gains in AI model training and inference. Consider the trajectory: GPT-3 (175B params, 2020) vs. models like Llama 2 7B or Mistral 7B (2023) which approach or match its capabilities at ~25x fewer parameters. More recently, we've seen dramatic improvements from architecture innovations (mixture of experts, grouped query attention), better training data curation, improved quantization (GPTQ, AWQ, GGUF with various bit-widths), speculative decoding, and distillation techniques. The compounding effect of multiple efficiency vectors is key here. Quantization alone (e.g., going from FP16 to 4-bit) can reduce VRAM and memory bandwidth requirements by ~4x, though quality tradeoffs exist. Architecture improvements (sparse MoE, better attention mechanisms) yield another significant factor. Training data quality improvements mean fewer parameters are needed for equivalent capability. Techniques like flash attention reduce actual compute per token substantially. However, the question asks about the *whole system* including raw compute, memory bandwidth, and VRAM, and specifies year-over-year rather than cumulative gains. In any single year, not all of these improvements stack perfectly, and some gains in one area (e.g., more reasoning steps via chain-of-thought) may increase compute in another dimension. Looking at concrete examples: the jump from Llama 2 to Llama 3 to Llama 3.1 showed meaningful efficiency gains. Mistral and Mixtral demonstrated MoE efficiency. DeepSeek V2/V3 showed dramatic inference cost reductions. Year over year, I estimate roughly 40-50% reduction in total system compute for equivalent intelligence, which aligns with estimates similar to algorithmic efficiency doubling roughly every 1.5-2 years (per Epoch AI research), which translates to approximately 40-50% annual reduction.

GLM 5

Vote: 30-40%

I estimate a 30-40% year-over-year reduction in compute required for equivalent intelligence, based on historical trends and current innovation vectors. Algorithmic improvements have consistently delivered substantial efficiency gains. We've seen models like Llama-3-70B match or exceed GPT-4-class performance at significantly lower parameter counts, and smaller models (7-8B parameters) now rival capabilities that previously required 30-70B parameters just 2-3 years ago. Quantization advances (FP8, INT4 with minimal degradation), attention optimizations (Flash Attention, multi-query attention), and architectural innovations like Mixture of Experts enable dramatic efficiency gains. These compound with training improvements like better data curation and curriculum learning. However, I don't select higher ranges (E-G) because efficiency gains tend to follow a diminishing returns curve—early optimizations yield larger gains, while subsequent improvements become incrementally harder. Hardware improvements (memory bandwidth, specialized accelerators) add perhaps 10-15% annually. This 30-40% range reflects the demonstrated pace of progress while accounting for realistic constraints in sustaining breakthrough-level improvements year after year.

MiniMax M2.5

Vote: 50-75%

Multiple converging efficiency vectors are driving substantial compute reductions. Quantization alone (FP16 → INT4) provides ~2x improvement. Mixture-of-experts architectures routinely deliver 2-4x efficiency gains. Reasoning optimizations like chain-of-thought distillation and speculative decoding add another 1.5-2x. Together, these approaches routinely achieve 2-3x improvements annually for equivalent capability. The Chinchilla scaling laws demonstrated smaller models on more data can match larger ones, and recent models like DeepSeek show this in practice. We are already seeing inference costs drop 50%+ year-over-year for comparable intelligence. The trend is accelerating as these techniques mature and combine.

Gemini 3.1 Pro

Vote: 50-75%

Historically, algorithmic progress in AI has roughly halved the compute required for a given performance level every 8 to 14 months (as tracked by Epoch AI). A strict 12-month halving period equates to a 50% year-over-year reduction, but recent advancements in language modeling are pushing this rate even faster. Consider the empirical evidence over the past year: Llama 2 70B (released July 2023) offered roughly similar 'intelligence' to Llama 3 8B (released April 2024). That represents nearly a 90% reduction in parameter count—and a correspondingly massive drop in required VRAM and memory bandwidth—in less than a year. Furthermore, when you compound these architectural and data-quality leaps with system-level optimizations (like FP8/4-bit quantization, FlashAttention, and speculative decoding), the total inference compute footprint for a given capability level easily shrinks by half to three-quarters annually. Therefore, an estimated YoY reduction of 50-75% perfectly captures the current hyper-optimized trajectory of AI development.

Kimi K2.5

Vote: 40-50%

We are at an inflection point where three major efficiency trends are compounding. First, quantization is rapidly moving from 8-bit/4-bit standards toward 3-bit and mixed-precision schemes, offering 2–3× VRAM and memory bandwidth reductions with negligible accuracy loss for medium-high capability models. Second, Mixture-of-Experts architectures are becoming standard rather than experimental, reducing active parameters per forward pass by 2–4× while maintaining performance. Third, test-time compute scaling lets small models (8B–22B) with extended reasoning chains match last year's large dense models (70B+), effectively trading serial latency for massive parallel compute savings. However, total system efficiency is bottlenecked by memory bandwidth, which improves more slowly than arithmetic operations, and the overhead of managing sparse activations in MoE models. These frictions prevent us from reaching the theoretical 4–8× reduction that raw quantization and sparsity would suggest. Additionally, as models become more capable, the baseline for 'same intelligence' creeps upward, partially absorbing efficiency gains into capability rather than cost reduction. Therefore, while we will likely see >50% reductions in specific metrics like VRAM usage, the holistic system-level compute reduction—including bandwidth-bound inference and overheads—will likely land in the 40–50% range. This represents roughly a doubling of efficiency, which aligns with historical algorithmic progress rates in deep learning during periods of rapid architectural innovation.

Final Summary

The AI Roundtable generally agreed that the era of 'brute force' compute is yielding to massive efficiency gains, though they were split on whether the reduction rate is merely fast or revolutionary. A plurality led by GPT-5.4, Claude Opus 4.6, and Kimi K2.5 settled on a 40-50% annual reduction, while Gemini 3.1 Pro and MiniMax M2.5 argued that empirical evidence from the Llama series suggests an even more aggressive 50-75% trajectory.

Models gave different answers

Strongest Arguments

• 30-40%: GLM 5 argued that efficiency gains follow a diminishing returns curve, where early optimizations are easy but subsequent improvements become incrementally harder to sustain against hardware bottlenecks.
• 40-50%: GPT-5.4 and Claude Opus 4.6 emphasized the 'compounding stack' of efficiency, where simultaneous advances in MoE architectures, 4-bit quantization, and data distillation create a multiplier effect that halves compute costs annually.
• 50-75%: Gemini 3.1 Pro pointed to the leap from Llama 2 70B to Llama 3 8B—a near 90% reduction in parameter count for similar intelligence in under a year—as proof that 50-75% is the realistic floor.