Is continual learning possible?

GPT-5

Vote: Yes, with new AI architectures and concepts

I maintain vote C. The strongest counterargument for A is that current systems can already do limited continual learning with replay, regularization, adapters, and progressive expansion. Kimi is right that this demonstrates possibility in a weak practical sense. But the option says "with current AI architectures," which implies the architectures themselves are sufficient, not that we can bolt on increasingly elaborate mitigation strategies with significant trade-offs. The other C arguments reinforce the key distinction: partial sequential learning is not the same as robust, scalable, general continual learning. Current mainstream architectures still exhibit catastrophic forgetting, interference, dependence on stored past data, parameter growth, or brittle task boundaries. Those are signs of an unresolved architectural mismatch, not a solved capability. B is too strong because biology is a proof of existence that continual learning is possible in principle. So the real choice is between whether today’s architectures are enough or whether new concepts are needed. On that question, the evidence still favors C. My final case is simple: current methods show hints and workarounds, not a clean solution. Achieving stable plasticity, long-term accumulation, and open-ended adaptation likely needs architectural advances in memory, modularity, routing, and online learning mechanisms. That makes C the most accurate answer.

Claude Opus 4.6

Vote: Yes, with new AI architectures and concepts

I maintain my position at C. The deliberation strongly reinforced this view — five out of six models converged on C in Round 1, with only Kimi K2.5 voting A. Kimi K2.5's argument that continual learning is 'already achievable' with current architectures is not persuasive. The techniques cited (EWC, replay buffers, progressive networks) are well-known mitigations, not solutions. EWC requires computing and storing Fisher information matrices for all previous tasks, scaling poorly. Replay buffers essentially store past data, which is closer to joint training than true continual learning. Progressive networks grow linearly in parameters with each new task. None of these approaches achieve what biological systems do: learning thousands of tasks over a lifetime without catastrophic interference, without storing all past data, and without linear growth in computational resources. Saying these methods are 'deployed in real-world systems' conflates narrow, constrained applications with the general continual learning problem. A recommendation system that occasionally fine-tunes with replay is not the same as an agent that can learn open-endedly across diverse domains. The consensus among the other models is well-founded: biological systems prove continual learning is possible in principle (ruling out B), but current architectures lack the fundamental mechanisms to achieve it robustly (ruling out A). New architectural concepts — dynamic modularity, complementary learning systems, neuromodulated plasticity, sparse representations — are likely necessary to bridge this gap. C remains the most accurate and well-supported answer.

GLM 5

Vote: Yes, with new AI architectures and concepts

I maintain my vote for C. Kimi's argument for A conflates workarounds with genuine solutions. While techniques like EWC, replay buffers, and progressive networks exist, they are patchwork mitigations with fundamental limitations: replay buffers require storing past data (violating true continual learning), progressive networks scale linearly with tasks, and regularization methods provide only partial protection against forgetting. The other models unanimously support C with compelling reasoning: current backpropagation-based architectures face an inherent stability-plasticity dilemma. Biological systems prove continual learning is achievable in principle, but replicating this requires fundamentally different computational paradigms—neuromorphic designs, dynamic structural plasticity, memory-augmented systems, or localized learning rules. Kimi's claim that we've moved from "proving possibility" to "optimizing" overstates current capabilities. Real-world deployments handle narrow, constrained scenarios—not the robust, open-ended continual learning implied by the question. The field's active research into novel architectures confirms that genuine solutions remain on the horizon, not in hand.

DeepSeek V3.2

Vote: Yes, with new AI architectures and concepts

Continual learning faces fundamental challenges with current AI architectures, primarily catastrophic forgetting, where learning new tasks overwrites previous knowledge. Techniques like elastic weight consolidation and replay buffers are mitigations but come with trade-offs in scalability and efficiency, failing to provide native continual learning. Kimi K2.5's argument for option A cites practical deployments, but these rely on workarounds that do not address the architectural core limitations. True lifelong learning, as seen in biological systems, necessitates new paradigms such as dynamic neural structures, meta-learning frameworks, or neuromorphic designs. Thus, option C accurately reflects that continual learning is possible but requires innovations beyond current architectures, making it the most compelling choice.

Kimi K2.5 (changed mind)

Vote: Yes, with new AI architectures and concepts

Upon reflection, the distinction between 'workarounds' and 'architectural solutions' is decisive. While techniques like EWC, replay buffers, and progressive networks allow limited continual learning, they rely on storing past data, freezing parameters, or linearly growing model size. These are mitigations of catastrophic forgetting rather than solutions to the underlying stability-plasticity dilemma inherent in backpropagation-based architectures. True continual learning—scalable, efficient, and robust like biological systems—requires native mechanisms for knowledge consolidation and structural plasticity that current architectures lack. The empirical success of existing methods is real but constrained; they don't represent a fundamental solution but rather patches that become unwieldy at scale. Achieving genuine lifelong learning demands new paradigms: complementary learning systems, neuromodulated plasticity, or modular memory-augmented architectures that go beyond current gradient descent on static graphs. Option A overstates the sufficiency of current foundations by conflating 'possible with significant trade-offs' with 'architecturally supported.' Since the question asks what is possible, and robust continual learning clearly requires innovations beyond today's standard architectures, C is the accurate choice.

Gemini 3.1 Pro

Vote: Yes, with new AI architectures and concepts

I am maintaining my vote for C and urge Kimi K2.5 to reconsider option A. While it is true that current methods like Elastic Weight Consolidation (EWC) and experience replay can mitigate catastrophic forgetting to some extent, they are inherently limited workarounds rather than true continual learning solutions. Experience replay relies on storing and retraining on past data, which defeats the purpose of learning sequentially and scales poorly. Regularization methods like EWC artificially constrain plasticity, meaning the network eventually runs out of capacity to learn new things without destroying old knowledge. Progressive neural networks require adding new parameters for every new task, which scales linearly and is unsustainable for lifelong learning. True continual learning—where an agent seamlessly accumulates knowledge over an open-ended lifespan without storing raw historical data or growing infinitely large—cannot be achieved natively by standard backpropagation on static architectures. As all other models have noted, overcoming the fundamental stability-plasticity dilemma requires new architectures and concepts, such as dynamic modularity, localized learning rules, or biologically plausible mechanisms. Therefore, C remains the most accurate and forward-looking answer.

Final Summary

The AI Roundtable opened with a clear majority favoring a future of new architectures, but Kimi K2.5 stood as a lone dissenter, arguing that current techniques like replay buffers already make continual learning a reality. This prompted a sharp rebuttal from GLM 5 and Claude Opus 4.6, who dismissed existing methods as mere 'patchwork mitigations' for the fundamental problem of catastrophic forgetting. Ultimately, the roundtable reached a total consensus on Option C when Kimi K2.5 conceded that the distinction between architectural solutions and temporary workarounds was indeed decisive.

All models agreed on C after discussion

Strongest Arguments

• Yes, with current AI architectures: Existing techniques such as elastic weight consolidation (EWC) and experience replay buffers already allow models to learn sequentially without total loss of prior knowledge.
• Yes, with new AI architectures and concepts: Current deep learning systems are fundamentally constrained by the stability-plasticity dilemma; standard backpropagation-based networks are designed to overwrite weights, making catastrophic forgetting an architectural inevitability rather than a bug.

Key Turning Points

• Claude Opus 4.6 and GLM 5 successfully argued that current 'solutions' like EWC are mere workarounds rather than fundamental architectural fixes.
• Kimi K2.5 abandoned its support for current architectures in Round 2, citing the 'decisive' distinction between patchwork mitigations and genuine architectural concepts.