TheSequence Opinion #699: 2030 or Bust? The Compute Surge and the Bottlenecks Ahead | By The Digital Insider

Can we achieve AGI by following the scaling laws or would we need new algorithms?

Created Using GPT-4o

Artificial General Intelligence (AGI) – machines with human-like cognitive abilities across diverse tasks – has long been a goal in AI research. The path to achieve AGI remains highly debatable. Would it be as a byproduct of the scaling laws or would we need algorithmic breakthroughs? This essay explores a controversial idea somewhere in the middle. By following the scaling laws, we might have a chance to achieve AGI by 2030. However, after that, we are likely to hit some computer bottlenecks that will force us to work on new algorithmic improvements^.

In recent years, the primary driver of AI breakthroughs has been the scaling of computing power and model size. Advanced neural networks like GPT-4 owe much of their capability to huge numbers of parameters and the massive computational resources used to train them. Many experts suggest that if this exponential growth in compute continues at its current rate, we might achieve AGI by around 2030. However, serious concerns are emerging about how far pure scaling can go. Energy consumption, financial cost, and physical limits in hardware suggest that simply throwing more compute at the problem will encounter diminishing returns in the 2030s. This essay examines the evidence on both sides – whether scaling alone can deliver AGI – and explores why architectural innovations and algorithmic breakthroughs will likely be needed to sustain progress once current scaling trends reach their limits.

Compute Growth Trends and Projections

#AGI, #Ai, #AIResearch, #Algorithms, #CognitiveAbilities, #Computer, #Computing, #Energy, #EnergyConsumption, #Financial, #GPT, #GPT4, #Growth, #Hardware, #How, #Human, #Innovations, #Intelligence, #It, #Model, #Networks, #Neural, #NeuralNetworks, #OPINION, #Power, #Research, #Resources, #Scaling, #ScalingLaws, #Trends, #Us, #Work

---

Published on The Digital Insider at https://is.gd/k5QCIj.