Jeff Dean & Noam Shazeer - 25 years at Google: from PageRank to AGI — Dwarkesh Podcast Summary

Executive Summary

Dean and Shazeer connect AI progress with hardware and algorithm co-design. Arithmetic has become relatively cheap while moving data is costly, which helps explain the emphasis on matrix multiplication. They argue that improvements come from architecture, algorithms and data as well as larger hardware budgets. The interview also describes how research ideas combine and why humility and a record of ambitious directions matter. Their forward-looking views are from February 2025.

Key Takeaways

General-purpose hardware no longer supplies the same effortless gains

Dean contrasts an earlier period of waiting for faster hardware with slower improvement in general-purpose CPUs. Specialized accelerators fill some of that gap. The discussion explains a change in engineering incentives rather than guaranteeing a fixed improvement rate for every chip or workload.

Data movement can dominate the design

Shazeer argues that arithmetic is cheap relative to moving data and that algorithms have adapted to those hardware economics. Matrix multiplication offers a way to do substantial computation around reused data. The lesson is to consider movement and memory costs, not only count arithmetic operations.

Hardware and algorithms should be designed together

Shazeer discusses working across algorithm and hardware teams, including decisions about numerical precision. A design that works mathematically can still be inefficient on a particular machine. Co-design therefore asks what the workload needs and what the hardware makes cheap rather than optimizing either side in isolation.

Breakthroughs often combine ideas already being discussed

Dean says many ideas are partly in the air and that a new advance may combine existing work with an additional element. His account values awareness of neighboring research. It does not reduce invention to copying, but shows why understanding the current toolkit can make a new combination possible.

Algorithmic improvement matters alongside scaling

Dean attributes model-generation gains to architecture, algorithms and data as well as hardware and size. He also considers automated exploration of more ideas. These are possible routes for research, not proof that automatic architecture search will produce a breakthrough at a particular cadence.

Humility can make research more effective

Shazeer describes being willing to drop an idea when someone finds a better one. The point is to make progress rather than defend ownership of a proposal. A research culture can reward changing direction on evidence without treating every discarded idea as a personal failure.

Keep ambitious directions visible

Dean describes an internal slide deck for unusual ideas and possible directions. Recording them makes it easier to share, revisit and test them. The practice supports a search across possibilities; it does not mean that every recorded idea is equally feasible or deserves immediate resources.

The Conversation Breakdown

Hardware shapes algorithms

⏱ 6:20

The guests explain why slower CPU improvement and the cost of moving data changed the engineering landscape. They connect those constraints with specialized accelerators and the algorithms that run well on them. The section grounds co-design in actual workload costs rather than only headline chip performance.

Core Frameworks & Ideas

Hardware and algorithm co-design

Choose computational methods and hardware capabilities together, including memory movement and precision.

Ideas in the air

Combine existing research tools with a new element while keeping track of work in neighboring areas.

Scale plus algorithms

Separate gains from more hardware from gains in architecture, data and learning methods.

Actionable Next Steps

✓Measure data movement and memory costs as well as arithmetic in a technical system.
✓Discuss workload requirements with hardware and software teams together.
✓Keep a list of research directions that can be revisited and tested.
✓Change course when evidence supports a better idea rather than defending the original proposal.