Need for Speed

In previous posts I argued that we will soon have a lot of AI workers. Here we look at how fast each of them will be. There is a big difference between a million AI workers plodding along at human speed and "only" ten thousand that each work a hundred times faster. In short, today's AIs perform the tasks they can do about ten times faster than humans. At the point they can do human work in general, I expect individual copies would work at closer to human speed, though they would still finish long tasks considerably faster by working around the clock and splitting work across copies. Suitable hardware could raise their raw speed to perhaps a thousand times that of humans. How fast do AIs run today? Language models produce text by sequentially generating tokens, each corresponding to about three-quarters of a word. Here is how fast providers read and write tokens for a single user, as measured by Artificial Analysis in October 2026: Most models write around 50 to 300 tokens per second whatever their size and read inputs fifty to a hundred times faster, as estimated from the time to the first output token. Reading is much faster than writing because models can process all of their input tokens in parallel, but must generate output tokens one after another. We discuss how tokens are generated in more detail in Appendix A. Token speed is the most straightforward measure of how fast a language model runs, but it says little about how much work the model gets done. Small models lik…