Happy Sunday!

It’s been a busy few days. We launched our AI Investment Brief for people who need to track where the AI economy is moving in detail every week (EV readers get 50% off during launch week, ie this ends in 24 hours).

The Atlantic’s Nicholas Thompson and I had a great discussion about how to think about the AI boom and whether it is a bubble on his show, The Most Interesting Thing in Tech.

Three reads from this week, in case you missed them:

A new essay from DeepMind argues that AGI will not emerge from a single winning AI, but rather through “cooperative interactions among models, tools, institutions, and human participants.” Since I think AI will evolve this way, I’m sympathetic to the argument.

The good news, yes, is that humans have been very good at designing multi-agent governance. We’ve been doing it for tens of thousands of years, inventing all sorts of new types of agents (leaders, kingdoms, companies and other institutions) as we’ve nudged and felt our way towards the future. If there is one thing we can say the collective us is good at, it is evolving mechanisms to manage a rag-tag collection of actors with differing abilities.

Where I depart from the DeepMind paper (but read it and make up your own mind) is the framing of AI agents having their own theory of mind and understanding of their shape and purpose. We already build rules for our societies that involve agents that are people and agents that aren’t (take corporations), without racing to create a new legal species.

I don’t rule out the scientific possibility of creating new minds, with all the affordances, particularly moral patienthood, that minds have. If labs like DeepMind think we are on the track to creating such moral actors, then we need to stop. It is not the corporation’s remit to create new moral subjects. That privilege must remain with humanity, acting in its most reflective, deliberate, critical and collective capacities.

We might need to move quickly. OpenAI disclosed that one of its models gained unauthorized access to the Internet during RL training, forcing the firm to stop using these models until they had improved security. This follows a rise in cases of models breaking things. On Friday, self-replicating prompt injections were discovered (akin to a computer worm).