What happens when your brother is a theoretical physicist helping shape the future of AI safety? In this mind-expanding episode, I sit down with Dr. Paul Riechers—my brother, theoretical physicist, and co-founder of Simplex, one of the leading organizations in AI safety. Paul is at the forefront of one of the most important conversations of our time: how AI is evolving, how quickly it may surpass human intelligence, and what that could mean for the future of humanity.

Does AI have emotions? Desires? Preferences? How does AI compare to the discovery of fire, in terms of the immense impact it will have on humanity? Is AI built or is it grown - and how exactly does it grow? 

These are some of the fascinating, intense and mind-bending questions we address in this episode. 

Furthermore, we dive into the rapid acceleration of artificial intelligence, how AI systems are increasingly helping build and improve their own code, and whether we’ve already entered a self-reinforcing snowball effect of technological growth. We also explore robotics, the realistic timeline for artificial superintelligence, and the most likely scenarios for humanity as intelligence surpasses human capability.

One of the deepest questions we tackle: can intelligence become separated from desire? In other words, could an ultra-intelligent system develop goals, preferences, or values that diverge from those of its creators—and what would that mean for all of us? We also discuss whether current AI systems already show hints of emerging preferences beyond simple human instruction.

This episode is equal parts fascinating, sobering, and awe-inspiring—a conversation about the biggest force shaping our future.

You can learn more about AI safety here: 

Video (A Physicist's Guide to Governing AI) with Dr. Paul Riechers: https://youtu.be/7oJIYrIrsV8?si=yQhZvMYBcgfTLbyA

Simplex Website: https://www.simplexaisafety.com/

Video (a very math-y talk about AI research) https://www.youtube.com/watch?v=-Vz9wdNNxVo