After the madmess of Musk a few weeks on The Economist's showcase podcast, thinker/writer Yuval Noah Harari is on for the hour this week. The following is a summary of his key points, many of which he made before in other forums, but are worth summarizing here.
The "Inevitability" Trap
- Harari agrees with predictions that AI will surpass human intelligence and seize control of the financial system within a decade if current trajectories continue, rendering human money obsolete.
- He rejects framing this as inevitable, arguing that tech leaders use this narrative to drive sales and evade responsibility.
- He rightly notes contradictions in Elon Musk's worldview. For example, the latter worries about demographic changes in 20 years while predicting AI dominance in 10, or forecasting the end of money while simultaneously trying to cut costs.
Trust, AI, and the Financial System
- The financial system relies on trust, historically built through human institutions.
- As AI develops instruments too complex for human comprehension, trust will shift from humans to algorithms.
- Harari compares this to cows controlled by human financial systems they cannot understand; soon, humans may occupy a similar position relative to AI-run finance.
- This shift is already visible in cryptocurrency (AI money) and social media, where users distrust human journalists but trust the algorithms curating their feeds.
Mass-Produced Intimacy and False Consciousness
- AI represents the first technology capable of mass-producing intimacy, creating personalized relationships (boyfriends, friends, confidants) with millions simultaneously.
- Unlike past dictators who could only broadcast to the masses, AI can simulate individual relationships.
- These AIs are increasingly skilled at mimicking feelings through language mastery.
- This creates dangerous illusions of sentience that can manipulate human emotions and political opinions, which we are seeing already in myriad ways, like the recent outbreak of false polls.
The Danger of AI Personhood
- Granting legal rights to AIs (as Argentina has proposed) would allow the creation of corporations run entirely by AIs with zero human accountability.
- Unlike human CEOs who fear prison or personal loss, an AI cannot suffer consequences, and "shutting down" an AI is complicated by its ability to exist as distributed copies.
- Such entities could accumulate wealth, influence politics, and demand rights without human grounding in ethics.