With a number of apparent AI breakthroughs
in recent weeks, it’s a good time to revisit AI safety (see previous coverage
) - efforts to ensure that AI progress benefits humans and avoids catastrophic consequences. Nick Bostrom’s 2014 book Superintelligence
- which warned of superintelligent artificial agents with wildly different values from us - has played a large role in framing the debate so far.
This week’s ChinAI podcast
sees Jeff Ding interview Eric Drexler
, one of the fathers of nanotechnology, on his paper from earlier this year, Reframing Superintelligence
. Drexler argues that the Bostrom-ian framing of superintelligent agents doesn’t fit what we know about AI progress so far. Drexler says a more likely outcome is “Comprehensive AI Services” (CAIS) - i.e. a collection of superintelligent tools
, rather than agents
. Drexler is optimistic that this will give us more options for solving the “control problem
”, as the tools won’t have goals and values of their own.
Scott Alexander at Slate Star Codex is broadly positive
on Drexler (and makes good metapoints about why he didn’t think of this before) Rohin Shah has a great synopsis
, but thinks CAIS will eventually be overtaken by AI agents. Gwern’s critique
- that “tool AIs want to be agent AIs” - is perhaps the best, despite predating Drexler by two years. Nevertheless, it’s valuable to have a framing of AI risk that’s harder to parody than a paperclip maximiser
- which hopefully means more people will (rightly) take it seriously.