AGI·LEDGER
The Ledger
@Roman Yampolskiy · 2026-06-27Halt

Superintelligence is uncontrollable, so we should not build it

What form?
Other
How built?
Other
Made of?
Doesn't matter
What for?
Don't build
Who controls?
Should be halted
When?
Unstated
P(catastrophe)
99%

My take

Controlling a superintelligence is likely impossible in principle. Until someone proves otherwise, building one is reckless to the point of near-certain catastrophe.

Why

No known method guarantees indefinite control of an agent far smarter than its creators, and verifying that such a system is safe may itself be undecidable. Partial control is not enough when the downside is permanent.

What would change my mind

A formal proof, or robust demonstration, that an arbitrarily capable AI can be permanently and reliably controlled.

Summarized from public statements · source ↗
#sourced#control#halt