To get the benefits of advanced AI while avoiding loss of control, we should build powerful but non-agentic systems — a "Scientist AI" that models and explains the world and answers questions, without goals or agency of its own.
Much of the danger comes from agency, self-preservation, and goal-directed autonomy. A system that only understands and explains, without acting to achieve its own objectives, removes the most dangerous failure modes while keeping the scientific upside.
A proof that fully agentic AI can be made provably safe, which would make the non-agentic restriction unnecessary.