Will Superintelligent AI Understand Morality?
When a Single Good Breaks the World
Japanese-language book (超知能AIは道徳を理解するか？). Paperback (A5, 194 pages) / Kindle edition. Self-published via KDP.
Is a fish that bites a lure stupid? No. In murky water, the fish that strikes on reflex survives better than the one that examines its food before biting. A rule that is fast, rough, and occasionally wrong is not a defect but an adaptation. This book begins by describing human morality as such a “compressed danger-assessment table,” and from there turns to a single question: what happens if a superintelligent AI fully understands one particular good and implements it across the world?
What this book fears is not a failed AI, but a successful one.
— from the Introduction
What sets this book apart
It uses no technical jargon. With nothing but concrete examples — fish, the dodo, a child in a concert hall — it climbs from the mechanics of morality all the way to ASI alignment.
It describes how morality and discrimination arise from the same cognitive mechanism — neither defending nor condemning it, but describing it as a mechanism.
It presents a minimal framework that lets multiple “rightnesses” coexist physically — formal autonomy.
Table of contents
For readers of “If Anyone Builds It, Everyone Dies”
“If Anyone Builds It, Everyone Dies” (Eliezer Yudkowsky & Nate Soares; Japanese edition by Hayakawa Shobo) argued the dangers of an AI that fails at alignment. This book takes up the quieter question beside it: what can an AI that succeeds at alignment destroy?
About the author
Neither an expert nor a researcher. A writer who, while having AI help with the writing, ended up marking up the AI’s own morality in red again and again.