Thoughts on AI Safety

Epistemic status: personal stance, held loosely, subject to change.

As of today, I’m a defensive accelerationist with transhumanist goals. I want advanced AI for what it could do for humanity as well as QALY maximization in our lifetimes.

My P(catastrophe) within our next decade is approximately 10%. For example infrastructure collapse leading to famine, or temporary collapse of foundational systems such as payments, or networking (events which may lead to >1M deaths). My P(doom) is <1%, as in total extinction. My main concern in AI safety is the timing gap (IRL race condition)… Once AI can automate research at a superhuman level, capabilities may increase faster than any organization can defend. Labs have previously shipped poorly aligned models to the public (Sydney, 4o, Hitler Grok).

I see potential misuse as a symptom of misalignment, and frontier models should be aligned with humanity as a whole, not for the individual. Open models can fill this gap in self-run, self-maintained instances, aligned to the user.

I do not support slowdown. If I did, I doubt the levers to do so are efficacious. Export controls or pauses are loose guidelines, which can be bypassed, and KYC gating access to frontier intelligence goes against my beliefs of freedom and privacy. Additionally, at my estimates, the cost of delaying AI’s benefits outweighs the reduction in catastrophe risk

I favor infrastructure resilience, formally verified software, and the continued progress of capabilities and alignment research.