I don’t think alignment is a solved problem for people/organizations.

It should be at least as hard as the principal agent problem?

Eh, I think AI researchers already think alignment is as "solved" as they think it should be, no?

@YonatanCale maybe some of the really Pollyanna-ish ones like LeCun, I guess? Even Altman says alignment work remains critical, and half of AI capabilities research estimate a P(doom) of 10% or higher.

@connorwilliams97
Wait,
If the only things left to do in "alignment" are things like "get cars to be more reliable", then this is a capabilities problem in my terminology, no?
Sam speaks about alignment, but as things needed to get better capabilities. He says they go hand in hand. (no?)

Do you think Sam personally has a P(doom) of around 10%+ ? I doubt that, I don't hear him worrying about AI killing everyone, he just uses words like "alignment" and "safety" to mean something else. (no?)

@YonatanCale he literally said AI could be "lights-out for all of us'"

The majority of AI alignment researchers? Or all AI researchers?

@IsaacKing The majority.

@Duncan I think Isaac meant: The majority of AI alignment researchers? Or the majority of all AI researchers?

@Duncan Steven's interpretation is correct. Apologies for the ambiguity.

@IsaacKing "AI alignment researchers", or whatever the equivalent term at the time is.

can you clarify what you mean by AI take off? we are already deep in AI take off now, so this seems like an odd thing to condition on

@L An AI that appears to have more control over the world than do humans.

@Duncan Ah, so this is a grammatical confusion between "AI takeoff in progress" and "AI takeoff completed successfully." I also was confused. I feel like we are currently in "boulder rolling down hill" condition, but have not yet reached "boulder all the way at bottom of hill" condition.