Beyond AGI (Artificial General Intelligence)

Tyler Cowen recently posted "Holden Karnofsky emails me on transformative AI" over at Marginal Revolution.

I made two comments. One was similar to my recent post on Rodney Brooks, but much shorter – see this Brooks post as well. I'm posting the other one here as a place-holder. I may or may not elaborate it into a more substantial post at some future time.

Just what IS transformative AI? And AGI, what's that? Or superintelligence? They're highly abstract ideas that can easily be decked out in a grand way.

I've been ...