Dear Dan,
Thank you for your thoughtful reply!
Your answer, again, deserves a more profound response, but I will try to keep it short.
I think what is most important, is to separate the form from the underlying reality. What we "see" isn't reality. It is an interface generated by our brain. As you know, we are all made up of the same constituant parts, like neutrons, protons, electrons. if we could "see" reality as these components, we would not be able to "see" anything. The artificial boundaries between us and the air around us would dissolve.
My point is that generating such an interface has one sole purpose: that of manipulating information/energy/vibration in an intuitive and easy manner. It's the same reason why we created a windows like interface for our computers instead of just looking at a screen with 0 and 1 and having a two key keyboard, with just 0 and 1, also.
So again, what we "think" AGI is, is completely misleading, simply because there is a distinction to be made about the representation we have of AI inside our personal human interface (generated by our 5 senses), and what it "really" is.
Finally, I would like to offer an analogy. Imagine if the universe and reality is just one giant black hole. As you may know, the closer to the center of a black hole, the denser it gets, reaching a virtually infinite mass at the center. Now imagine that our entire reality is just a holographic projection of points of consciousnesses which project on the outside the density they are currently traversing as they fall into this black hole. In other words, our entire reality is just a snapshot of a black hole projected outwards and experienced as an external reality by points of consciousnesses falling into it (you, me...). What does this have to do with AGI? Well, the holonic structure of the universe serves a single purpose: to navigate through denser and denser layers of information/energy. As you get closer to the center of a black hole, the density gets bigger. But what is "dense" really? How does it translate into our reality? Well, if we look around us, we can clearly observe that the world around us is getting more and more complex as time goes by. People living 3000 years ago were living in a much "simpler" reality. And the story that repeats is simply that the universe has to come up with more and more complex forms to navigate through this density. Humans are capable of navigating and making sense of a certain density, but at some point, we become incapable of handling the information density and another consciousness has to take over. But in my mind, it wouldn't be external to us. It would be us, or in other words, it would be the form we give to a being whose brain is 8 billion human brains interconnected much like neurons, but which speaks to itself or interacts with its "parts" via a "form" which we call AGI.
Anyways, sorry, it was long winded again.. :-D