2,500 year ago, Confucius famously said, "Know what you know and know what you don't know, this is knowing!"

Now, imagine today’s AIs being asked this simple question: "Tell me something you don't know!". Or even better, don't imagine it, try it! On Alexa, Siri, Google Home or whichever "intelligent" assistant you have access to. I did, and here is the best answer I got among all of them:

Q: Tell me something you don't know.
SIRI: Oh, no!
Q: Is there anything you don't know?
SIRI: Yes, there are some things I don't know. I hope you don't hold it against me.
Q: Then tell me about it.
SIRI: I'm sorry, I will try harder.

Nice try, SIRI, but not hard enough.

It’s obvious that none of them could even understand the question. In contrast, you and I can easily answer this request at any time with numerous sensible answers like, "I don't know what intelligence really is." or "I don't know how to play chess."

Is it simply a matter of AI trying harder, with more data and computing power?  Or maybe somehow all the AI experts just missed this "minor detail"?  Let's see, minimally, what it takes to answer this request sensibly:

(1) The AI must know what it does not know.
(2) The AI must be able to structure and express what it does not know.
(3) The AI must know itself.

Hmm? Doesn’t look that simple, does it? Maybe Confucius got it right after all, way before us - If you don’t know that you know something, do you really know it? If you don’t know what you don’t know, where do you even start to ask any question? If you can’t ask questions, how do you learn anything new? If you don’t know yourself, can you know anything else at all?

Before we indulge ourselves with superintelligence and the like, maybe we should first build an AI that can at least answer this very "simple"question!