We get to know the large language models (LLM) through one of its example ChatGPT’s astonishing success to interact with us like a human and solve problems like a machine:smile:.

I think the language for us humans is a way to express ourselves by using priorly agreed upon sounds and expressions. The “agreeing” part is important because while speaking/expressing ourself, we generally do it in a way we deem the most appropriate (or acceptable). That means we choose the phrase or word that are mostly used: we have a commonality in our speaking and expressions.

LLMs work because they are modeled based on finding the most common/appropriate expressions when responding. That is why I think the title of the paper “attention all you need” explains a lot about the model and “stochastic parrot” is also appropriate term for the part so far mentioned.

The “surprising” part in ChatGPT and other models is that this most common(appropriate) answer seems to also work in math and scientific problems. However, if we have enough correct samples, since the most common thing is the correct one, it is not so surprising that would work. That means we would also expect mistakes we commonly make to be seen in these models unless they are corrected in their descriptions/training. It may be also possible to apply one correct commonality to another to get a better outcome. That may explain if we can find new discoveries in science.

On the other hand more surprisingly, these models teach us that being smart in life often means choosing the most accepted/common way among the seen possibilities. In a more concise way we can comfortably say that “history repeats itself” and “history is a great teacher” for us humans. Maybe being not stupid also means not doing the most unexpected in most situations and not repeating the same mistakes and expecting a different outcome.

I think there are many lessons we can learn by emulating an interacting-human (virtual human) and of course should not forget(or behave non-existent) how these tools can be used for harm.