L(ife|LM) imitates math

From Intelligence to Imitation: The Semantic Drift of AI and the Limits of LLMs

Semantic drift is fascinating. I find myself amused observing how the term Artificial Intelligence (AI) has been downgraded to mean just about anything involving clever math. It used to be that AI meant what we would now call AGI (at least until the hype cycles downgrade AGI to mean “a bit smarter LLM.”) This is also an example of Wittgenstein’s “meaning is use” - if we use the above-mentioned clever math as if it is AI, we’d just as well might call it such.

LLMs are the most excellent example of the great philosopher’s ideas on how language and the ability to think are linked (the limits of my language are the limits of my world). LLMs are not “embodied” in a world at all - they don’t have any “senses” in the way we do. They do not know anything but language. Which is perfectly fine as they were originally developed for text translation - except now we expect them to “think.”

Well, what would they think about, outside language?

(Yes, there are ongoing efforts to augment LLMs with tools and function calls that are changing this, but that’s till far away from actual embodiment.)

We’d might as well call LLMs “language generators.”

People are commonly asking “why can’t LLMs do math?” (well, most people don’t use the term LLM, they’re mostly saying “ChatGPT sucks bacause its bad math failed when I tried cheating on a test with it”). It’s because to an LLM, a number has no special significance outside the language structure it’s used in. And I don’t think it surprises anyone that, up to a point, it’s true for us humans too. It takes concentrated, sometimes painful effort, to do complex logical operations (Daniel Kahneman called it “System two”), but anyone can form and understand the sentence structure involving complex terms outside of their understanding.

The next question is - if that’s true, why do LLMs appear to sometimes produce good answer to math questions. I think the answer is two-fold. Firstly, their immense training data usually contains enough common, low-level math or logical problems alongside their solutions that the language generators will just regurgitate the solution, like a hugely expensive lookup table. Secondly, the human ability to estimate numbers (the Approximate number system theory) seems to me to be very similar to the LLMs ability to summarize text. So in essence, LLMs are not “solving” math problem logically, they are “summarizing” the problem into the solution, though syntactic rules that they’ve sort-of ad-hoc learned during training.

tl;dr:

The term Artificial Intelligence has gradually shifted from its original meaning to refer to any complex mathematical process, exemplifying Wittgenstein’s idea that meaning is shaped by use; LLMs, while powerful language generators, lack true embodiment and logical reasoning, instead "summarizing" math problems rather than solving them logically. ← and you know it’s true because it’s ChatGPT’s summarization of the above text ;)