Chatbots like ChatGPT by Open AI can write poetry, summarize books, and answer questions, often with human-like fluency. The curious thing is that, while they are complex and advanced models, and can perform mathematical calculations based on what they have learned, the results can vary and are often incorrect. How? Let’s see.
Chatbots are designed to determine probabilities, not to make rule-based calculations. Probability is not accuracy, and language is more flexible and forgiving than mathematics.
‘Artificial intelligence chatbots struggle with math because they were never designed for it,’ says Kristian Hammond, a computer science professor and artificial intelligence researcher at Northwestern University in an interview for the New York Times.
As the American newspaper says, it seems that the smartest computer scientists in the world have created an artificial intelligence that is more of a poet than a mathematician.
AI is a poet and comedian, but it can’t add and divide accurately
At first glance, this represents a break from the past of computing. Since the first computers appeared in the 1940s, a good summarized definition of computing has been ‘mathematics on steroids.’
Computers have been tireless, fast, and accurate calculating machines. For a long time, computers have been exceptional at crunching numbers, far surpassing human performance.
Traditionally, computers were programmed to follow step-by-step rules and retrieve information from structured databases. They were powerful but fragile. That’s why previous efforts in the field of artificial intelligence hit a wall.
However, more than a decade ago, a different approach emerged and began to produce surprising advances. The underlying technology, called neural network, is based on the human brain.
This type of artificial intelligence is not programmed with rigid rules, but rather learns by analyzing large amounts of data. It generates language based on all the information it has absorbed, predicting which word or phrase is most likely to come next, just like humans do.
Sometimes, artificial intelligence chatbots have struggled with simple arithmetic and mathematical problems that require multiple steps to reach a solution, something recently documented by some technology reviewers.
According to OpenAI, their new version of ChatGPT achieved an accuracy of nearly 64% on a public database of thousands of problems that required visual perception and mathematical reasoning. This figure is a slight improvement over the 58% of the previous version.
It seems like a life lesson for students, worth remembering long after they have forgotten the Pythagorean theorem: don’t believe everything an artificial intelligence program tells you. Don’t trust it too much.
