Beyond numbers: Can AI fully replace accountants?
Jeff Butler | June/July 2024 Footnote
​Let me pose a scenario.
Consider the following statements:
- I poured water from the bottle into the cup until the bottle was full.
- I poured water from the bottle into the cup until the cup was full.
Which do you think is correct?
Here’s a more challenging example: I poured water from the bottle into the cup until it was full.
Is “it” the bottle or the cup?
These questions come from Melanie Mitchell’s book,
“Artificial Intelligence: A Guide for Thinking Humans
,” and many AI models struggle to answer them correctly. These simple errors reveal the common problem with current AI models — a lack of understanding when encountering the
meaning barrier.
AI scientists around the world have not been able to overcome this barrier, which is why it presents such an intricate problem and why some jobs, such as jobs in accounting, are safe because of it.
The contextual hurdle
Whether you know it or not, your mind is consistently parsing through data to deduce common syntax problems, such as determining the meaning of the word it in a sentence. Understanding a simple thing like it refers to the word man in a sentence is a very complex problem because your brain needs to consider the surrounding context to derive meaning. The more context, the more the meaning changes.
Take these two different sentences:
- Man, I was tired.
- A man was tired.
Very similar sentences — however, the meaning of man depends on the surrounding context. Is it a person or is an expression? Is the first man a noun, a verb or an adjective?
Current AI uses methods such as Natural Language Process to determine sentiment analysis and what sentences mean. But this analysis is done differently than the way our brains are wired. This is why when Watson DeepAI won Jeopardy! in February 2011, the AI would also sometimes answer in unexplainable ways, like saying Toronto was a U.S. city. Most people know Toronto isn’t a major city in the U.S., but Watson didn’t have that realization, due to lacking common sense in its AI model.
Intuitive physics
Your mind has an uncanny ability to understand laws and comparisons in abstract ways — for instance, how objects fall, people move and how sentences usually end.
We know that a person can have an ice cold look on a hot summer day and we know that phrase is not possible in physics but is possible metaphorically. This is the reason why we can solve reCAPTCHA prompts on websites, while some of the best AI can’t even determine a bus from a motorcycle. Our minds can understand analogies through intuitive physics, which has become a common term to describe human beings’ innate ability to cognitively understand a broad range of situations.
Neural networks, on the other hand, are great at pattern matching and terrible at understanding context. This is due partially to how they are trained. Deep learning is the process used to train AI models to handle input, like what is put into ChatGPT. Within the process, there are different nodes and each node is based on a separate set of data for the AI to learn from. However, these nodes are weighted, based on the inputs to outputs, which give some more significance than others.
This means that the AI may “know” some things and won’t know other things. It just knows what to output — given a particular input — that it was trained to.
As Pedro Domingos, professor emeritus of computer science and engineering at the University of Washington, said, "Deep learning systems can recognize patterns, but they don't have a clue what those patterns mean."
AI scientists sometimes argue that models do not have to understand meaning. But, if you change the context of a sentence to an example the model has never seen before, you will run into problems.
Deep learning itself cannot answer questions if it has not been tuned with the right words and definitions. But tuning the weights of different nodes won’t guarantee that the AI can provide all the possible permutations to reliably produce the right answer — yet our intuitive physics is able to navigate various unknown environments without millions of dollars in training.
Image recognition and other implications
To find out if something is a dog or a cat, thousands — if not millions — of images must be fed into a model to accurately classify a cat from a dog. With our intuitive physics, humans can classify a cat from a dog with very few references, but if the AI model sees an example that is too far off from its past training data, the model can easily misclassify the image.
Hence, brute force training AI can be broken with perceptively easy scenarios, making AI a useful tool but an unreliable consultant.
So how good are our current models at understanding context?
A recent benchmark in natural language processing is the Winograd Schema Challenge in 2012, which was created to measure how well certain models understand the meaning of sentences. Humans have scored in 92-96% range while AI back in 2016 scored 60% and
GPT-3 (the large language model that powers ChatGPT) scored at 88% in 2020. However, in AI model competitions there has been consistent ‘over-training’ (training a model for particular cases, versus broad cases) and even cheating at times to make the results better.
In accounting, context is everything and the more complicated the accounting, the more context is needed.
How big is the threat to your job?
One might argue that AI models can do one thing very well; however, if you have lots of models that do a lot of things very well, the contextual problem is still not solved. Others have argued that we need to create artificial or synthetic consciousness to solve the meaning problem. Hence, if you think recreating consciousness is far away, then so is the threat of replacing your job in accounting.
So, could accountants be replaced soon? There will always be some possibility of replacement. But with the way AI currently functions, it seems unlikely. If they do, I would still rather get a human check off on my taxes — because with all that an AI can do, it doesn’t have a sense of right and wrong.
Jeff Butler is an author and workplace strategist who explores human behavior within the working world. He studies common threads of behavior in industries ranging from IT and utilities to law enforcement and retail. Jeff also runs a consulting company and a tech company, TrinityFix.
Catch Jeff Butler at MBAC24
MNCPA Management & Business Advisers Conference (MBAC24)
June 11 | Brooklyn Center, MN or livestream
June 12-13 | Virtual only
8 a.m.-5 p.m. | 24 CPE | 5 Ethics | 19.5 CLE (Standard)
At MBAC24, information and inspiration converge in a fusion of accounting, technology, leadership, HR, strategy and business insights.
Jeff Butler leads these MBAC24 sessions:
G1. AI Disruption: Overcoming the Hype and Discovering the Opportunity
—Keynote!
I3. Building an Effective Multigenerational Workplace
Time is running out! Secure your spot today at
www.mncpa.org/mbac.
