3 Common Ways AI Gets It Wrong: Why Verification Matters
A few weeks ago I told you that AI lies to you.
At MyTeacherAide, we’ve put rigorous guardrails around our AI to eliminate curriculum hallucination, but we are one of the only AI Teacher tools doing this.
ChatGPT doesn’t have those guardrails. Neither do most AI Teaching tools.
They often lie to you.
That’s why you always need to verify the output form AI with your OI - organic intelligence!
But, not all AI is wrong in the same way.
In my extensive use of AI, I’ve noticed 3 ways that AI get’s it wrong.
Let me show you what I’ve found:
Obviously wrong
Recently, Google’s “AI Overviews” sparked controversy when they gave some puzzling advice.
One AI Overview suggested that non-toxic glue would be a fantastic way to make cheese stick to pizza!
Another stated that geologists recommend eating one small rock per day for health reasons:
“Dr. Joseph Granger recommends eating a single serving of pebbles, geodes, or gravel with each meal, or hiding small rocks in foods like ice cream or peanut butter.”
One of the things that makes these examples so comical is that they are so obviously wrong.
2. Not obviously wrong
But sometimes the errors are more subtle.
In February of 2023, Google’s AI chatbot Bard did its first demo.
The initial prompt was, “What new discoveries from the James Webb Space Telescope can I tell my 9-year-old about?”
One of the responses was that the James Webb Space Telescope “took the very first pictures of a planet outside of our own solar system”.
In reality, the first photo of this kind was taken 16 years prior to the JWST launching.
This error cost Google dearly, with their stock price falling roughly $100 billion on the next day of trading.
3. Right-but-also-wrong
In May I spoke with a teacher who used AI to create a multiple choice quiz.
All the answers were right; there were no factual errors.
The problem?
The answer to every question was ‘D’.
That’s a problem.
The test was factually right but structurally wrong. It was right-but-also-wrong!
The message is clear: Always verify the output of artificial intelligence.
If we use AI-generated content in our classrooms, we must be able to guarantee its accuracy. To do this, we must verify the output.
Always audit artificial intelligence with your organic intelligence!
Happy teaching,
Paul Matthews, Co-Founder & CEO of MyTeacherAide
