Why This Is Harder Than It Sounds
Generative AI produces text that is fluent, well-organized, and confident regardless of whether it is true. That combination is genuinely difficult for human readers, because we have spent our whole lives using fluency and confidence as rough signals of expertise. Those signals no longer work. This page is about what to use instead.
You already know how to evaluate sources. You check who wrote something, when, and whether they had reason to shade the truth. None of that applies here. There is no author, no publication date, no institutional backing, and no motive to mislead — which is precisely the problem, because there is also no accountability and no mechanism that rewards being right.
A search engine points you at a source you can judge. An AI tool hands you an answer with the sourcing removed. Even when it cites something, the citation may be generated rather than retrieved.
So the question shifts. Instead of “is this source credible,” ask “what would have to be true for this to be right, and can I confirm it independently?”
What to Check, and How
This is the failure mode that has caused real professional and academic consequences, including sanctioned attorneys and retracted student work.
AI tools generate citations that are structurally perfect — plausible author names, a real-sounding journal, a volume and page range, sometimes a DOI — for papers that do not exist. They also produce real citations attached to claims the paper never made.
What to do: open every source. Not search for the title; open the actual item. If you cannot find it in a library database, a publisher site, or Google Scholar, it does not exist, no matter how real it looks. If you do find it, confirm it actually says what the AI claimed.
Faster approach: do not ask AI for sources at all. Use library databases to find literature, then use AI to help you understand or summarize what you have already retrieved. That inverts the workflow into one the tool is actually good at.
Maricopa librarians will help with this and it is genuinely part of their job.
Specific facts are where these tools fail most often and most invisibly. A summary can be broadly accurate while every figure in it is slightly wrong.
Check independently:
- Statistics and percentages
- Dates, especially anything recent
- Names of people, laws, regulations, institutions, or programs
- Technical specifications, dosages, measurements, code that will actually run
- Anything specific to Maricopa — our policies, deadlines, course requirements, department processes
A useful habit: if you would need to be right about it in front of someone who knows the subject, verify it.
These tools learned from text written by people, and they reproduce what was common in that text. That shows up as:
- Default assumptions about who does what job, who is a student, whose perspective is the standard one.
- Uneven depth. Ask about a well-documented topic and you get nuance; ask about an under-documented community, language, or field and you get thin, generic, or subtly wrong material.
- A narrow default voice — formal, American, academic — which quietly treats other registers as deviations.
- Flattening of disagreement. Contested questions often come back as settled ones.
This has direct equity consequences. AI detection tools, for instance, have documented higher false-positive rates for writing by non-native English speakers — a case where uncritical use of an AI system causes measurable harm to identifiable students.
What to do: notice whose perspective is missing, ask explicitly for the counterargument, and be especially careful with output about groups you are not part of.
Models are trained up to a point in time. After that, they know nothing, and they will usually answer anyway rather than tell you.
This matters most for policy, technology, personnel, prices, and current events. It also matters for AI itself — a tool may describe its own capabilities inaccurately, or describe a competitor’s based on a version that is a year out of date.
What to do: for anything time-sensitive, ask the tool what its cutoff is, then treat everything after that as unverified. Better, go to the authoritative source directly.
Push back on an AI tool and it will usually fold, even when it was right. Assert something false confidently and it will often go along. Ask it to support a position and it will find support, whether or not the position is defensible.
This makes AI a poor choice for testing whether you are right about something. It is very good at helping you articulate a view and very bad at telling you the view is wrong.
What to do: ask for the strongest case against your position by name. Ask what a skeptic would say. Ask what evidence would change the answer. And notice when you are feeling validated — that is the moment to be most careful.
Before using AI output for anything that matters:
- What here is checkable? Names, numbers, dates, citations, quotes. Mark them.
- What would I be embarrassed to be wrong about? Check those first.
- What is missing? Whose perspective, which counterargument, what caveat.
- Can I explain how I got this? If you cannot describe your process and why you trust the result, you are not ready to use it.
For low-stakes work this is fast. For anything published, submitted, graded, or acted on, it is the minimum.
For Faculty
This page is also usable as course material. The evaluation routine above works as a classroom exercise: give students AI output on a topic they know well and ask them to find what is wrong with it. Students who have done that once are noticeably more skeptical afterward, and it teaches source evaluation more effectively than telling them to be careful.