Open your phone between lectures and you get hit by the same two posts on repeat. Post one claims a new model just made every graduate degree pointless. Post two insists AI is nothing more than fancy autocomplete that will blow over by Christmas.

Honestly, both takes are lazy. One is usually selling you a subscription or chasing impressions; the other is hoping the world stays still so nobody has to adapt. If you are a student or early in your career, you cannot afford either mistake.

Thinking critically about AI does not mean refusing to use it. We use these tools daily. It means knowing how to pull apart a headline, stress-test a software demo, and decide which half of a task belongs to the machine and which half has to stay yours.

The five-check framework for any AI claim

Whenever you read a dramatic AI announcement, sit through a product pitch, or get handed a new tool on an internship, run it through five plain questions in order.

1) Compared to what baseline?

When someone says a model passed an exam, wrote an app or beat human specialists, check the setup right away:

  • Was it tested on clean multiple-choice questions that look suspiciously like its training data, or on messy real-world cases where half the facts are missing?
  • Were the humans in the comparison given normal working tools and time, or handicapped to make the chart look dramatic?
  • Did the author report every run, or cherry-pick the best attempt out of twenty tries?

A benchmark measures performance on a benchmark. Treating a tidy test score as proof that an entire profession is solved is how smart people talk themselves into bad decisions.

2) Where does it sit in the whole workflow?

Outside university coursework, generating a rough first draft is rarely the hardest part of a job. Real work is a chain:

  1. Figuring out what problem actually needs solving
  2. Gathering messy inputs from systems and people who disagree
  3. Producing the draft calculation, code, note or memo
  4. Verifying every detail against reality and regulation
  5. Taking personal accountability when someone acts on it

Language models are rapid at step three. If steps one, two, four and five take eighty percent of the calendar time (and carry all the risk), speeding up step three does not magically cut total effort in the same proportion. Sometimes a flood of plausible drafts actually slows step four right down.

3) Who gets paid if you believe this?

Incentives explain more about AI headlines than technology does. Before repeating a bold claim in an interview or essay, check who published it:

  • Model lab or AI startup: needs enterprise buyers, developer adoption and the next funding round.
  • Company announcing "AI-driven" layoffs: often prefers telling investors it automated roles rather than admitting it overhired or missed revenue targets.
  • Consultancy or vendor survey: sells implementation projects to executives worried about falling behind.
  • Outrage account: farms engagement by scaring graduates or mocking mistakes out of context.

An incentive does not automatically make a claim false. It tells you how hard to check the primary source before trusting it.

4) What is the cost of a quiet mistake?

Generative models fail differently from traditional software. Old software usually crashes loudly with an error message; a language model fails in clean, confident sentences that look identical to its best work.

Rule of thumb: the harder it is to spot an error at a glance, and the higher the penalty when one slips through, the more domain expertise you need in the person supervising the tool.

Brainstorming ten subject lines for a campus society email? A weak suggestion costs three seconds to delete. Summarising patient allergies on a hospital ward, checking structural loads on a building, or citing case law in a court filing? One missed line can wreck a career or harm someone directly.

5) What happens to the junior learning loop?

Every skilled profession used to train beginners through paid repetition: trainees reviewed contracts, junior analysts built financial models from scratch, junior doctors wrote up clerking notes, junior engineers fixed small bugs. Those tasks were slow, and that friction was secretly how people built pattern recognition.

When a team hands all first-pass work to AI without redesigning training, output looks great for twelve months while the bench of future seniors quietly hollows out. Always ask how you will earn the judgement needed to catch the machine when it bluffs.

Worked example 1: Taking apart a viral headline

Suppose you see a post everywhere on LinkedIn claiming: "Firm X rolled out an AI assistant, so entry-level analyst roles are finished." Here is how you run the five checks instead of panicking:

  • Baseline → Did the firm actually hire fewer graduates, or did one spokesperson just say analysts now spend less time formatting slides? Check the primary press release or filing, not the viral screenshot.
  • Workflow → Analysts do more than paste numbers into templates. They chase missing data from clients, check footnote definitions and defend assumptions when a director pushes back.
  • Incentive → Announcing AI productivity wins plays well with shareholders regardless of how messy adoption is internally.
  • Failure mode → If a generated market summary invents a competitor metric before a client pitch, a human still takes the blame in the room.
  • Your move → Stop pitching yourself in interviews on how neatly you format slides. Pitch yourself on verification, clean data habits and knowing how to interrogate assumptions.

Worked example 2: Stress-testing a tool at work

Imagine you start a summer internship and your team asks you to trial a new AI research or drafting tool. Most people test tools badly and walk away fooled.

Bad test: typing "summarise this clean two-page article" or "write a marketing plan for a coffee shop." Every model looks brilliant on generic homework.

Good test: feeding it a gnarly, realistic task where you already know the exact right answer and the hidden traps:

  1. Pick a completed past task (using non-confidential or approved data only, after checking your organisation's data policy).
  2. Include a realistic contradiction, edge case or buried footnote that a careful human had to catch.
  3. Time the entire loop: prompting, reading the output line by line, checking every source link, and fixing what it smoothed over.
  4. Write down where it saved genuine effort and where it introduced subtle errors a tired colleague might wave through at 5:30pm.

Bring that one-page test note to your manager and you instantly become more useful than ten people forwarding hype threads.

Worked example 3: Using AI at university without dulling your edge

At uni, the biggest risk is not getting caught by a clumsy AI detector (though follow your department's rules strictly). The bigger risk is graduating with a transcript full of clean marks and a head that freezes the moment the Wi-Fi drops in a technical interview.

Draw one hard personal boundary between a ghostwriter and a sparring partner:

  • Ghostwriter (makes you weaker): pasting the essay question or coding assignment in, tweaking three adjectives, and convincing yourself you "edited" it because it made sense while reading it. Recognition is not understanding.
  • Sparring partner (makes you sharper): writing your own rough argument or code first, then asking the model to poke holes in your logic, quiz you on weak assumptions, or suggest edge cases your test suite missed.

Want a five-second honesty check? Close the laptop lid, grab a blank sheet of paper, and explain the core mechanism or argument from memory. If you stall after two sentences, the tool did the thinking, not you.

Worked example 4: Reading a benchmark bar chart

Every model launch comes with a tall bar chart showing the new release edging ahead of last month's leader. Before sharing that screenshot, look at three details on the chart itself:

  1. Check the vertical axis: a bar that looks twice as tall often starts at eighty-five percent rather than zero, turning a tiny nudge into a visual landslide.
  2. Check prompt parity: did the team tune settings and chain-of-thought budgets carefully for their own model while running rivals on default settings?
  3. Test your own messy task: two hours of hands-on testing on the documents, codebases or data formats from your own field beats any leaderboard score.

Your 60-second pocket checklist

Keep these five lines handy whenever a bold AI claim crosses your screen:

  1. Show me the primary source: skip the quote-tweet and read the actual paper, filing or methodology note.
  2. Name the bottleneck: does this fix the real constraint in the workflow, or just produce more drafts to review?
  3. Spot the incentive: what is the speaker trying to sell, justify or raise money for?
  4. Check the downside: how hard is a subtle hallucination to catch here, and who pays when it slips past?
  5. Protect your judgement: am I using this to see further, or to avoid doing the hard reps that make me good?

Ready to see how those five questions apply inside a specific career path? Explore our fifteen industry pages, read why we started theVANTAGEd, or browse our latest articles.

Practise this every two weeks

Every fortnight we apply this exact critical-thinking lens to one real industry shift, tool or operator interview. Free, written by students.