Why We Put an Algorithm on Trial with Year 10

One of the lessons in our Year 10 module has no laptops, no prompts, and no AI tool open anywhere in the room. It is a courtroom, of sorts: a fictional algorithm stands accused of denying someone a loan, and the students are prosecution, defence, and jury.

I mention it whenever someone asks what "AI literacy" actually looks like in practice, because it is the clearest answer I have. Most people, understandably, picture AI literacy as tool training: how to write a good prompt, which app to use for which task. That is a real skill and we teach it, but it is not the skill that matters most by Year 10, and the trial exercise is built around a harder one: can a student who has never written a line of code interrogate a decision an algorithm made, and say, precisely, where it went wrong.

Quite a few parents I know are educators themselves, teachers mainly, and I hear some version of the same worry from them regularly. As one put it to me: "I don't like what's happening with AI. It makes my students lazy and educators like me stressed, and changes the classroom, not in a good way." What they are describing, more precisely, is cognitive surrender: given a tool that can produce a passable answer instantly, the easier path is to stop thinking altogether and hand the friction over to the machine, and a classroom full of students taking that easier path is a genuinely harder classroom to teach.

I do not think that is wrong, as a description of what happens by default. But I do not think it is the only option, either. The question worth asking is not how to keep AI out of the room, which is not achievable in any case, but how to change what we ask students to do in it, so that offloading the thinking to AI stops being the easy way to pass the task. The trial is one answer to that question, not a defence against cognitive surrender so much as a lesson designed so there is nothing to surrender to in the first place: no AI tool in the room can stand up and argue a position under cross-examination on a student's behalf.

Why a trial, and not a worksheet

A worksheet on algorithmic bias asks students to read about the problem and answer questions on it, which is roughly the same as asking someone to learn to argue by reading a transcript of other people arguing. The trial format asks each student to actually do the thing: build a case, anticipate the other side's rebuttal, and defend a position under questioning they did not fully control. That is a different cognitive demand entirely, and it is closer to what judgment actually requires, deciding, under some pressure, whether a specific claim holds up, rather than recognising in the abstract that bias in AI systems is "a thing that can happen."

The evidence in the case is deliberately mixed, not a clean villain. The fictional algorithm gets some decisions right and some wrong, for reasons that trace back to what data it was trained on and what proxy it used for creditworthiness, which mirrors how these problems actually surface in the world: rarely as an obviously broken system, usually as a system that is right often enough to be trusted and wrong often enough, in a particular pattern, to matter.

What the format teaches that a tool-training session cannot

Tool training teaches students what a system can do. The trial teaches them what a system's confidence is worth, which is a separate question and, I would argue, the more durable one, because the specific tools students use will be different again in two years, while the habit of asking "on what basis, exactly, did this conclusion get reached" transfers to whatever comes next. A student who has stood up in front of classmates and had to defend or dismantle an algorithm's reasoning under questioning does not forget how to ask that question later, when the system in front of them is a real one making a real decision about them.

It also does something a worksheet cannot: it makes the abstract case personal without being confessional. Nobody in the room is required to share their own experience with an unfair decision, which some students would find uncomfortable, but everybody has to reason through what fairness would actually require in a specific, contested case. That distance, close enough to matter, far enough to argue about safely, is doing a lot of the pedagogical work.

Why this matters for Malaysian schools specifically

The 2027 AI curriculum mandate will require schools to show they are teaching AI literacy, and the easiest thing to produce as evidence is a scheme of work built around tool familiarity: this is ChatGPT, this is how you use it responsibly, tick the box. I understand the appeal; it is quicker to build and easier to assess. But it answers the wrong question for what students will actually face, which is not "do you know how to use an AI tool" but "can you tell when one has let you down, and say why." Judgment-first lessons like the trial are harder to plan, harder to mark consistently, and, in my experience, the only version of AI literacy that still looks like literacy five years from now.

What This Means for You

If your school's AI literacy programme could be delivered equally well by a vendor's self-paced module, it is probably teaching tool use, not judgment, and the two are not interchangeable however similar they sound in a policy document. Judgment has to be practised somewhere a student can be wrong, corrected, and asked to defend their reasoning again. A trial, a debate, a case study argued out loud, the format matters less than the requirement that a student build and defend a position under real questioning. That is the part no AI tool teaches on its own, and the part schools are actually accountable for.