Evaluator Cheat Sheet

Print this in landscape, or keep it open on a laptop, and have it in front of you for the session. Bring a tracking sheet as well, one per session.

What you have to judge

By the end of the session you place each member on the four individual objectives, and the team on the two group ones. Ask for what you are still missing while the team is in the room.

Judge each member on	Scope
Navigate and explain their work in the codebase	Individual
Criticise and defend the design of their work	Individual
Diagnose failures and validate features through the use of tests	Individual
Outline information flow across compiler passes and identify where language features are handled	Individual
Assess the implementation complexity of an unfamiliar language feature and argue a position on its design	Group
Evaluate and criticise whole-compiler design decisions	Group

How you are marked

Every member of the team you evaluate gives your team one mark out of 100 for how the session was run, against these:

The evaluating team	What that looks like
Directs questions at the right people	Questions match the work each member said they did.
Spreads the questions across the team	Every member is asked enough for their understanding to show.
Draws the knowledge out	A hesitant or incomplete first answer gets a follow-up.
Covers the required ground	Questions reach all seven coverage areas, and the group questions are substantive.
Runs the session professionally	Starts on time, manages the clock, lets the team navigate and show its own code, and engages without hostility.

Follow-up questions

Fill the blanks from what the team has just said.

To	Ask
Clarify	You said ______. Can you show me where that happens?
Go deeper	What happens if ______ instead?
Trace	Take ______ and walk it from the parser through to the output.
Justify	Why ______ rather than ______? What did that cost you?
Rescue a stall	Let us back up: what does ______ do at all? Open the file and read it with us.
Hypothetical	If we added ______ to the language, where would it touch first?
Redirect	______, you wrote ______. How does that interact with what we just heard?

Asking well

Follow up on a weak answer before you record it, and ask the same person a second way.
Let a silence run a few seconds. A student who is thinking looks like a student who is stuck.
Ask for the reasoning behind an answer rather than arguing with the answer.
Keep your face still. The team is reading it while they talk.
Ask one question at a time.
Say when an answer has landed, then move on.

The clock

1st	2nd	Where you should be
2:00	3:20	Presentation. Write down each member’s name and the areas they claim.
2:10	3:30	Q&A opens. Start with a member who named a specific feature.
2:40	4:00	Halfway. Every member should have answered by now — check the tally.
3:00	4:20	Ten minutes left. Fill the gaps: unticked coverage areas, unasked required questions.
3:10	4:30	Stop. Before you leave the room: a line on each member, and the ten contribution points split across the team.

Note