I conducted an ideation session for my TDS course. It was an open session along with students, TAs and instructors. It roughly explained how we conduct TDS - and in particular, how we design questions that teach something even when students can (and are encouraged to) work with AI agents as well as each other.

Here’s the link to the session

Here’s my summary of the session:

  1. Pick what to teach. My AI Advice page lists AI-era skills. Yours may vary.
  2. Evaluate questions against six tests:
    1. Learning test. Is the step the student must do themselves the skill we want - and not just pointless difficulty?
    2. Forward test. Does one student’s submission score for another?
    3. Solver test. Would one shared tool score for everyone?
    4. Agent test. Could Codex solve the question with the student’s logged-in browser / terminal?
    5. Faking test. Is faking costlier than doing it honestly, and does faking still teach?
    6. Oracle test. Is grading server‑side, robust (behavior, not text-matching), and rate‑limited or revealed after the test?
  3. Measure learning transfer. How do we know the actually learnt the skill?

(Interestingly, just passing these principles to Claude / ChatGPT gives me pretty good AI and collaboration proof questions on many topics.)

And a couple of meta principles:

  • Agents improve fast. What we teach, how to teach, and how we evaluate, needs to evolve rapidly.
  • Test and fix fast. Being able to experiment (e.g. test questions) make it easy to try new ideas. Agents speed this up.
  • Design in the open. Maybe students (and others) learn from the process - and shape it with ideas.

Comic

Links: