Post #3747736

Wednesday we had a big systems faculty meeting to discuss - of course - how to make sure our courses continued to work in the face of AI being able to one-shot nearly any programming assignment we would create for undergrads. One clear take-home from it was that we need to move of the "did you learn what we want from the projects?" into the assessments instead of where we used to have it -- working code handed in. Last semester I added weekly very short (<= 5 minute) weekly quizzes to my class of 250 students. I was worried about the logistics, but it turns out to have worked pretty well: I designed the quizzes to be easy to grade, and gradescope's clustering let my TAs collect, scan, and grade them often before class was over, which is pretty awesome. I'm upping the game a little for the coming semester, making the quizzes just a hair harder, filling in the weeks I missed this semester, and attaching a few more points to them. And ironically - I ended up using AI to wrangle a lot of it. In case it's useful for others... (All work done using google's antigravity 'agy' CLI tool) 1) Cleaned up the typst code for all of the quizzes, extracting what was a bunch of one-off copy-paste quiz descriptions into a coherent template and refactoring all of the quizzes to use the template. Then had it add a makefile and some targets to build a single PDF of all of them, etc. Took about 15 minutes. 2) Had it rename and align all of the quizzes with the weeks of the class instead of being numbered sequentially, which made it annoying for me to see where I'd left gaps. Then had it match up the topics list on the schedule with the topics extracted from the quiz question to identify topic coverage. 3) I sent a draft of the quizzes to one of my TAs, who commented that I'd made some of the questions awfully similar to the previous semester, but the students could access the quizzes because we scanned them in to gradescope. Whoops, forgot about that. So had gemini match each new question about its nearest counterpart in the previous semester and create an "excess similarity" report (shown below). It probably wasn't perfect but the top 4 it identified were really too close, and that's what I needed to fix. 1/2