Seminar concept for lecturers

The problem

An essay written at home no longer shows reliably what students can do. With an LLM, a language model such as ChatGPT or Claude, a well-worded and technically plausible text of 2500 words can be produced in under an hour without anybody having engaged with the topic.

Detecting, prohibiting and asking for a declaration do not help. Detection of LLM text is unreliable, a prohibition cannot be enforced, and a declaration of independent work verifies nothing. All three approaches also ask whether a text was produced with an LLM. For grading, a different question matters: whether the person developed judgement while writing.

The idea

The format permits LLMs at all stages and assesses three abilities on three occasions:

The format therefore does not separate students who use an LLM from those who do not. It separates students who engaged with their text from students who did not. A student who drafts and revises with an LLM and understands the result is doing what the seminar is meant to teach, and is graded no differently from one who worked without an LLM.

Schedule

The seminar needs 13 sessions and one week without a session for writing. It is designed for five to eight participants.

The components in detail

Topics

Before the term starts, the lecturer compiles a list of narrowly defined topics. A topic is suitable if it is not trivial to explain, if 2500 words are enough for it, and if it differs enough from the other topics that students cannot share material. Broad topics such as “TLS” or “differential privacy” are not suitable: they are too large, and an LLM writes a usable overview of them without effort. A suitable topic is a single finding or mechanism, for example the question why companies rarely deploy passkeys although they are more secure than passwords.

The presentation about someone else’s draft

The presentation is the largest departure from a conventional seminar. It examines whether a student can read a technical text closely, judge it, and discuss it in public. A student who knows their own topic well but reads carelessly does poorly here.

The presenter chooses one of three approaches: a faithful account for a sound draft, a critique with a counterproposal for a draft with substantial problems, or three open questions for a draft that is interesting but incomplete. The choice is part of the grade, because it shows whether the draft was judged correctly.

A ring, not pairs

For the presentations, person 1 reads person 2’s draft, person 2 reads person 3’s, and the last person reads person 1’s. When two people read each other, both have an incentive to go easy on the other; in a ring that incentive is gone. The assignment is announced only after the revised drafts have been handed in, so that nobody writes for a particular reader. The assignment is not anonymous: in a group of this size, everybody would soon know who wrote what.

The pair editing before that does run in pairs. It is ungraded feedback, and the pair-editing partner is a different person from the later presenter.

The reviewer report

The lecturer reads each revised draft and writes a short reviewer report with exactly two issues: a question about clarity or structure that can be dealt with in 30 to 45 minutes, and an objection on substance that requires thought, such as a counterargument or a missing consideration. Students see the report for the first time in the last session.

Limiting the report to two issues keeps the last session manageable for students and the lecturer’s workload bounded. The reviewer report is the main basis of the revision; the feedback from the presentation comes on top. A student whose draft received a weak presentation is therefore not at a disadvantage.

Notes stay in the room

During the presentation about their draft, the writer takes notes on paper that the lecturer hands out, collects afterwards and returns only in the last session. The notes can therefore not be expanded into a revision plan at home with an LLM. Students can still think about the criticism, and they should.

Supervised revision

Students write on their own laptop, which stays offline. They may bring their sources as local files and a printout of their revised draft, but no prepared passages of text. The lecturer answers organisational questions, but no questions on content.

Bonus

Students can submit badly written paragraphs from published texts and explain in a paragraph of their own what is wrong with them. A good contribution improves the grade by one step; at most two contributions count. The contributions serve as material for the practice sessions in later terms.

Assessment criteria

The sessions are about taste: the ability to tell a good technical text from a bad one. Grades, however, follow criteria that can each be justified separately.

What the format makes harder, and what it does not

The format does not make strategic behaviour impossible. It makes superficial work more costly:

• A student who has a draft generated without understanding it can hardly deal with the two issues of the reviewer report in the last session.
• A student who prepares a presentation with an LLM and merely delivers it struggles in the discussion and with the on-the-spot task.
• A student who prepares a revision plan at home does not know the reviewer report that the revision has to answer.
• A good draft has to be revised too, because the reviewer report names two issues for each draft.

The format does not catch a student who uses an LLM throughout and understands what it produces. That is intended.

Alternatives that were rejected

• Pass or fail only. Solves the grading problem but gives up differentiation.
• A practical project in place of a text. That would be a different kind of course: a seminar is meant to teach reading scientific literature and writing scientific texts.
• An oral examination on each essay. The examination type provides for a presentation with discussion, not an oral examination; additional examined parts make grades contestable.
• Anonymous assignment of drafts. Ineffective in a group of five to eight. It also teaches that thorough criticism needs anonymity.
• Mutual reading in pairs. Invites both sides to go easy on each other; hence the ring.
• A final essay due 48 hours after the supervised session. Two texts would result, and an honest revision could no longer be told apart from later polishing by an LLM.

Workload for the lecturer

The following figures are planning values for five to eight participants.

Limitations

• The last session also measures speed and stamina. A student who revises thoroughly but slowly is at a disadvantage. Limiting the report to two issues reduces this but does not remove it.
• The format is built for small groups. With more than eight participants, two presentation sessions are not enough, and the work on the reviewer reports grows with each student.
• It assumes students who are confident writing in the language of the seminar. For groups with very uneven language skills or subject knowledge, the format would need to be adapted.
• The presentation examines reading, not knowledge of the topic. The course announcement should say so, so that students arrive with the right expectation.

Questions about the format and experience from your own runs are welcome.