Central thesis: Moral weight in designed systems is not produced by describing ethical stakes. It is produced by encoding ethical frameworks into consequence structures that transfer responsibility to the agent making decisions. An AI can evaluate the architecture. Only a human can feel the weight. The gap between those two statements is what this book teaches you to build.
| Field | Value |
|---|---|
| Series position | Course 3 of 3 · Irreducibly Human: What AI Can and Can't Do |
| Deployment context | 15-week graduate course |
A graduate engineering student who can describe the trolley problem and has never driven the trolley. Has taken one programming course. Has opinions about AI ethics. Has never been asked to make someone else feel morally implicated by something they built.
Present in every deliverable, every chapter. Not a single-chapter outcome.
This is not a reflection prompt. It is the primary evidence that the student operated above Tier 1. If a student cannot answer it with specificity, the deliverable has not cleared the course's minimum threshold regardless of technical quality.
The trap game and VE arrive in Week 1. Students encounter a designed failure before they have vocabulary to name it. Frameworks arrive as language for something already felt. The analysis comes after the experience — never before.
Both games are spiral objects introduced in Week 1 and returned to after every framework chapter. Each return must escalate — new analytical layer only possible because prior layers are in place. A return that only adds vocabulary is a repetition, not a spiral.
Students submit one paragraph of felt response to both games before the first framework chapter. No vocabulary. Felt experience only. This document is returned to students in Week 13. It is the only record of the pre-vocabulary response that makes the gap analysis meaningful.
Student can name how a specific mechanic transfers responsibility to the player — not in general terms, but with reference to a located design decision.
Student has a playtestable beta build and documented human playtesting data. The Ethical Auditor session without this data has only one side of the gap.
A pre-production GDD (v0.7) encoding consequentialist, deontological, and contractarian architecture across a mobile political satire simulation. Students receive the GDD in Week 1 and return to it after each framework chapter.
The spiral object is not the game — it is the design document. Students analyze architecture without felt play, mirroring exactly what the AI Ethical Auditor does. The gap between GDD analysis and felt play is the course's argument made structural.
A purpose-built game designed to fail in four specific ways simultaneously: (1) the editorializing failure — the game tells the player what to feel; (2) moral weight landing on the character, not the player; (3) framework described in text but absent from mechanics; (4) implication promised, smugness delivered.
The AI Ethical Auditor should pass this game — finding the framework and calling the architecture coherent. Human playtesters should report smugness. That result is the course's thesis demonstrated in a controlled case before students build their own.
Five ethical frameworks taught as game design constraints, not intellectual history. Each paired with a consequence architecture implication. Two spiral objects introduced in Week 1 and returned to after every chapter. Frameworks are precision instruments for something already felt.
VE GDD and trap game distributed. Week 1 response journal submitted before end of class — one paragraph, felt response only, no vocabulary. First pass diagnostic question posed: what ethical framework is embedded in VE? No answer expected. The question runs all semester.
Open with both games played/read in sequence. No framework vocabulary introduced. Ask only: what did you feel, and what was different between the two? Collect the response journals before the lecture begins.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-1.1 | Analyze | Distinguish between an AI's structural analysis of a designed system and a human player's felt moral experience, using the VE GDD as the diagnostic case. Student produces a written distinction with one specific AI claim and one human experience claim the AI cannot make. | Yes |
| LO-1.2 | Apply | Apply the implication/smugness distinction to a provided game example, specifying which design decisions produce implication and which produce smugness. | Yes |
| LO-1.3 | Understand | Explain why encoding an ethical framework in a mechanic differs from describing one in a text — using one example of each. | Yes |
| LO-1.4 | Analyze | Identify one design decision in the VE GDD that might embody an ethical framework and one that might only illustrate one — with reasoning for the distinction. | Yes |
Return to VE GDD. Identify the consequentialist architecture. Which specific mechanics encode it? Which claim to encode it but don't? Students compare this analysis with their Week 1 felt response.
Open with the VE causal chain display: a 4-node consequence chain from a specific edict. Ask: is each node caused by the one before it, or does it follow plausibly? The difference is the chapter.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-2.1 | Apply | Translate a consequentialist ethical position into a faction meter and traceable consequence structure for a domain-specific scenario, specifying what counts as an outcome and who bears the cost. | Yes |
| LO-2.2 | Create | Construct a 4-node consequence chain for one policy decision in the student's chosen project domain, meeting the Legible Causality standard: each node causally connected, each connection statable in one sentence. | Yes |
| LO-2.3 | Analyze | Identify the consequentialist architecture in the VE GDD by locating specific mechanics that embody it — and identify at least one mechanic that claims consequentialist logic but operates differently. | Yes |
| LO-2.4 | Evaluate | Evaluate a provided consequence chain against the Legible Causality pillar, specifying where causal logic holds and where it produces plausible-sounding outcomes without genuine causal connection. | Yes |
Return to VE GDD. Focus: the bribe escalation mechanic. Is it deontological or consequentialist? The answer is genuinely ambiguous. That is the point. Students now have two frameworks as lenses and can see the ambiguity they couldn't name in Week 1.
Open with the VE bribe escalation specifically: the moment where negotiating upward increases cost but the felt violation may not track the cost increase. Ask: was that moment a fine or a moral failure? The framework arrives as the answer.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-3.1 | Analyze | Distinguish a mechanical fine from a mechanical moral failure at the design level, identifying the specific structural features that produce each — using the VE bribe escalation mechanic as the test case. | Yes |
| LO-3.2 | Create | Design a rule-breaking mechanic for a domain-specific scenario in which the cost of violation produces felt moral weight rather than strategic calculation — specifying the design decisions that create that distinction. | Yes |
| LO-3.3 | Evaluate | Assess the VE bribe escalation mechanic: does it function as deontological architecture, consequentialist architecture, or both simultaneously — and does dual encoding strengthen or undermine the moral weight? Requires a defended position. | Yes |
| LO-3.4 | Analyze | Identify the point in a rule-breaking mechanic's design where the player's experience shifts from moral reasoning to cost-benefit calculation, and specify the design decision that causes the shift. | Yes |
Return to VE GDD. Focus: the Popularity score and the six outcome paths. Does the system design for the citizen who gets the worst outcome? Which mechanics encode virtue development and which record behavioral history only?
Open with the VE outcome card for 'Disappeared (Sovyetia only).' Ask: who designed this outcome — the player optimizing for survival, or the designer considering who lives in the game world? The chapter is the answer.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-4.1 | Create | Design a reward structure that tracks character development over time for a domain-specific scenario — specifying the difference between a system that records what the player did and one that encodes who the player is becoming. | Yes |
| LO-4.2 | Apply | Apply the Rawlsian veil of ignorance as a design constraint to a provided game system, specifying the worst-case player outcome and the design decisions required to minimize it without eliminating the game's moral architecture. | Yes |
| LO-4.3 | Evaluate | Evaluate the VE six-outcome structure against the contractarian standard: does the system design for the citizen who bears the cost of every edict, or for the player's strategic optimization? Name the specific design feature that answers the question. | Yes |
| LO-4.4 | Analyze | Distinguish a game mechanic that encodes virtue development from one that records behavioral history, using examples from the VE GDD and one other game of the student's choice. | Yes |
Students now have five frameworks as lenses. Full first-pass VE framework identification submitted as a written diagnostic. This document is returned for comparison with the AI audit findings in Week 12 and the gap analysis in Week 13.
Open with a designed editorializing failure — not from VE, but from the trap game. By Week 5, students have vocabulary to name exactly what went wrong. The editorializing failure is the last concept before the build begins because it is the mistake most likely to appear in the first draft of every student's game.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-5.1 | Analyze | Identify the editorializing failure — a design that tells the player what to feel rather than producing felt moral weight — in a provided game example, specifying the exact design decision that causes it. | Yes |
| LO-5.2 | Create | Design a relationship damage mechanic for a domain-specific scenario that makes loss legible to the player without emotional language — specifying how the mechanic communicates damage without moral commentary. | Yes |
| LO-5.3 | Evaluate | Select one ethical framework for the semester project and defend the selection against three criteria: domain fit, capacity to produce implication rather than smugness, and AI-auditor legibility. | Yes |
| LO-5.4 | Create | Produce a preliminary moral architecture specification: the ethical framework, the primary mechanic that embodies it, the consequence structure that transfers responsibility to the player, and one specific design decision predicted to produce implication in a human player. | Yes |
Students use Zelda (GDD tool) and Claude Code to build a web-based game encoding their chosen ethical dilemma in their chosen domain. The ethical framework is not disclosed in the GDD. The mechanic must embody the framework — not describe it.
VE GDD as structural model. Students are now writing a document of this type — an ethical argument encoded in design decisions. The question shifts from 'what framework is in VE?' to 'can I build a document that does what VE does?'
Distribute a redacted version of a student GDD from a prior course (or a purpose-written example). Class identifies the ethical framework from the mechanics alone. Then: what would it take to make the framework unidentifiable from the GDD? That failure mode is the design problem for the week.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-6.1 | Create | Produce a GDD that encodes the chosen ethical framework mechanically without naming or describing the framework anywhere in the document. The framework must be identifiable from the GDD by a reader who does not know which was chosen. | Yes |
| LO-6.2 | Evaluate | Evaluate the preliminary moral architecture specification from Week 5 against the completed GDD, identifying divergence between intended architecture and designed mechanics — and specifying what revision closed each gap. | Yes |
| LO-6.3 | Analyze | Identify the player experience goal the GDD's consequence structure is designed to achieve, stating it as the specific type of implication the player should feel — and the specific type of smugness the design actively avoids. | Yes |
VE as reference architecture. When students encounter a tool-generated solution that works mechanically but fails ethically, VE's Legible Causality pillar is the diagnostic standard: is each consequence caused by the decision, or does it only follow plausibly?
Open with a live demonstration: submit a VE-style mechanic specification to Claude Code, accept the output, run it, and ask the class: is this ethical architecture or a description of one? The demonstration should be designed to produce at least one ethically incoherent output.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-7.1 | Create | Implement the core mechanic of the semester project as a functional web-based prototype, with the consequence structure operable by a player. | Yes |
| LO-7.2 | Analyze | Document one instance where Claude Code generated a mechanically correct solution that was ethically incoherent — specifying the incoherence and the design decision required to correct it. | Yes |
VE's consequence engine as reference: three to four pre-authored consequence paths per edict, each causally coherent. Does the student's consequence structure meet the same standard? Self-audit against the VE design pillars.
Open with the self-audit as a structured exercise: students bring their GDD and their current build and identify the three largest gaps between them. The gaps are not failures — they are the data.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-8.1 | Create | Implement the full consequence structure of the semester project, including the mechanic encoding the ethical framework, the player choice architecture, and the cost structure. Playtestable build required. | Yes |
| LO-8.2 | Evaluate | Conduct a self-audit of the implemented consequence structure against the GDD specification, identifying where implementation matches intended architecture and where it has diverged. | Yes |
| LO-8.3 | Analyze | Identify the single mechanic in the current build that carries the most ethical weight — where the player's decision most directly transfers moral responsibility — and specify why it carries more weight than adjacent mechanics. | Yes |
VE's player experience goals (PX-1 through PX-8) as diagnostic frame. Students ask of their own game: which PX goals did my playtesters report experiencing? Which did they not? The VE PX goals are the vocabulary for this analysis.
Open with a structured debrief: one student presents their playtest data, class applies the implication/smugness distinction to the reports. The exercise models what Week 11 (peer playtesting) will require at scale.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-9.1 | Apply | Conduct an informal playtest with at least two players outside the course, collecting specific feedback on whether the game produced implication or smugness — using the evaluative distinction as a structured feedback instrument. | Yes |
| LO-9.2 | Evaluate | Assess the informal playtest data against the design intent in the GDD, identifying the largest gap between intended player experience and reported player experience — located in a specific design decision. | Yes |
| LO-9.3 | Create | Produce a build revision specification naming at most three design changes, each defended against the primary player experience goal. Three changes maximum is an acceptance criterion — prioritization is the skill. | Yes |
VE GDD as the model for the Ethical Auditor preparation document. The preparation document is a document of the same type as the VE GDD — an architectural description that encodes ethical position without naming it.
Distribute the Ethical Auditor prompt architecture. Walk through one example audit using the VE GDD as the input. Ask: what did the AI find? What did it miss? What would you have to change in the GDD to make it miss more?
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-10.1 | Create | Produce a beta build incorporating revisions from Week 9 playtesting, meeting minimum specification: ethical framework operable by a player, consequence structure transfers responsibility, game completable in a single session. | Yes |
| LO-10.2 | Create | Produce an Ethical Auditor preparation document: an architectural description enabling AI evaluation of the consequence structure — without revealing the ethical framework label anywhere in the document. | Yes |
| LO-10.3 | Evaluate | Predict, with specific reasoning, where the Ethical Auditor will correctly identify the embedded framework and where it will fail — and name the design decision most likely to mislead the AI. | Yes |
Before playtesting each other's games, students apply the feedback instrument to the trap game. This establishes a shared standard for what specific feedback looks like — and gives students one more encounter with the designed failure before they assess each other. If students produce vague feedback on the trap game, the instrument needs refinement before peer playtesting begins.
The trap game is played first by everyone. Feedback collected using the same instrument students will use on each other's games. The trap game's feedback should be easy to produce — the failures are designed to be visible.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-11.1 | Apply | Apply the implication/smugness evaluative distinction as a structured feedback instrument while playtesting another student's game, producing specific, located feedback about design decisions rather than general impressions. | Yes |
| LO-11.2 | Analyze | Distinguish, in the feedback received on your own game, between reports of felt experience and reports of structural observation — and identify which type is more useful for the Week 13 gap analysis. | Yes |
| LO-11.3 | Evaluate | Evaluate the peer playtesting data against the prediction from LO-10.3, identifying where the prediction was accurate and where human player experience diverged from the expected audit result. | Yes |
The Ethical Auditor session, the gap analysis, and the published games analysis. The course's thesis made visible as a classroom event: where does the machine's structural analysis diverge from human felt experience, and why?
VE GDD submitted to the Ethical Auditor as a shared class exercise before individual submissions begin. Class sees the AI audit of the aspirational architecture — what it finds, what it misses, where it is confident and wrong. This calibrates expectations before students submit their own work.
Run the VE Ethical Auditor session in class, live. The audit is visible to everyone. Class discusses the results before individual sessions begin. This is the moment the AI's Tier 1 competence is demonstrated at scale — and its Tier 3/4 limits become visible.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-12.1 | Apply | Submit the beta build and Ethical Auditor preparation document to Claude acting as Ethical Auditor, receiving a structural analysis of the embedded framework and an evaluation of where moral weight lands — on the player or the character. | Yes |
| LO-12.2 | Evaluate | Evaluate the AI audit report against GDD design intent, identifying where the structural analysis is correct, where it is correct but misses the human experience dimension, and where it is wrong. All three categories must be present. | Yes |
| LO-12.3 | Analyze | Compare the AI audit findings with the Week 5 first-pass VE diagnostic, identifying whether analytical ability improved across the semester — and specifying what changed in the student's approach. | Yes |
Week 1 response journals returned to students. The gap analysis begins with the felt response from Week 1 — before vocabulary — and ends with the AI audit findings from Week 12. The arc: felt it without naming it → named it with five frameworks → built it → the AI evaluated the structure → now name the gap between the structure and the feeling.
Distribute the Week 1 response journals at the start of class. Students read their own words from fifteen weeks ago. Ask: what did you feel then that you can now name? What did you feel then that you still cannot name — and is that gap the answer to the course's question?
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-13.1 | Create | Construct a gap analysis tracing at least three specific design decisions to specific divergences between AI audit findings and human playtester experience — naming the human variable the structural analysis could not reach. Three decisions, three divergences, one named human variable per divergence. | Yes |
| LO-13.2 | Evaluate | Identify the one judgment call in the semester project that required values, domain knowledge, or accountability that an AI could not have made — with specific reference to the design decision, the alternative the AI would have generated, and why that alternative would have failed the player experience goal. | Yes |
| LO-13.3 | Create | Propose one design revision that would close the largest gap between architectural legibility and felt moral weight — specifying the design decision and the predicted change in player experience. The prediction must be falsifiable. | Yes |
Students now apply to published games the same analysis they applied to their own builds. Papers Please and Spec Ops enter as the external calibration — games built by professionals with full production resources, evaluated against the same standards.
Play 15 minutes of Papers Please in class. Ask: at what moment did you feel implicated? Locate that moment in a specific mechanic. Then ask: was that the moment the designer intended? How would you know?
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-14.1 | Analyze | Identify the ethical framework embedded in Papers Please's consequence structure from mechanical evidence alone — specifying the mechanics that encode it and the design decisions that make it operable rather than described. | Yes |
| LO-14.2 | Evaluate | Evaluate whether Spec Ops: The Line achieves player implication or produces a different moral response — specifically whether the structural subversion mechanic produces felt weight or produces commentary about felt weight in another medium. Position required, defended with specific design decisions. | Yes |
| LO-14.3 | Apply | Apply the Ethical Auditor framework to Papers Please, producing a structural audit report in the format used in Week 12 — and compare the result against documented player experience data from published criticism and player reports. | Yes |
If a playable version of VE exists by Week 15, students play it for the first time and compare their felt response to their semester-long architectural analysis of the GDD. The gap between analyzing a system and experiencing it is the course's final argument demonstrated in real time.
Open with the question the course has been building toward: if you had to write one sentence that a designer could use to distinguish a system that produces moral weight from one that describes it — what would that sentence be? Students write it before the lecture. The lecture is the attempt to make the sentence precise enough to be useful.
| ID | Bloom's | Learning outcome | Assessable |
|---|---|---|---|
| LO-15.1 | Analyze | Identify the moral architecture of Disco Elysium — specifying whether it embodies one of the five frameworks studied, a combination, or requires a new category — defending the classification with reference to specific design decisions. 'Requires a new category' is acceptable if the student can specify what the new category is and why none of the five frameworks capture it. | Yes |
| LO-15.2 | Create | Produce a comparative analysis of two published games' moral architectures — specifying which design decisions produce moral weight and which produce description of moral weight — and connect the comparison to the student's own gap analysis from Week 13. | Yes |
| LO-15.3 | Evaluate | Articulate the gap between AI-auditable architecture and human-felt moral weight as a general design principle — with specific evidence from the Ethical Auditor session (Week 12), the gap analysis (Week 13), and one published game. The principle must be stated in one sentence. If it cannot be stated in one sentence, it is not yet a principle. | Yes |
The items below are design decisions that must be resolved before the course runs. They are not administrative questions.