Expert Encoding System

Section 01

What We Are Encoding

Great educators are not great because of their content. They are great because of how they respond to student thinking. The way they praise, the way they challenge, what they notice first, what they refuse to let slide, how they adjust when a student is struggling versus thriving.

That responsive intelligence is what we are encoding. The word “voice” undersells it. We are encoding a pedagogical identity — a composite of seven distinct layers, each of which varies independently between educators.

The current platform has one hardcoded educator voice — your voice — with 7 pedagogical moves, a critique structure, key phrases, and a teaching philosophy baked into four prompt files. It works beautifully for one workshop.

Expert Encoding is the mechanism that makes this a platform. It is the difference between a tool that delivers one workshop and a system that captures what makes any educator great and reproduces it at scale. It is the core IP.

Section 02

The Seven Layers of Pedagogical Identity

Section 03

System Architecture

The encoding system sits alongside the existing workshop engine as a parallel tool for educators. It produces encoded prompt data that the existing API routes consume.

Key Architectural Principles

• Purely additive. The encoding system does not modify existing code paths. The hardcoded workshop continues to function exactly as it does today. Encoding data is consumed through two functions — assemblePrompt() and assembleFewShotMessages() — which are the entire integration surface.
• Framework-agnostic. The encoding system does not assume CMRC or any specific pedagogical framework. The Framework block is populated from the educator’s own vocabulary. This means the synthetic student generator and the workshop submission shape can accommodate different frameworks in future.
• Modular prompts. The final encoded prompt is assembled from discrete blocks (identity, framework, moves, tone, calibration, scaffold) that can be swapped depending on the interaction stage. This keeps prompts focused and within token budgets.
• Progressive fidelity. Encoding starts rough and improves through iterative calibration. The educator sees the AI get better in real time. The system tracks version history of encoded prompts.

Section 04

The Encoding Process

The encoding process has five phases, each capturing different layers. Total time investment: approximately 90 minutes. This is deliberate — encoding an expert should feel like a serious professional process, not a quick form.

Section 05

The Calibration Loop

This is the heart of the system. Get this right and the encoding works. Get it wrong and you have a generic AI pretending to be a specific person.

The Loop: One Round

Each round of the critique calibration loop has four phases:

Phase A: Read and Critique

The educator sees a synthetic student’s submission, presented exactly as the existing critique page presents it — same layout, same typography, same structure. Student background is shown as one sentence at the top. Quality band is not shown (to prevent bias). Provocation history is behind a toggle.

Below the submission: a single large text area. No structure imposed. No form fields. The prompt says: “Critique this student’s work as you would in person. Be yourself.” This is critical — the moment the interface imposes structure, you get structured responses instead of authentic ones.

Phase B: Self-Reflection

After saving the critique, four questions extract metacognitive process:

1. “What was the first thing you noticed about this work?”
2. “What did you choose NOT to say, and why?”
3. “On a scale of 1–5, how confident are you in this student’s potential?”
4. “If you could tell this student one thing, what would it be?”

Phase C: Side-by-Side Comparison

After the educator writes their critique, the AI generates its own critique of the same student using the current draft prompt. Two columns: left is “Your words,” right is “The encoded version.” Identical formatting. No value-laden labels like “human” versus “AI.”

Phase D: Gap Feedback

A structured feedback form. All fields optional except at least one must be completed:

1. “What did the AI miss?” — Free text. Captures false negatives.
2. “What did the AI say that you would never say?” — Free text. Captures anti-patterns. Extremely high-value signal.
3. “How was the tone wrong?” — Multiple choice (too harsh / too soft / too generic / too formal / too casual / wrong emphasis) + free text.
4. “Rate the AI’s attempt” — Slider, 1–10. Anchored: 1 = “completely wrong voice”, 5 = “recognisably similar but noticeably off”, 10 = “I could have written this.”
5. “What’s the single most important thing to fix?” — Free text. Forces prioritisation.

Student Sequencing

The 5 synthetic students are presented in this deliberate order (not by band number):

What Happens Between Rounds

After each round of feedback:

1. The system takes the educator’s gap analysis
2. Feeds it to Claude as a meta-prompt: “Here is the educator’s critique and the AI’s attempt. Here is what the educator says is wrong. Update the system prompt to address these gaps.”
3. Stores the updated prompt as a new version
4. Uses the updated prompt for the next round’s AI critique

This creates a version history. Each version should be better than the last. The educator sees improvement in real time.

Completion Criteria

• Educator satisfaction: When they rate 7+ for two consecutive rounds, encoding is “complete enough.”
• Convergence: If free-text feedback is getting shorter, marginal returns are diminishing.
• Maximum: 10 rounds. If satisfaction hasn’t been reached, the system flags this honestly.

The Holdout Test

After encoding is complete, generate 2–3 new synthetic students the educator has never seen. The AI critiques them using the final encoded prompt. The educator rates: voice fidelity, diagnostic accuracy, calibration accuracy (each 1–10) and flags any anti-pattern violations. This is the truest measure of encoding quality.

Section 06

Synthetic Students

We need five distinct quality bands. Each represents a meaningfully different kind of student and tests a different dimension of the educator’s range.

What Each Synthetic Student Contains

Each synthetic student is a complete WorkshopState object, matching the exact shape in workshop-state.ts:

• Persona name and one-sentence background
• Quality band (hidden from the educator during encoding)
• Failure modes (what specifically goes wrong in their thinking)
• Complete briefing responses
• Briefing summary
• Full ideation state (collect/model/render + provocation history showing trajectory)
• Complete submission (challenge, product anatomy, user journey, stewardship)

Provocation History Is Critical

The provocation history must be generated too, because it shows the student’s trajectory. A Band 1 student improves dramatically across 5 provocations. A Band 4 student nods along but doesn’t actually change their thinking. This trajectory is essential context for the educator’s critique.

Domain Coverage

The 5 synthetic students for each workshop should cover at least 4 of the 8 domains (Education, Health, Travel & Transport, Government, Retail, Manufacturing, Environment & Energy, Creative Arts & Culture), ensuring the educator encounters varied subject matter.

Section 07

Modular Prompt Assembly

The final encoded system prompt is not one monolithic blob. It is assembled from discrete blocks that can be swapped depending on the interaction stage.

The Seven Prompt Blocks

Total declarative prompt: ~3,000 tokens. This leaves room for workshop-specific context (the submission, lecture content, etc.).

Stage-Specific Assembly

Each workshop stage uses a different combination of blocks:

Shared blocks (Identity, Framework, Tone) ensure consistency. Stage-specific blocks (Moves, Evaluation, Scaffolds) ensure appropriate behaviour.

Few-Shot Examples

The educator’s actual critiques become few-shot examples, placed in the messages array (not the system prompt):

// Assembled message array for a real student critique
[
  { system: "[assembled blocks for critique stage]" },
  { user: "Here is a student submission: {Band 2 submission}" },
  { assistant: "{Educator's actual Band 2 critique}" },
  { user: "Here is a student submission: {Band 4 submission}" },
  { assistant: "{Educator's actual Band 4 critique}" },
  { user: "Here is a student submission: {real student's submission}" }
]

Two examples are optimal — one strong student, one weaker student — showing the AI the range of the educator’s voice. Three is better if token budget allows.

The Integration Surface

// These two functions are the entire integration surface.
// Everything else in the encoding system exists to produce
// the data that feeds into them.

function assemblePrompt(
  encoding: EncodedEducator,
  stage: "critique" | "ideation" | "briefing" | "lecture",
  context: Record<string, unknown>
): string;

function assembleFewShotMessages(
  encoding: EncodedEducator,
  stage: "critique" | "ideation" | "briefing" | "lecture"
): Array<{ role: "user" | "assistant"; content: string }>;

Section 08

Data Shapes and Types

All types live in src/lib/encoding/types.ts. They import from the existing workshop-state.ts types.

import type { WorkshopState, SubmissionData } from "@/lib/workshop-state";

// --- Synthetic Students ---

export type QualityBand = 1 | 2 | 3 | 4 | 5;

export interface SyntheticStudent {
  id: string;
  personaName: string;
  background: string;          // one sentence
  qualityBand: QualityBand;
  failureModes: string[];      // what goes wrong in their thinking
  workshopState: WorkshopState;
  isHoldout: boolean;          // reserved for final test
}

// --- Encoding Session ---

export type EncodingPhase =
  | "interview"
  | "ideation"
  | "briefing"
  | "critique"
  | "lecture"
  | "holdout"
  | "complete";

export type CritiqueRoundPhase =
  | "writing"       // educator writes critique
  | "reflecting"    // self-reflection questions
  | "comparing"     // side-by-side view
  | "feedback";     // gap feedback form

export interface SelfReflection {
  firstNoticed: string;
  choseNotToSay: string;
  confidence: number;          // 1-5
  oneThing: string;
}

export interface GapFeedback {
  missed: string;              // what AI missed
  neverSay: string;            // what AI said that educator never would
  toneIssues: string[];        // checkboxes + free text
  rating: number;              // 1-10 slider
  priorityFix: string;         // single most important thing to fix
}

export interface EncodingRound {
  roundNumber: number;
  studentId: string;
  stage: "critique" | "ideation" | "briefing" | "lecture";
  educatorResponse: string;
  aiResponse: string | null;
  selfReflection: SelfReflection | null;
  gapFeedback: GapFeedback | null;
}

export interface EncodingSession {
  id: string;
  workshopId: string;
  status: EncodingPhase;
  currentRound: number;
  rounds: EncodingRound[];
  promptVersions: EncodedPrompt[];
  createdAt: string;
  completedAt: string | null;
}

// --- Encoded Prompt Blocks ---

export interface EncodedPrompt {
  version: number;
  identityBlock: string;
  frameworkBlock: string;
  toneBlock: string;
  calibrationBlock: string;
  ideationMovesBlock: string;
  ideationScaffoldBlock: string;
  critiqueEvaluationBlock: string;
  critiqueScaffoldBlock: string;
  briefingScaffoldBlock: string;
  lectureScaffoldBlock: string;
}

// --- The Full Encoded Educator ---

export interface EncodedEducator {
  id: string;
  workshopId: string;
  prompt: EncodedPrompt;       // latest version

  // Few-shot examples per stage
  critiqueExamples: Array<{
    submission: SubmissionData;
    critique: string;
  }>;
  ideationExamples: Array<{
    context: string;
    provocation: string;
  }>;
  briefingExamples: Array<{
    responses: string;
    summary: string;
  }>;
  lectureExamples: Array<{
    question: string;
    answer: string;
  }>;

  // Metadata
  encodingVersion: number;
  calibrationScores: number[];
  holdoutScores: {
    voiceFidelity: number;
    diagnosticAccuracy: number;
    calibrationAccuracy: number;
    antiPatternViolations: boolean;
  } | null;
}

Section 09

File Structure

Every new file the encoding system introduces, and every existing file it modifies. Green is new, yellow is modified.

Total new files: ~25. Modified existing files: 3 (API routes only, purely additive changes).

Section 10

Build Sequence

The build is ordered so that each step produces something testable. Steps 1–4 are foundational. Steps 5–9 build the critique calibration loop. Steps 10–13 build the remaining encoding phases. Step 14 integrates everything with the workshop.

Section 11

Integration with the Existing System

The existing workshop continues to work exactly as it does today. The encoding system is purely additive. Here is how the modified API routes work:

// src/app/api/critique/route.ts (modified)

import { chat } from "@/lib/claude";
import { getCritiqueSystemPrompt } from "@/lib/prompts/critique-prompt";
import { assemblePrompt } from "@/lib/encoding/assemble-prompt";
import { assembleFewShotMessages } from "@/lib/encoding/assemble-few-shots";

export async function POST(req) {
  const { submission, encoding } = await req.json();

  let systemPrompt: string;
  let messages: Array<{ role: string; content: string }>;

  if (encoding) {
    // Use encoded educator prompt
    systemPrompt = assemblePrompt(encoding, "critique", { submission });
    const fewShots = assembleFewShotMessages(encoding, "critique");
    messages = [
      ...fewShots,
      { role: "user", content: `Review this submission:\n${formatSubmission(submission)}` }
    ];
  } else {
    // Fallback to existing hardcoded prompt
    systemPrompt = getCritiqueSystemPrompt(submission);
    messages = [
      { role: "user", content: "Please review this submission and deliver your critique." }
    ];
  }

  const result = await chat(systemPrompt, messages);
  return NextResponse.json({ critique: result });
}

The pattern is identical across all three API routes: check for encoding data, assemble prompt if present, fall back to hardcoded prompt if not.

Section 12

Database Schema

This schema is prepared as a migration file for when Supabase auth is in place. For version one, the same data shapes are stored in localStorage.

-- Encoding sessions
create table encoding_sessions (
  id uuid primary key default gen_random_uuid(),
  educator_id uuid not null references profiles(id),
  workshop_id uuid not null references workshops(id),
  status text not null default 'interview'
    check (status in (
      'interview', 'ideation', 'briefing',
      'critique', 'lecture', 'holdout', 'complete'
    )),
  current_round integer default 0,
  created_at timestamptz default now(),
  completed_at timestamptz
);

-- Synthetic students for encoding
create table synthetic_students (
  id uuid primary key default gen_random_uuid(),
  session_id uuid not null references encoding_sessions(id) on delete cascade,
  workshop_id uuid not null references workshops(id),
  persona_name text not null,
  background text not null,
  quality_band integer not null check (quality_band between 1 and 5),
  failure_modes text[],
  workshop_state jsonb not null,
  generation_metadata jsonb,
  is_holdout boolean default false,
  created_at timestamptz default now()
);

-- Encoding rounds (each critique-compare-calibrate cycle)
create table encoding_rounds (
  id uuid primary key default gen_random_uuid(),
  session_id uuid not null references encoding_sessions(id) on delete cascade,
  stage text not null check (stage in ('critique', 'ideation', 'briefing', 'lecture')),
  round_number integer not null,
  student_id uuid not null references synthetic_students(id),
  educator_response text not null,
  ai_response text,
  gap_feedback jsonb,
  self_reflection jsonb,
  created_at timestamptz default now()
);

-- Versioned encoded prompts
create table encoded_prompts (
  id uuid primary key default gen_random_uuid(),
  session_id uuid not null references encoding_sessions(id) on delete cascade,
  version integer not null,
  identity_block text,
  framework_block text,
  tone_block text,
  calibration_block text,
  ideation_moves_block text,
  ideation_scaffold_block text,
  critique_evaluation_block text,
  critique_scaffold_block text,
  briefing_scaffold_block text,
  lecture_scaffold_block text,
  created_at timestamptz default now()
);

-- Few-shot examples
create table encoding_examples (
  id uuid primary key default gen_random_uuid(),
  prompt_id uuid not null references encoded_prompts(id) on delete cascade,
  stage text not null check (stage in ('critique', 'ideation', 'briefing', 'lecture')),
  input_data jsonb not null,
  educator_response text not null,
  created_at timestamptz default now()
);

-- Holdout evaluations
create table holdout_evaluations (
  id uuid primary key default gen_random_uuid(),
  session_id uuid not null references encoding_sessions(id) on delete cascade,
  student_id uuid not null references synthetic_students(id),
  ai_critique text not null,
  voice_fidelity integer check (voice_fidelity between 1 and 10),
  diagnostic_accuracy integer check (diagnostic_accuracy between 1 and 10),
  calibration_accuracy integer check (calibration_accuracy between 1 and 10),
  anti_pattern_violations boolean,
  anti_pattern_details text,
  created_at timestamptz default now()
);

Section 13

Edge Cases and Mitigations

Synthetic students don’t feel real

If synthetic students read as obviously AI-generated, the educator will critique the artificiality rather than the content.

The educator’s voice is too subtle to capture

Some educators’ power is in pause, timing, facial expression — things that don’t translate to text.

The educator is inconsistent across critiques

Human educators have good days and bad days. They respond differently to work that touches their interests.

Charlotte teaches outside CMRC

The system must not assume CMRC or any specific framework. The Framework block is populated from the educator’s own vocabulary.

The AI plateaus below satisfaction

After 10 rounds the educator still rates below 7.

Token budget pressure

Assembled prompts (~3,000 tokens) plus few-shot examples (2–3 full critiques) plus the actual student submission could push against context limits.

Section 14

Verification Protocol

How we know the system works, tested in sequence:

Technical Verification

1. Synthetic student realism: Generate 5 synthetic students. Read them cold. Can you tell which band they’re in without being told? Do they feel like real students?
2. Encoding flow: Walk through the full encoding process. Does the interface let the educator be themselves, or does it constrain them?
3. Calibration convergence: After 5 rounds of feedback, does the AI’s critique sound recognisably like the educator? What’s still missing?
4. Prompt assembly: Verify that assemblePrompt() produces valid system prompts for all four stages. Check token counts are within budget.
5. Regression: Existing workshop still works with hardcoded prompts. No changes to student-facing behaviour when no encoding is present.

Voice Verification

1. Holdout test: Show yourself 2–3 AI-generated critiques of students you’ve never seen. Could you have written these? Would your students believe you wrote them?
2. Cross-encoding test: Run the same process with Charlotte. Her encoding should sound distinctly different from yours. If the two encoded voices are indistinguishable, the system is capturing something generic, not something personal.
3. The ultimate test: Show an encoded critique to someone who knows the educator. Can they identify whose voice it is without being told?

Acceptance Criteria for Version One

Section 15

Version One Scope: Your Workshop and Charlotte’s

Version one delivers the complete Expert Encoding system, proven on two educators with distinct pedagogical identities. Here is exactly what it includes and what it defers.

Included in Version One

Deferred to Version Two

Charlotte’s Workshop: What We Need to Know

Regardless of the answer, the encoding system itself is framework-agnostic. The seven prompt blocks capture her concepts, her moves, her values. The only place the CMRC assumption lives is in the WorkshopState / SubmissionData type shape and the synthetic student content. If Charlotte’s framework maps reasonably onto collect/model/render/control fields (even with different labels), version one works. If it’s fundamentally different, we parameterise the submission shape — which is a clean, bounded change.

The Version One Experience

For you: Navigate to /educator/encoding. Walk through all five phases (~90 minutes). See your pedagogical identity decomposed into seven layers. Watch the AI get progressively better at sounding like you across five calibration rounds. Verify with holdout students. Your encoded voice is now live for every student who takes your workshop.

For Charlotte: Same process, same interface. Her encoding captures a distinctly different pedagogical identity. Her students get critiques that sound like her, not like you. The proof that the system works is that two outputs of the same encoding process are recognisably different people.

For students: Nothing changes about their experience. They still go through Lecture → Briefing → Ideation → Submission → Critique. The only difference is that the voice behind the critique, the ideation provocations, and the briefing summary is now specifically their educator, not a generic design educator.