Large Language Thing

Home/Concepts/Negative feedback in education

Negative feedback in education

Every disturbance a system faces falls into two classes: those anticipated at design time, and those not. Against the first, prediction suffices. Against the second, only…

The certificate that lied

A polytechnic's data-analytics diploma reached its fourth cohort in 2022 still teaching SPSS as the primary tool, a decision-tree module built around a defunct pricing framework, and a capstone rubric that rewarded static dashboards. Employers had moved on. Job postings in the region had shifted almost entirely to Python and SQL competence two recruitment cycles earlier; the programme's own graduates, when surveyed at six months, reported spending their first ninety days on the job unlearning habits the diploma had certified. The programme director had not been negligent in any obvious sense. Enrolment was healthy. Pass rates were excellent. The external examiner's report, filed annually, praised the coherence of the syllabus. Nothing inside the institution's normal reporting had told her anything was wrong.

That is the specific shape of the failure: a curriculum certifying skills the market stopped valuing two cohorts ago, discovered not through any internal signal but through graduates' own difficulty finding relevant work — reported back, if at all, informally, months after the fact, by which point another cohort was already three months into the same syllabus.

What actually went wrong

The instinct is to blame the syllabus committee, or the director, or a lag in institutional bureaucracy. All three carry some blame, but the deeper fault is structural. The programme's intake of information about its own performance was designed around three streams — assessment results, module evaluations, and periodic accreditation review — and all three streams describe the curriculum against itself. Assessment measures whether students learned what the syllabus specified. Module evaluation measures whether students enjoyed learning it. Accreditation review measures whether the syllabus still matches the document that was approved three years prior. None of these streams contains any information about the labour market. The curriculum was, in the relevant sense, running open-loop: a plan fixed at design time, executed faithfully, with no channel through which the world's subsequent behaviour could register as an error signal.

Engagement telemetry existed — login frequency, time-on-task, forum activity — but it measured student attention, not market alignment. Even the capstone projects, which in principle touched live employer briefs, fed their outcomes back only as grades, stripped of the information that would have mattered: which techniques the partner organisation actually used afterwards, which the graduate found irrelevant on arrival, which skill the employer had assumed but the syllabus never covered. The curriculum had sensors. It had almost no sensors pointed at the disturbance that mattered.

Negative feedback and why it is the missing piece

Negative feedback is the practice of measuring what a system actually does, comparing that to what was wanted, and driving the difference toward zero. It does not require a model of the disturbance. Watt's governor does not know why the load changed; it only senses that the flywheel has slowed and throttles the valve accordingly. This is not a weaker form of control than prediction — it is a different kind, suited to exactly the class of problem prediction cannot touch: disturbances nobody enumerated in advance.

A curriculum is a control system whose plant is the labour market and whose actuator is the syllabus committee. Predictive curriculum design — forecasting which skills will matter in three years and building toward them — is feedforward, and it is not worthless; good programme directors do it constantly, reading sector reports, attending employer forums, extrapolating from hiring trends. But feedforward degrades the moment its model goes stale, and nothing forces anyone to notice the staleness, because feedforward carries no mechanism for checking itself against outcomes. The diploma's SPSS module was once a defensible forecast. By its fourth cohort it was an unmeasured error accumulating quietly, because nothing in the programme's design closed the loop between what graduates could do and what employers wanted done.

Closing that loop means treating labour-market signals, real-time job-posting data, alumni destination surveys at intervals shorter than a full cohort cycle, and employer feedback on capstone work as continuous inputs to curriculum revision — not as occasional inputs to five-year reaccreditation. The residual to drive toward zero is not "did students pass" but "does the gap between certified skill and demanded skill keep shrinking." That gap is the error signal. Nobody was measuring it in real time. That is the entire failure, restated precisely.

Where this sits on the intake axis

A curriculum built once from labour-market data and then held fixed behaves like a Large Language Model: an excellent feedforward model of the market as it stood at the moment of corpus collection, structurally blind to anything after that cutoff. The analogy is exact, not decorative — the diploma's syllabus was, in effect, a corpus of 2019 hiring practice frozen into a three-year credential.

A capstone project that pulls in a live employer brief, measures student performance against it, and adjusts grading and feedback within that single project behaves like a Large World Model: the loop closes, sensed error corrects the interaction, but only for the duration of the episode. Once the capstone ends and the cohort graduates, the loop opens again, and nothing carries that particular correction forward into the next cohort's syllabus design unless a human manually decides to.

The position this domain actually needs — every assessment stream, every engagement signal, every curriculum revision, every labour-market indicator running continuously, each belief about "what this qualification is worth" tagged with when it was measured and from which source, revisable the moment a contradicting signal arrives — is the Large Universe Model position on the intake axis. Not a fixed corpus. Not a bounded scene. A loop that does not open.

The failure was never a bad syllabus; it was a syllabus with no channel back from the world it was meant to serve.

The objections that actually bite

The sharpest technical objection is that continuous intake of every relevant stream is a recipe for instability, not robustness. A curriculum that revised itself against every fluctuating job-posting trend would oscillate — chasing a temporary spike in demand for one framework, dropping it the following term when the spike passed, producing graduates whiplashed between incoherent syllabi. This is correct, and it is why the answer is not "react to everything" but hierarchical loops running at different speeds: a fast inner loop of engagement and assessment telemetry adjusting pacing and support within a term, a slower outer loop of labour-market and alumni-destination data revising module content annually, and a still slower loop revising programme structure only when multiple independent signals agree across several cycles. Provenance is what makes this tractable — knowing that a demand spike comes from one employer's temporary hiring surge, dated last month, rather than from a sustained sector-wide shift measured over two years, is precisely what lets a director route it to the right loop instead of reacting to it directly.

The second objection worth taking seriously is that many of the relevant quantities are not observable in useful time. A graduate's skill mismatch may not show up as a measurable residual until eighteen months into employment, well past any point where the curriculum that produced it can be corrected for that cohort. Career trajectories, the slow atrophy of a technique's market value, the counterfactual harm of skills never taught — these leave no clean residual to instrument. This is a genuine limit, not a rhetorical one. The honest response is that extending intake — six-month and eighteen-month destination surveys, employer exit interviews tied by provenance to specific capstone cohorts, sector-wide postings data refreshed monthly rather than reviewed at reaccreditation — enlarges the observable set without pretending to complete it. A well-designed system flags where no current signal supports a long-held belief about a module's relevance, rather than defaulting to trusting a syllabus three years out of date because nothing has explicitly contradicted it yet. That is a weaker claim than omniscient curriculum design. It is also strictly better than what the diploma had, and there is no further category of evidence beyond continuous, provenance-tagged, multi-rate observation that would improve on it — only better instrumentation of the same kind.

Continue