Measuring Coherence
Of all the demands a skeptic can make of “increasing coherence over increasing context,” the most natural is also the most dangerous: measure it. By how much is this more coherent than that? Put a number on it, the demand goes, or admit the criterion is empty. It sounds like rigor — and the loose reply, that the criterion is about direction rather than magnitude, sounds like evasion. This essay gives the answer that is neither. It meets the part of the demand that is legitimate — the demand for comparison — with exactly the structure ethics needs; and it declines the part that is not — the demand for a single number — not from vagueness, but because that number, if supplied, would reward precisely what the framework condemns.
Two demands, not one
The confusion that makes this objection feel devastating lives entirely in a conflation of two different demands, and the first task is to pull them apart.
The demand for comparison is legitimate, and the framework must meet it. To say that one act is more moral than another, or that a development is progress, is to compare; and the master criterion is comparative through and through. There is no honest way to refuse this and keep the thesis, because the thesis is a comparative claim. Even the tempting escape — “it is process-based; only the direction matters” — does not avoid comparison. “Tends toward increasing coherence” already says that later states are more coherent-over-context than earlier ones. Direction is comparison; you cannot lean on the one to dodge the other.
The demand for a cardinal metric — a number on a scale, so the distance between two states can be named as a quantity — is a different and far stronger thing, and it is the one to decline. The honest target, then, is not “no measurement” but the right amount of structure: a partial order built from local comparative judgments — enough to compare where comparison is real, never so much as to pretend to a complete ranking or a common scale that is not there.
The dominance result
Here is the move that does most of the work, and it needs no arithmetic. The criterion is two-dimensional: it is coherence and context, never coherence alone. And a two-dimensional comparison yields a determinate verdict wherever one option beats another on both dimensions at once — the structure logicians and economists call dominance, and the rest of us use whenever we judge one rectangle larger than another because it is both taller and wider, no ruler required.
Lay the moral cases out the same way. A change that raises coherence and widens context dominates: unambiguous improvement, nothing to measure. A change that raises coherence while narrowing context is the counter-dynamic: coherence achieved by shutting things out, unambiguous regress, and again nothing to measure — a cult or a sealed ideology is visibly coherence-up and context-down, and you can see the regress without weighing a thing. These two dominance cases are exactly the ones that carry the moral weight, which is the precise and defensible sense in which the framework can “rest on the direction”: the verdicts that matter most fall out of dominance, with no number anywhere in sight.
The remaining quadrant — coherence down while context widens — is the subtle one, and it is a mistake to file it under regress. Widening a context almost always shatters the coherence built for the narrower one: admit a disruptive truth, take in a perspective the old model had no room for, and consistency drops while the new material is still raw. A child meeting a fact that breaks its simple picture, a community admitting people it had shut out — both lose coherence in the moment, and neither is regressing. That temporary decoherence is not the counter-dynamic; it is the friction of integration, the necessary middle of a widening move — the same dip the tree of agreement shows on its way to a deeper re-convergence. What settles the verdict is what the decoherence is on the way to: a system metabolizing the disruption toward a wider coherence is growing, while one that cannot — that fragments, or flees back into the narrow safety of the counter-dynamic — is not. The momentary coherence reading does not tell those two apart, which is why the framework reads direction and trajectory rather than a snapshot.
What dominance also cannot decide is the case where the two dimensions pull against each other — coherence up a little, context down a lot, or two goods that trade against one another. There the rectangle is taller but narrower, and no amount of looking tells you which is “larger” until you have said what you are measuring for. Those cases the framework owes a different kind of answer, and much of the rest of this essay is about being honest concerning which cases they are.
Why refusing the number is principled
To decline the cardinal metric is not to wave a hand. There are three reasons the refusal is principled, and they are best given together, because each answers a different suspicion.
The first is that the specialists have already looked, and the number is not there to be had. There is a developed formal literature on probabilistic measures of coherence — Shogenji, Olsson, Fitelson, Bovens and Hartmann among them — and what it has produced is less a winning measure than a row of impossibility and non-uniqueness results. Bovens and Hartmann make the point in the sharpest possible form: coherence cannot rest on a single measure at all, but only on a vector of components that are weakly, never strongly, separable — there is no one scale to reduce them to; and where one insists on a single coherence ranking over information sets of any size, that separability provably fails. The belief-individuation problem compounds it, logically equivalent sets receiving wildly different coherence scores depending on how their propositions happen to be carved; and a further result shows probabilistic coherence is not even truth-conducive unless the sources are independent and credible — an external, context-like condition. The framework is not failing to supply a number that careful work would have found; it is declining to supply one that careful work has shown is not there. The specialists, hunting a measure of internal coherence, kept rediscovering the framework’s own claims: that coherence has no single global scale, and that internal coherence alone is not enough.
The second reason is the strongest, and it is not impracticality but misfire. Suppose a single internal-coherence number could be defined. What would score highest on it? A perfectly sealed, doctrinaire system — one that has reached flawless internal consistency by the simple method of excluding everything that does not fit. A one-dimensional coherence metric crowns the counter-dynamic. This is Goodhart’s law arriving exactly on schedule: a measure made into a target stops being a good measure, and a coherence number made into the moral target would reward the very narrowing the framework exists to condemn. So the refusal is not a confession of vagueness; the number is declined because the thing demanded would invert the theory.
The third reason is that an incomplete ordering is a respectable result, not a failed attempt at a complete one. Amartya Sen has long defended maximizing over optimizing — choosing an option no worse than the alternatives — precisely so that evaluation survives where a complete ranking is unavailable; a partial order, on his account, is often the truth of the matter rather than a way station toward something better. Ruth Chang’s work on parity shows that two items can be genuinely comparable without sharing a cardinal scale, so that incommensurability does not collapse into incomparability. (Parity itself — a fourth relation standing beside better, worse, and equal — is contested, and the framework need not enlist in that quarrel: what it borrows is only the weaker claim the dispute leaves standing, that comparability can outrun cardinal measurability, and Sen’s partial orders already secure that much on their own.) “Comparable, partially ordered, sometimes incommensurable” is not a corner the framework was driven into; it is a position serious people hold on the merits.
Definable, just not on one global scale
A harder version of the objection says the trouble runs deeper than measurement: that coherence cannot even be defined precisely enough to compare. That charge is answerable, and the answer keeps the framework from claiming more than it needs. Paul Thagard has given coherence a precise computational form as constraint satisfaction — a set of elements with positive and negative constraints among them, where coherence is the degree to which those constraints can be jointly satisfied, made concrete in working models of explanatory and even ethical coherence. The lesson is exact, and it is worth being candid about how far it goes. Thagard’s models genuinely compute: they assign numerical weights to the positive and negative constraints and settle on a coherence value for a bounded network of propositions. That is not an embarrassment to the argument; it is the argument. Local, bounded coherence is numerically tractable, and the framework is glad to use it where it applies. What those weights cannot do is scale — there is no non-arbitrary way to extend a local constraint network into a single, context-independent calculus over everything that matters, and the impossibility results above are the record of that failure. Bovens and Hartmann make the same point from the probabilistic side: one cannot even define a measure of coherence without first fixing the role it is to play — the coherence that makes a law firm efficient is not the coherence that keeps an ant-hill alive — which is exactly why a single role-independent scale was never on offer, and why their own construction yields a partial ordering rather than a number. The number is real in the small and a mirage in the large. Local definability without global commensurability is exactly the combination the framework needs, and the formal work supplies it.
What the order is built from
None of this rescues the partial order from the charge of hand-waving unless the local comparisons have content — unless one can say what makes a change count, in a given case, as more coherent or wider in context. They do have content, and it is the same value-free content the relocation of “ought” relied on. A change raises coherence when it absorbs more of what pressed against the old model — more anomalies, more inconvenient facts — without contradiction. It widens context when it takes in more of reality and more of the affected perspectives. And it warrants the description at all only if the resulting model stays viable under the consequences of acting on it. Anomalies absorbed, perspectives admitted, viability under feedback: these are the local judgments the partial order is assembled from, and not one of them needs a number or smuggles in a value word.
The trade-offs, owned
Everything to this point has been the easy ninety percent: the dominance cases, where the order is determinate, are also the cases that carry the moral weight. The hard tenth is where coherence and context pull against each other, or where two goods are genuinely incommensurable — and here the framework’s integrity depends on not faking an answer.
Before that, the demand itself deserves a harder look, because it smuggles in an ontology. To ask for a number weighing “two goods” against each other presupposes that the two goods are there to be weighed — discrete, pre-individuated, commensurable quantities, set out like masses on a scale. Perspectival reality does not hand us goods in that form. What presents as a clean collision between two fixed goods is very often an artifact of a narrow framing; widen the context and the supposed units re-individuate — one resolves into several, two fold into a larger one, a third path neither party had seen comes into view. So the first answer to “which good wins?” is frequently not to compute the trade but to question whether the goods are the separate, settled things the question assumes — and the real work, far more often than the metric-demand allows, is agents achieving enough coherence over their shared context to re-form the goods, not to score them as given.
Some collisions, though, survive every widening — and that is the ground Isaiah Berlin spent a career defending. Goods can be genuinely plural and incommensurable; liberty and equality, mercy and justice, the claims of the one and of the many do not reduce to a single scale, and some of their collisions are not puzzles awaiting a clever solution but tragedies — real losses, whichever way one chooses. Berlin’s further warning is the one to take most to heart: the systems that have promised to dissolve every such conflict have, again and again, become the systems that suppress the conflicts they cannot solve, because a theory committed to a single harmonious answer must treat a stubborn plurality as an error to be corrected. A framework that manufactured precise verdicts for genuinely incommensurable trade-offs would be that theory. The honest move is to refuse.
So where the dimensions trade, the framework gives a procedure rather than a verdict: widen the context, integrate the perspectives, seek the agreement the tree describes — and accept that the procedure may return not a single answer but a smaller field of defensible ones. Where two parties hold mutually exclusive needs, “integration” does not mean dissolving them into one; it means the move up a level — a coherent pluralism in which both persist as distinct, viable nodes rather than one being absorbed or silenced. And where even that is unavailable, the counter-dynamic still discriminates: among the resolutions on offer, the one that narrows least — that suppresses the fewest perspectives, seals off the least — is the better, even when none is clean. What the framework will not do is pretend the residue away. Some conflicts are tragic; saying so is not a hole in the theory but the thing that keeps the theory from becoming the kind that paves over tragedy by force.
A compass, not a calculator
For practice, the partial order has to be usable by someone in the middle of a hard case, with no scale to consult. What the framework offers there is not scores but directional markers — observable signs of which way the arrow points.
The marks of narrowing, the counter-dynamic showing itself, are concrete: disconfirming evidence rejected rather than absorbed; affected parties left out of the reckoning; rising rigidity and dogmatism; dissent punished; the context sealed against whatever might disturb it. The marks of widening are their mirror: anomalies and inconvenient facts taken in; more of the affected included; the model kept revisable; the strongest contrary perspective sought out rather than avoided. None of these is a measurement, and together they are enough — they let an agent or a community tell, in a concrete case, which way the arrow points, which is exactly what the dominance result needs and no more. They are also what distinguishes a system in the painful middle of widening — losing coherence as it takes in more — from one that is fragmenting or fleeing into the counter-dynamic: the first is visibly integrating the disruption and staying revisable; the second is expelling it, or sealing against it. The instrument the framework hands you is a compass, not a calculator: it shows direction reliably, and declines to fake a magnitude.
The shape of the answer
Put together, the reply to “quantify it or it is meaningless” has a definite shape. Meet comparison and refuse the cardinal metric: coherence-over-context yields a partial order, not a number. Lead with dominance: the verdicts that matter — genuine progress, and the counter-dynamic — fall out of “both up” versus “coherence up, context down,” with nothing to measure. Give the three reasons a number is the wrong thing to want — the impossibility results, the Goodhart misfire, the legitimacy of incomplete orderings — so it is plain that a metric would be a defect, not a missing feature. Hand the genuine trade-offs to the process, and offer diagnostics rather than scores to the practitioner who has to act.
Stated that way, the objection loses its grip. The framework measures, but comparatively and locally; it decides the weighty cases by dominance; and a single coherence number is not a missing feature but a defect avoided — as a target it would fall to Goodhart and crown the counter-dynamic, and manufacturing precise verdicts for incommensurable goods is, as Berlin warned, the totalitarian move. The partial order, with incomparable elements at its edges, is not the framework failing to be a calculator; it is the framework declining the error a calculator would be.
Sources & further reading
This essay engages its literature directly rather than through the book’s per-chapter end notes.
Coherence measures and the impossibility results. Tomoji Shogenji, Erik Olsson, and Branden Fitelson on probabilistic coherence and its limits; Luc Bovens and Stephan Hartmann, Bayesian Epistemology (2003), “Why There Cannot Be a Single Probabilistic Measure of Coherence” (Erkenntnis, 2005), and “An Impossibility Result for Coherence Rankings” (2006) for the vector and non-separability results, with “Solving the Riddle of Coherence” (Mind, 2003) for coherence as a role-relative, confidence-boosting property yielding only a partial ordering. That the same program keeps building role-specific measures that track truth (e.g. Hartmann and Trpin, “Why Coherence Matters,” 2023) is not a counterweight but the point: coherence is measurable in the small, for a fixed role, and has no single global scale. See also the SEP entry on coherentist theories of epistemic justification.
Incommensurability and incomplete orderings. Amartya Sen on maximizing versus optimizing; Ruth Chang on parity; Isaiah Berlin on value pluralism and the reality of tragic conflict.
Coherence as constraint satisfaction. Paul Thagard’s explanatory and ethical-coherence models (ECHO and its kin).
On targets and measures. Goodhart’s law — a measure made a target ceases to be a good measure.
Within AoM. The Is–Ought Relocation (the value-free structural criterion these local judgments share); The Tree of Agreement (the procedure the trade-offs are handed to); and coherent pluralism (Part 2), where the “distinct viable nodes” answer to mutually exclusive needs is worked out in full.