Machine Testimony Working Paper Series · No. 1 · September 2026
Auditing a belief-forming rule against 19,668 respondents
Machine Testimony
This paper reports an audit of a system that forms provisional beliefs about individuals from partial evidence. The system was evaluated against a population whose correlation structure was not chosen by the author. Two defects were identified, both of which affected every belief the system would form.
The rule used to mine regularities across a population recovered the dataset's known five-factor structure at a rate of 0.185, against a chance rate of 0.184. The rule was selecting consequents according to their frequency in the population rather than according to any relation to the antecedent. Separately, the confidence assigned to a newly formed belief was a linear score rather than a probability, and scored worse than a procedure that ignores all evidence and predicts the population base rate.
Both were revised and re-measured. Recovery of the latent structure rose to 0.875, using 94 per cent fewer regularities that covered a larger number of claims than the unrevised rule. Assigned confidence reached parity with the base rate and did not exceed it; the paper is explicit that this constitutes the removal of a negative rather than a demonstrated positive. The computational behaviour of the revised system is reported, together with a correction to a result this programme published earlier.
Keywords: agent memory, calibration, association mining, Wilson interval, machine accountability, negative results.
The system examined here forms provisional beliefs about individuals. It records what a person is observed to hold, mines regularities across a population, and uses those regularities to form an expectation about an individual on a question that individual has not answered. Each expectation carries a confidence, and the system declines to form one where the available evidence is insufficient.
Prior to this study, the only evaluation harness available for that machinery generated its own population. Such a harness can establish that the mechanism executes, and that it recovers structure deliberately placed in front of it. It cannot establish that the rule recovers structure in people, because the structure was specified by the same author who specified the rule.
A rule that measures the wrong quantity will pass such an evaluation whenever the wrong quantity is correlated with the intended one in the constructed world. That is what had occurred, and it was not detected for as long as the only evaluation available was of the author's own construction.
The evaluation uses Open Psychometrics' Big Five item responses, collected around 2012 through an online personality test, with consent recorded at the time of collection and subsequently published for research use. The instrument comprises fifty items rated one to five. Of 19,719 responses collected, 19,668 remain after dropping respondents who answered fewer than twenty items.2
Two properties of the dataset make it suitable. First, the mapping to the system's semantics is close to exact: a response of four or five corresponds to holding a proposition, one or two to holding its negation, and three, the neutral response, to a silence. The system forms an expectation only into a silence, so the design supplies a held-out question directly. An answer the respondent gave is hidden, and the system is asked to recover it.
Second, the latent structure of the instrument is known. Fifty items are organised into five factors of ten items each, so a pair of distinct items drawn at random belongs to a common factor with probability 9/49, or 0.184. This provides a ground truth against which the mined regularities can be assessed, which a self-generated population cannot provide.
The rule as originally specified required a candidate regularity to clear a support floor and then to hold at a reliability rate: among respondents holding proposition P, at least sixty per cent must also hold proposition Q.
Assessed against the known structure, the regularities produced by that rule joined items of a common factor at a rate of 0.185, against a chance rate of 0.184. The rule recovered none of the structure it was intended to find.
The cause is apparent in the marginal distribution of the selected consequents. Mined consequents had a mean base rate of 0.76, against an overall item mean of 0.57. Where seventy-six per cent of the whole population holds Q, the condition that sixty per cent of the holders of P also hold Q is satisfied by the frequency of Q alone, and carries no information about P. The rule was therefore selecting consequents by frequency rather than by association.
The revised rule requires a candidate regularity to exceed the consequent's own base rate by a margin, and applies that test to the lower bound of the observed rate rather than to the rate itself.3 A raw rate treats three observations out of three as certainty; an interval lower bound does not. A pair supported by a small number of respondents must therefore be substantially cleaner than one supported by a large number in order to reach the same standing.
| Rule | Regularities | Within-factor | Consequent base rate | Claims covered | Marginal lift |
|---|---|---|---|---|---|
| Rate only | 1,269 | 0.199 | 0.76 | 120 | +0.040 |
| Bounded lift | 72 | 0.875 | 0.48 | 152 | +0.128 |
| Chance | — | 0.184 | — | — | — |
Within-factor is the proportion of mined regularities joining two items of a common factor. Claims covered is the number of held-out responses for which the system was willing to form an expectation. Marginal lift is the improvement over the base rate on those responses.
Recovery under the revised rule is 4.8 times the chance rate, and marginal lift is approximately three times its previous value. The revised rule achieves this while forming expectations about a larger number of claims than the unrevised rule, not a smaller one. Ninety-four per cent of the regularities are removed and coverage increases, which indicates that the removed regularities were firing frequently without contributing information.
With the mining rule revised, the same harness was applied to the second component of the machinery: the confidence assigned to a belief at the point it is formed.
That quantity was defined as a base constant plus fixed increments for each supporting verdict, bounded at both ends. It is not a probability, but it was consumed as one by downstream components, and the system's own calibration benchmark scored it as one.
| Estimator | Brier | Brier skill | Predicted | Observed |
|---|---|---|---|---|
| Linear increments, constant anchor | 0.3687 | −0.689 | — | — |
| Posterior mean, anchor estimated | 0.2674 | −0.176 | 0.45 | 0.65 |
| Posterior mean, borrowing penalised | 0.2281 | −0.003 | 0.58 | 0.66 |
Brier skill is computed against a reference forecast that predicts the population base rate on every case.4 Zero denotes parity with that reference; negative values denote worse than it.
A Brier skill of −0.003 denotes parity, not skill. The revised estimator is statistically indistinguishable from a procedure that ignores every record the system holds and predicts the population base rate on every case. The result is not evidence that the system's confidence carries information about which beliefs will subsequently hold. What has been removed is an estimator performing substantially worse than an uninformed reference, whose output was being interpreted as a probability throughout the system. Establishing positive skill remains outstanding.
Three observations arose in the course of the revision. The first is that the anchor rather than the estimator accounted for most of the error. The base constant was 0.35, a value assumed about a population that had not been observed; expectations in this population held 68 per cent of the time. Replacing the constant with the system's own measured rate improved skill by approximately a factor of five, whereas altering the functional form of the estimator had little effect. The house rate is now estimated from the record, and is shrunk toward the former constant until sufficient observations accumulate.
The second is that a measurement was confounded by a design choice. An initial parameter sweep indicated that a very small pseudo-count performed best. It performed best because assigned confidence was bounded below the rate these expectations achieve, so the sweep rewarded whichever setting reached the bound most quickly. An internal guard identified the consequence, namely that a single success would move a generator to the ceiling, and the parameter was set on principle rather than on the sweep.
The third is that two independent safeguards were compounding into a fixed error. Every expectation formed by a young installation is borrowed, since a small number of individuals cannot support a mined regularity. A borrowed expectation was bounded at the ceiling multiplied by a discount, or 0.45, while borrowed expectations were holding 65 per cent of the time. This is not conservatism but a twenty-point error that no quantity of subsequent evidence could correct, since the bound applied to the output. Borrowing now raises the evidential threshold instead: a borrowed regularity requires more of its own record before its confidence departs from the house rate.
A rule that is correct but cannot be executed at scale is not usable. The revised system was measured against synthetic populations at sizes exceeding those any deployment has reached.5
| Subjects | Seconds | Pairs | µs per pair |
|---|---|---|---|
| 200 | 0.356 | 6,320 | 56.4 |
| 1,000 | 0.315 | 6,320 | 49.9 |
| 4,000 | 0.505 | 6,320 | 79.9 |
| 10,000 | 1.076 | 6,320 | 170.2 |
| 20,000 | 2.371 | 6,320 | 375.2 |
| Propositions | Seconds | Pairs | µs per pair |
|---|---|---|---|
| 40 | 0.223 | 1,560 | 143.1 |
| 80 | 0.601 | 6,320 | 95.1 |
| 160 | 1.769 | 25,440 | 69.5 |
| 320 | 5.982 | 101,442 | 59.0 |
Cost per pair rises with population size, since intersecting two bitmasks costs one machine word per sixty-four subjects. It does not rise with vocabulary size, which a cubic loop would.
Two regressions were introduced alongside the revision and were detected by the same measurements. A walk over the vocabulary had been placed inside the pair loop, converting a loop quadratic in vocabulary by design into a cubic one. Its signature was observable before its cause: cost per pair rose with population where it should have been constant. Separately, subject membership was held in ordinary sets, whose intersection is linear in the population. Bitmask representation measured approximately thirty times faster at one thousand subjects and ninety times at fifty thousand.
The more consequential defect was in the pass that forms new beliefs, and it is not observable on a new installation, where a cap halts the run after twenty-five hypotheses. It appears in the steady state that an established installation occupies on every subsequent pass: most propositions are already determined, few are eligible, the cap does not engage, and the loops run to completion.
| Entities | Before (s) | After (s) | Store scans, before | Assertion reads, before |
|---|---|---|---|---|
| 400 | 5.165 | 0.199 | 1,200 | 2,863,200 |
| 2,000 | 47.585 | 13.68 | — | — |
| 8,000, sparse | — | 46.75 | — | — |
Store scans fell from 1,200 to 1, and assertion reads from 2,863,200 to 2,386, for a pass that in this state produces no output. The dense case remains quadratic, since where every entity shares a feature with every other there is nothing to skip.
The regression guards written against both defects assert counts rather than elapsed time: one store scan per pass, and at most one vocabulary consultation per proposition. A timing assertion on a shared runner produces false alarms. Each guard was verified by reintroducing the corresponding regression and confirming that it failed.
On the day the first defect was identified, this programme's commons benchmark had reported that pooling regularities across installations fails in a saturated population, that is, one in which almost all members already hold the claim at issue. The reported measurement was −0.116. The measurement was correct and the explanation offered for it was not: the cause was the defect described in Section 3, not a property of pooling.
Under the revised rule that case measures +0.264, and the harness's negative control continues to behave as specified. The claim is withdrawn here, in the reference implementation's claims ledger, and in the benchmark that issued it.
A second and smaller correction belongs with the first. One of the test suites covering pooled regularities was passing without exercising the system: it recomputed the former formula inline rather than invoking the engine, and would have continued to pass after the engine's behaviour changed. It has been rewritten to invoke the engine.
All figures reported here are produced by harnesses in the reference implementation. The external dataset is retrieved at run time and is not contained in the repository.
python3 benchmarks/external/run.py Sections 3 and 4 python3 benchmarks/scale/run.py Section 5 python3 server/tests_prior_lift.py the revised rule python3 server/tests_scale_mining.py the mining guard python3 server/tests_scale_leap.py the belief-forming guard
The full suite comprises 97 passing files across 2,721 assertions. The revisions described here are recorded in the reference implementation's history together with the measurements that prompted each of them.
Both defects affected what every installation would learn and what the shared corpus would contain. They were identified because the system was evaluated against a population it had not constructed, and before any installation existed to be affected, which is why this paper reports a revision rather than a recall. A programme that publishes a format for holding automated systems to account has no standing unless it applies the same treatment to its own reference implementation, including where the measurement is unfavourable.