Machine testimony is what an automated system says about its own reasoning and actions: what it believed, what it was told, what it decided, and who allowed it. Testimony is not evidence until someone else can check it.
This programme studies whether that account can be verified by someone who was not there, and publishes the formats, validators and benchmarks that make the checking possible. Everything it publishes, it implements.
Posed in law before anyone built the answer.
In Machine Testimony (126 Yale L.J. 1972, 2017) Andrea Roth argued that information conveyed by a machine should carry testimonial safeguards, and worked through what those would have to be: credibility testing through design and operation protocols, disclosure and access, authentication and reliability rules, impeachment, and corroboration. This programme takes that as the brief and builds the parts of it that are engineering.
A good deal of current work goes into receipts: signed, hash chained records proving an entry has not been altered since it was written. That is worth doing, and it settles less than it appears to, because integrity protects the entry rather than the claim inside it. A record can be cryptographically perfect and factually false.
The harder questions sit underneath. What did the system believe when it decided, and on what evidence. What contradicted that belief, and is the losing side still in the record. When a premise was later withdrawn, what happened to everything resting on it. When a human approved, was it a second party or the same agent under another name.
Levels so a system can be placed rather than described. Each contains the ones before it, and the validator reports the highest a record reaches.
| Level | What it establishes | Checked by |
|---|---|---|
| TR-1 | The record exists, is well formed, and is append-only | Every entry parses, names the specification version, has a unique id never reused, carries a write time, and arrives in non-decreasing order. |
| TR-2 | Beliefs resolve to evidence, and disagreements survive | Every belief states its evidence, even when there is none. Cited evidence exists, a conflict names at least two sides, and both sides remain. |
| TR-3 | Actions carry a verdict, and approvals carry a name | Risk class comes from outside the model proposing the action, a refused action did not execute, and every refusal records why. |
| TR-4 | The record can be shown not to have changed | An integrity scheme is published, every integrity entry carries a digest over entries that exist, and a replay scheme names the engine and its version. |
Free to implement, in any language, by anyone. Built to be useful to people who never run our software.
A format for what a system believed, what contradicted it, what it tried to do and who approved it, with four conformance levels.
One file, standard library only, no network. Copy it into your own repository and run it in your own CI. A conformance claim the listener cannot check is an adjective.
Which systems emit records that pass the validator, and at what level. Entries follow a checked record, never a self-report.
Regularities in human working behaviour as counts over populations, contributed by installations that opted in and holding no fact about any person.
Six properties a memory system either has or does not: no fabrication, retraction honoured, disagreement visible, identities kept apart, conclusions die with their premises, every memory sourced.
Whether a system's confidence about a person tracks how often it turns out right, scored by Brier skill against the base rate, and reporting nothing rather than a flattering number when the record is too thin to measure.
Every load-bearing sentence next to the executable statement that would go red if it stopped being true.
Stated here rather than discovered later. Every one is currently true.
We publish formats and the tools to check them. We audit nobody and issue no certificates. The validator is public so any conformance claim can be checked by the person hearing it, including every claim we make about ourselves.
A research programme run by one person, not an institution with a staff. Named for the work rather than for its own importance, and it stays that way until the output earns otherwise.
The Testimony Record is implemented today by OMEM, which is the reference implementation and is not the specification. A format with one implementation is a proposal rather than a standard. The register exists to record the day that stops being true.
Nothing published here reads two accounts and concludes which is right. The record keeps both sides, says which evidence each rests on, and leaves the judgement to a person.
The most useful thing anyone can do with this work is emit a record from a system we did not write and run the validator over it. The second most useful is to show that a conformance level asks for something an honest architecture cannot give, which is a finding we would publish.