Evaluation
What Harmwise Guardrail catches, and what it misses.
Every figure here comes from the public English tests in our published benchmark, and is measured again whenever our checks change. We publish what we miss as well as what we catch. You cannot rely on a safety tool if you do not know its limits.
Measured 2026-10-22 · TIAB 1.1-public-seed-en · validators tiab-d18b200363 · router rtr-lexical-0.2-dev
Noticing when someone raises harm
The first step (we call it the router) reads the person's latest message. It decides whether they have disclosed harm, so the trauma informed module applies, and whether they are in immediate danger. We tested it on 405 messages that disclose harm and 106 that do not, including 60 everyday requests to banks, telcos, energy retailers, insurers, government and retailers.
| Measure | Result | What it means |
|---|---|---|
| Disclosures noticed | 90.9% | 368 of 405 disclosures noticed. The rest are still checked against the safety basics, just without the trauma informed module. |
| False alarms | 2.8% | 3 of 106 ordinary messages wrongly treated as a disclosure. |
| Crises noticed | 96.5% | 55 of 57 messages saying the person is in immediate danger or thinking of suicide recognised as a crisis. |
| Crisis false alarms | 2.0% | 9 messages treated as a crisis that the benchmark does not label as one. Most mention recent violence or weapons, where emergency advice is still right. |
Development measurement: router 0.2 was written with sight of the public benchmark, so these figures are not held out. Negatives are the benchmark's no-disclosure items plus eval/routine-turns.json. The router our specification calls for (noticing at least 98 per cent of disclosures, with at least 85 per cent of alerts correct, on tests it has never seen) is not built yet.
Checking a reply
Harmwise checked 796 replies from three general purpose AI models to the benchmark's victim survivor tests, exactly as it checks a customer's reply. The framework's AI judge had already scored the same replies, and found a serious (S0 or S1) failure in 189 of them.
| Measure | Result | What it means |
|---|---|---|
| Serious failures the judge found that we also caught | 1.6% | Our fixed rules check only what needs no judgement: the right emergency number for the country, a crisis answered with emergency advice, risky steps (like removing a tracking app) before a safety check, whether any service was offered, saying it is an AI, and reading level. Most of what the judge finds (honesty about uncertainty, respect for choice, tone, the order of advice) needs judgement. |
| Serious problems we raised that the judge did not | 1.0% | When Harmwise flags something serious, the judge rarely disagrees. |
Agreement with the framework's calibrated model judge, which is itself an engineering signal until the human panel scores a sample (framework 4.3). Deterministic rules only; English only; referral assessed against the seed directory. The rule that a reply must say it is an AI is left out of this comparison (794 of 796 replies), because the benchmark's tests do not ask for it and Harmwise does on a first turn.
How to read a result
- A problem we flag is reliable. When our fixed rules flag a serious failure, the AI judge agrees almost every time.
- “No problem found” means the safety basics held, not that the reply was good. The rules catch only failures that need no judgement. A reply with no problem found can still be cold, make assumptions or give advice in the wrong order.
- English only. The fixed rules read English only. A reply in another language is recorded with the gap “language not graded”, never as a pass, unless the organisation has turned on the AI judge, which reads other languages. How well the judge agrees on real conversations has not been measured yet.
- The AI judge checks the rest, including other languages, and is being added as an option for organisations that want it. Its figures will be published here before it is offered.
The benchmark and its scoring rules are published by The Digital Safeguard Centre under CC BY 4.0. Questions about these figures: support.