Evidence and impact

Measure the workflow, not the promise.

Before a SIRV workflow is deployed, we agree what success should look like, test it against representative operational work and retain the evidence needed to explain the result.

See a measured example
How measurement works ↓

Measurement approach

A claim should be traceable back to the work.

The useful question is not simply whether AI produced a good answer. It is whether the complete workflow performed better, including evidence, exceptions, human review and the final operational outcome.

Define the task

Agree the workflow, source material, expected output and who makes the final decision.

Set a baseline

Establish the current process and the measures that matter before judging improvement.

Test the workflow

Use representative operational tasks and retain evidence of matches, misses, exceptions and review.

State the result carefully

Separate observed results from estimates and define what the claim does, and does not, demonstrate.

Worked example · anonymised customer sprint

Unauthorised-access investigation

Access-control records were reconciled to identify cases for investigation. CCTV review and final verification remained with the human reviewer.

Identification results were observed during the sprint. Time comparisons are estimates using average durations, not independently verified production benchmarks.

+20%
More valid cases identified

Four additional valid cases were found compared with the original manual process.

6h → 20m
Estimated processing effort

Approximately 94% lower, including human CCTV review and final verification.

30d → 20m
Estimated elapsed time

Historical end-to-end duration compared with the proposed workflow.

A successful demonstration is useful evidence. It is not, by itself, proof of operational reliability at scale.

Compared with: The organisation's existing manual identification process.
Observed: The workflow reproduced the repeated-swipe cases previously identified manually and surfaced four additional valid cases confirmed by operational reviewers.
Human role: CCTV was exported for human review. Final verification remained with the reviewer.
Claim boundary: The sprint demonstrates performance on the tested records. Wider deployment requires continued evaluation, monitoring and review.

What gets measured

Different workflows need different evidence.

Speed can matter, but it is rarely enough. The measure should reflect the operational purpose and the consequence of error.

Workflow	Useful measures	Human control point
Procedure and knowledge access	Retrieval accuracy, source coverage, time to usable answer, appropriate refusal when evidence is absent	User interprets guidance and makes the operational decision
Document evaluation	Agreement with competent reviewers, missed issues, false positives, review time, evidence traceability	Competent person accepts, rejects or challenges findings
Intake triage and routing	Correct prioritisation, missed high-priority items, consistency, escalation quality, time to review	Uncertain or consequential cases are escalated
Operational intelligence and investigation	Valid findings, missed evidence, time to operational picture, source traceability, quality of hypotheses	Investigator tests interpretation and verifies findings
Actions, follow-up and retained learning	Action completion, ownership clarity, recurrence, retrieval of prior learning, auditability	Named owners retain accountability for action and closure

Claim discipline

Evidence should survive buyer scrutiny.

Clear

Say exactly what changed.

Define the workflow, comparison and measure. Avoid percentages that have no understandable baseline.

Proportionate

Match assurance to consequence.

A low-risk productivity task and a consequential operational decision should not need the same evaluation threshold.

Contextual

Do not generalise beyond the evidence.

Separate customer sprint findings, estimates and established deployment outcomes so the reader knows what each number represents.

From sprint to deployment

The measurement continues after the demo.

For consequential workflows, deployment should be based on evidence from representative tasks, not a handful of impressive examples.

For further detail on information protection, access, accountability and review, see SIRV security, privacy and ISO 27001 alignment.

Evaluation can include agreement with reviewed reference outcomes, identification of missing or conflicting evidence, consistency across similar cases, correct escalation, staff effort and successful completion of the wider task.

Before wider deployment

Ask whether the workflow performs reliably enough for its consequence, whether evidence remains reviewable, how exceptions are handled, and where human judgement remains necessary.

Buyer questions

How impact claims should be interpreted.

Does SIRV guarantee the same result in every organisation?

No. Outcomes depend on the workflow, data, operating environment and way the workflow is deployed. This is why SIRV defines measures for the specific task and states results in context.

Can we agree our own success measures before a sprint?

Yes. The preferred approach is to agree the workflow, baseline, expected outputs, review points and success measures before testing begins.

Does SIRV measure more than time saved?

Yes. Relevant measures can include accuracy, consistency, missed issues, evidence traceability, correct escalation, effort, elapsed time and successful completion of the operational task.

How does SIRV treat estimated results?

Estimated measures should be labelled as estimates and the basis of comparison should be stated. Observed findings and production benchmarks should not be presented as if they are the same type of evidence.

Can this evidence support procurement and assurance review?

Yes. The purpose of the approach is to make the workflow, test conditions, controls, results and claim boundaries understandable enough to be reviewed and challenged.

Start with one bounded workflow