High-Confidence AI Decision Consulting

Not every AI questionneeds a roadmap.Some need a decision.

Make better decisions about the AI already in your business: what to expand, improve, renew, restrict or stop.

We measure your AI systems, workforce practices and business workflows, so every decision rests on evidence.

Discuss an AI decision →

agreement with the AI errors caught flagged, month 8
Behind this page: reviewers agreed with the AI more each month while the errors they caught fell from 70% to 15%. Simulated study, use case 1 →

Where we start

Two kinds of teams, the same question: is the AI still doing what you decided it should?

US health plans

AI already touches prior authorization

It recommends denials, drafts notices and routes reviews. Two states now limit AI in these decisions, and two more follow on January 1, 2027.

  • Is clinician review still catching AI errors?
  • Would we see a wrong rule in days, not a quarter?
  • How many denials would fail if challenged?

What measuring your AI does for the business

Spend stops leaking

You stop renewing AI systems the evidence doesn't support, and you stop cutting the ones that are working.

Work gets better

You see what the AI actually changes in turnaround time, capacity and rework, measured against how the work ran before.

Problems get caught early

You find the weak output or the missing check before a customer does.

We measure what changed and what the evidence supports. We never claim a result the evidence can't show.

What your AI decision depends on

Before you expand, renew, improve, restrict or stop an AI use case, we measure three things.

AI Risk

Risk in the workflow

What could go wrong, who could be affected, and how much risk remains, so you can decide whether it's acceptable.

AI Trustworthiness

Outputs you can check

Whether people can trace, check and correct the AI's outputs in the work that depends on them.

AI Savviness

People ready to use it well

Whether people recognize the AI's limits, verify important outputs, and know when to pause, override or escalate.

Together, these show whether the evidence supports your next decision, and what has to change if it doesn't.

Each month, every line you drew gets a state

The chance, from your own data, that the line meets its target.

Example, simulated

Holding
Over 95% chance it meets its target

No notice left without a person: 0 of 900 sampled.

Drifting
Between 5% and 95%

A denial rate edging toward its ceiling.

Broken
Under 5% chance it meets its target

Reviewers catching under half of AI errors.

Unmeasured
No data yet

Reported as unknown, never counted as a failure.

Tested on simulated data where the truth is known: a line that truly holds was never called Broken in 2,000 runs, and a truly broken line was called Broken in 99.9% of them.

See how a Decision Review works →

Decisions we measure for

Higher-scrutiny decisions

Why we measure

AI doesn't stand still. A vendor updates the model. A team changes the workflow. New people start using the tool. A pilot becomes part of everyday work.

A policy, a vendor's claim or a one-time review can't tell you whether the reasons behind your decision still hold. Trust is something you test against evidence, not something you declare.

We look at the AI in its real workflow: what it's used for, where people check or override it, and what you can show. When something important changes, we measure again, so you can decide what to expand, improve, renew, restrict or stop.

  1. Something changes
  2. Recheck what it affects
  3. Decide what to do next

What changed in AI this week, and which line it moves, is in the Weekly AI Debrief at the top right of every page.

Who runs the review

Suneeta Modekurty leads every Decision Review and runs the analyses herself. She is a bioinformatics data scientist and researcher who has spent ten years building AI and data science systems across several industries.

Her work follows one principle: an important AI decision should rest on evidence someone can examine, not on a vendor's claims. Her research in cancer and Alzheimer's disease includes a first-author abstract published in the Journal of Clinical Oncology (ASCO 2024 Annual Meeting), and she reviews manuscripts for PLOS ONE and Frontiers journals.

She is ISO/IEC 27701 and ISO/IEC 42001 Lead Auditor certified, has delivered live trainings on measuring AI on O'Reilly Media, and is the author of The AI-Human Synergy (2024).

Suneeta Modekurty, founder of METRIS, in a working session.

How an engagement runs

  1. Choose the decision. We define the AI system, the business workflow, the leader accountable for the decision, and the choice that needs to be made.Go on when a named leader and a data contact are in place
  2. Write the proof rules first. Before reviewing evidence, we agree what would support, qualify or block the decision.Go on when you sign the proof rules
  3. Record the starting point. We record how the workflow runs today, what evidence exists, and what cannot yet be observed.Go on when the starting records arrive
  4. Review the evidence. We examine agreed records, tests, outputs, procedures and how people use the system, and mark each material claim as supported, partly supported, unsupported, contradicted, or not observable.
  5. Decide and act. You receive the Decision Packet. Your accountable leader chooses a path: proceed, proceed with conditions, hold pending evidence, restrict or redesign, or stop. We help assign the actions that follow.The value test: a decision taken on the evidence
  6. Re-measure after change. When the model, vendor, workflow or people change, we recheck the evidence that change affects.

AI decisions should not outlive their evidence.

A vendor updates a model. A workflow changes. Usage expands. People adapt. Each material change is a reason to recheck the evidence affected by that change.

What if the evidence is missing?

We name the gap and the evidence needed to close it. Missing evidence never becomes a favorable finding.

What if our AI is outside your scope?

We tell you before the engagement starts, and explain what specialist work or referral you may need.

Who makes the final decision?

Your organization's accountable leader. METRIS™ supplies the evidence record and the decision support.

Proof you can rerun

Every number traces to the run that made it

Each figure comes from code with a fixed random seed. Anyone can run it again and get the same numbers. Clients see the code behind their own reports.

# Is clinician review still catching AI errors?
caught, missed = audit_window(month)   # counts only
p_holding = beta.sf(0.50, 1 + caught, 1 + missed)
state = "Holding" if p_holding > 0.95 else \
        "Broken"  if p_holding < 0.05 else "Drifting"
# seed 20261013 · numpy 2.5.3 · scipy 1.18.1

Have an AI decision coming up?

Discuss your AI decision →

Illustration. The evidence supporting an AI decision can change as the model, workflow and people around it change. Re-measurement shows whether the decision conditions still hold.

Two ways to start

Start with a free check, or bring us the AI decision you need to make.

Discuss an AI decision →

Built for leaders and teams deciding what to do with AI they already use in real work.

For clients and members

Client Community

Your own portal with your lines, reports and decisions, and a members' community. By invitation.

Sign in
METRIS / Decision window

Why it matters

What we measure

Example finding

Illustration only, not a real result.

Discuss this decision →