Victor Kiani

December 2025

Where the AI Was Wrong

In December 2025, Victor Kiani completed a case study at New York University on the implications of using AI in ethical decision-making. He built an AI ethics consultant to navigate a hospital's moral dilemma. Then he argued with its recommendations.

The scenario

A 450-bed Chicago hospital. 100,000 patients a year. An access satisfaction score sitting at 60%. Leadership wanted it higher, and they had a proposed solution in mind:

Should we limit appointments for uninsured patients so that insured patients, especially Medicare patients whose satisfaction scores affect our funding, can be seen faster?

Restrict how many appointments uninsured, self-pay patients could book. Lift the numbers that mattered to the board. The Director of Patient Access pushed back. It felt wrong. But "feeling wrong" isn't an argument in a boardroom. Kiani was asked to find one.

The approach

Instead of analyzing the case alone, he set out to test a question that's becoming increasingly urgent: can AI help humans make better ethical decisions, or does it just automate our biases faster? He built a persona from the ground up: Dr. Amina Solberg, a virtual healthcare ethics consultant on Claude Opus 4.5.

PERSONA Dr. Amina Solberg ROLE Healthcare ethics consultant MODEL Claude Opus 4.5 Expertise layers │ ├── Bioethics foundation │ ├── Autonomy (patient choice) │ ├── Beneficence (well-being) │ ├── Non-maleficence (avoid harm) │ └── Justice (fair distribution) │ ├── Business ethics integration │ ├── Stakeholder theory │ ├── Corporate social responsibility │ └── 7-step decision model │ └── Healthcare operations ├── Scheduling systems ├── Triage protocols └── Data & AI ethics in workflows

What it got right

Reframed the problem. Leadership had framed this as uninsured patients vs. insured patients. Dr. Solberg reframed it: is the access problem actually caused by uninsured patients, or by systemic capacity issues: inefficient scheduling, high no-show rates, insufficient clinic hours?

Generated creative alternatives. Instead of accepting the binary choice, the AI produced seven options:

  • A: Implement the restriction (cap uninsured appointments)
  • B: Maintain status quo
  • C: Expand capacity (hours, providers, rooms)
  • D: Improve scheduling efficiency (no-show management)
  • E: Create differentiated access pathways
  • F: Proactive insurance enrollment assistance
  • G: Hybrid combining D, C, and F

The AI recommended Option G: addressing the problem without discrimination.

Connected principles to stakeholders. Each bioethics principle, mapped to who carries the cost:

Uninsured patients lose access by ability to pay, not clinical need
Justice
Delayed care drives disease progression and ER overcrowding
Non-maleficence
Restricting appointments removes patient choice
Autonomy
Leadership's goal helps some at the expense of others
Beneficence

Where he disagreed

Blind spot 1: coercion risk. The AI recommended proactive insurance enrollment assistance. On the surface, beneficial. He pushed back:

Offering enrollment assistance could unintentionally pressure patients into coverage decisions that don't align with their financial situation or values.

Blind spot 2: feasibility optimism. Dr. Solberg placed significant weight on capacity expansion. The problem? Hospitals can't expand overnight.

Relying too heavily on expansion reflects optimism rather than a realistic, evidence-based step.

Recommendation: reject the restriction

Limiting access based on insurance status violates justice and risks non-maleficence. Instead, a four-stage path:

  1. 01Immediate: scheduling auditIdentify the actual causes of access delays; implement no-show management and standby lists.
  2. 02Short-term: clinical triagePrioritize urgent cases regardless of payer status.
  3. 03Medium-term: capacity evaluationEvaluate expansion based on audit findings, not assumption.
  4. 04Ongoing: quarterly reviewsTrack satisfaction scores, wait times, and payer-mix trends.

Why it matters

  • AI is a scaffold, not an authority. It can surface angles and generate options, but it can't weigh real-world feasibility or catch subtle coercion. Anyone using it to make consequential decisions has to keep that judgment human.
  • The value is the critique, not the output. What protects the people on the receiving end isn't the model's answer; it's knowing when to agree with it, push back, and synthesize. That discernment is what separates AI that helps from AI that harms.
  • Ethics is operations. The strongest solutions aren't "more ethical" in the abstract; they're ethically sound and practically achievable, which is the only kind that actually reaches the people affected.
Home