# Catch a clinician's errors during the visit, before the patient leaves

Penda Health's AI Consult checks clinicians' notes in the background at its Nairobi clinics. In a study of 39,849 visits, clinicians using it made 16% fewer diagnostic errors.

Organisation: Penda Health, with OpenAI
Status: Deployed in Penda Health's clinics in Nairobi; a study of 39,849 visits across 15 clinics ran from January to April 2025 and was published in July 2025
Place: Kenya
Categories: Health & care, Work & productivity
Built with: ChatGPT / OpenAI models
Try it with: Claude, ChatGPT / OpenAI models, Gemini, Grok
Time to try: 30 minutes

## Why it matters

Penda Health has run primary care clinics in Nairobi since 2012. Its AI Consult tool reads the clinician's notes during each visit and raises an alert only when it sees a possible error, while the clinician keeps every decision. In a 2025 study across 15 clinics and 39,849 visits, published by Penda Health and OpenAI, clinicians with the tool made 16% fewer diagnostic errors and 13% fewer treatment errors than colleagues without it. The errors were judged by 108 independent physicians, 29 of them from Kenya, who rated 5,666 randomly selected visits.

## How it works

AI Consult is built into Penda Health's electronic medical record. At key points in a visit, the clinician's notes are sent without patient identifiers to an OpenAI model (GPT-4o from August 2024), which checks them for errors. The reply comes back as a traffic light. Green is a tick: no concerns. Yellow is a bell the clinician can open if they choose. Red is a pop-up for a safety-critical issue, and the clinician has to read it before going on. The clinician always makes the final decision. For the study, Penda gave the tool to a randomly selected half of its clinicians as part of its quality improvement work. At first many red alerts went unresolved in both groups. Once the tool was actively rolled out, the share of visits left with a red alert fell to about 20% in the AI group, while the other group stayed near 40%.

## Who it's for

Clinic networks, hospital groups and health ministries that want a second check on routine primary care, and teams building clinical decision support.

## The prompt

```
You are helping me design a background safety check for clinicians in primary care: a tool that reads a visit's notes at set points and flags possible errors, while the clinician keeps every decision. Do not ask me for any real patient's name, record or identifiers, and do not give medical advice about a real case.

Our setting: [for example, 10 urban clinics where one clinical officer sees each patient from history to prescription]
Our clinical guidelines: [for example, the national primary care guidelines and our formulary]
Our record system: [for example, which fields the notes contain and when they are saved]

1. Propose the points in a visit where the check should run, and what part of the notes it reads at each point.
2. Define three alert levels (no concern, worth a look, safety-critical), with ten example situations for each, drawn only from the guidelines I give you.
3. Write the wording for each alert level in plain language a busy clinician can act on in under ten seconds.
4. List the rules for removing patient identifiers before any text leaves our system.
5. Design a quality-improvement study: how to give the tool to half of our clinicians, how independent doctors should rate a sample of visits for history, investigation, diagnosis and treatment errors, and what to track weekly (for example, the share of visits where a safety-critical alert was left unresolved).
6. List the risks to watch, including alert fatigue and wrong alerts, and how a clinician reports one.

Leave placeholders rather than invent guidelines, drugs or thresholds, and mark everything a senior clinician must sign off before use.
```

## Caveats

The study was a quality improvement project run by Penda Health, not a randomised trial of patient outcomes, and OpenAI provided financial support for it and consulted on the tool's development. The error figures are relative reductions, judged by doctors reviewing the notes of a sample of visits. At a follow-up call eight days after the visit, 3.8% of patients in the AI group and 4.3% of the others said they were not feeling better, a difference that was not statistically significant. There were 7 safety reports in the AI group and 5 in the other group, and OpenAI says AI Consult recommendations led to harm in none of them. PATH announced in March 2025 a randomised trial with 9,000 participants of a language model-based decision support tool at Penda Health; no published results were found when this card was checked.

## Sources

The OpenAI page and the study paper come from the two organisations that built and studied the tool, and OpenAI funded the study. PATH, a global health nonprofit, is independent of OpenAI and is running its own randomised trial at Penda Health, but has not published results. No independent evaluation of the study's findings was found.

1. OpenAI: "a 16% relative reduction in diagnostic errors"; "Penda's copilot used GPT‑4o from August 2024". Pioneering an AI clinical copilot with Penda Health (https://openai.com/index/ai-clinical-copilot-penda-health)
2. arXiv (Korom, Kiptinness, Adan and others): "16% fewer diagnostic errors and 13% fewer treatment errors"; "39,849 patient visits". AI-based Clinical Decision Support for Primary Care: A Real-World Study (https://arxiv.org/abs/2507.16947)
3. PATH: "providing guidance throughout the clinical encounter"; 9,000 participants. PATH launches clinical trial on the use of artificial intelligence in primary health care (https://www.path.org/our-impact/media-center/path-launches-artifical-intelligence-clinical-trial/)

Fact-checked: 2 Nov 2026

Page: https://escapevelocitysociety.com/picks/penda-health-ai-consult-clinical-safety-net

See Selected Projects: https://escapevelocitysociety.com/picks.md
