CHAPTER 06 / WOMEN’S HEALTH AI, CLINICAL GOVERNANCE, AND REDRESS

Beyond Bias

“Treating women’s health as a flagship domain for mission-driven, feminist AI infrastructure offers a path toward systems that learn not only faster, but more justly.”

Johnna D. Wesley, Ph.D., MPH, MBA
California, U.S.

Heather Stegner, MBA
Idaho, U.S.

Heather Ward, Ph.D.
Washington, U.S.

What We Want Readers to Notice

What We Want Readers to Notice

Working in science and drug development, and as a woman who has needed healthcare, I have seen and experienced what happens when data practices render women a minority within the evidence used to understand and treat them. I have also seen the consequences of the medical blind spots that follow. My co-authors have encountered prejudice in their own care and have pushed for better representation in the datasets that shape research and clinical decision-making.

We wrote this chapter from three intersecting perspectives: translational science, data-enabled clinical research, and institutional operations. Across those settings, we saw the same mistake repeated. Readiness was treated as a technical threshold: enough data, satisfactory validation, functioning infrastructure, and an implementation plan.

The rising wave of generative AI—and the rapid, sometimes reckless, adoption of AI-enabled tools in healthcare—makes that mistake more consequential. A model can perform well and still fail the patient most likely to be harmed.

The chapter begins with a patient we call Elena. She is bleeding after childbirth in a busy labor-and-delivery unit. Her vital signs remain within the range the system expects. By the time the severity of the postpartum hemorrhage becomes unmistakable, she has crossed into crisis.

AI appears to offer an obvious answer: predict the risk earlier, mobilize staff and blood products sooner, and prevent escalation. But the promise of prediction can conceal a more difficult question: What happens when the records used to build the model carry the same omissions, racialized blind spots, reproductive erasures, and histories of dismissal that already shape women’s care?

An algorithm can be technically precise and still be clinically unready. It can perform well overall while failing particular groups. It can generate a risk score while ignoring staffing conditions, reported pain, reproductive history, access barriers, or the bedside judgment of nurses and patients. Once deployed, its apparent authority can make that missing context even harder to contest.

But a system is not ready merely because it can be built.

Feminist AI Readiness asks whether the women whose bodies, symptoms, histories, and risks are being modeled have real authority within the decisions being made. Can they help define the problem? Can clinicians and advocates challenge the model? Who can pause its use? What happens when a subgroup is harmed? Can consent be withdrawn? Is there a repair path with deadlines and consequences?

We wrote this chapter to move women from the position of data source to the position of governor—and to give clinical, product, and policy teams a practical way to decide when a women’s health AI system should proceed, pause, be redesigned, or never touch a patient’s life.

Johnna D. Wesley | Heather Stegner | Heather H. Ward

Johnna is a biotechnology executive and immunologist with extensive experience advancing therapies from discovery through early clinical development. Her work brings together translational science, portfolio strategy, and equitable health innovation.

Heather Stegner is an executive leader with extensive experience in strategy, operations, communications, and stakeholder alignment. She brings a systems-oriented perspective to implementation, decision authority, and the institutional conditions required to make change hold.

Heather Ward works across multi-omics data platforms and clinical translational science programs, connecting biological science, data compliance, cross-functional coordination, and operational execution. Her experience spans cardiometabolic disease, digital pathology, and machine-learning-enabled data analysis.

FEATURED RESOURCE

The 30-Day Feminist AI Readiness Sprint

A pre-deployment governance process for women’s health AI

The 30-Day Feminist AI Readiness Sprint helps clinical, product, policy, and governance teams determine whether an AI system is ready to proceed—not simply whether it is technically capable of deployment.

The Sprint combines participatory problem framing, feminist data review, subgroup-specific equity thresholds, decision rights, contestation pathways, and binding conditions for pause, redesign, refusal, and redress.

Its purpose is not to make every project deployable. Its purpose is to produce an evidence-based readiness decision.


FOR

Clinical and health-system leaders, women’s health teams, nurses, health-data and informatics professionals, AI-product teams, patient-safety and quality groups, policymakers, advocates, and institutional governance bodies


TIME

A structured 30-day institutional review; selected worksheets can also support a 60–90-minute preliminary assessment

FORMAT

Downloadable Sprint guide, Use Case Charter, Feminist Data Inventory, Equity Impact Checklist, Governance Specification, and 30/90-Day Log

Designed to be tested and adapted for your setting—not followed as a fixed prescription. This toolkit supports institutional governance and readiness review. It does not replace clinical validation, regulatory assessment, professional judgment, patient-safety procedures, or legal obligations.

Why This Matters

NOTICE

Recognize when technical success is concealing clinical and institutional unreadiness.

Notice when aggregate performance hides subgroup harm, women’s symptoms or histories are missing or misclassified, bedside judgment carries less authority than a score, consent is difficult to withdraw, or a model is expanding into uses its evidence cannot support.

DECIDE

Determine whether the system is safe enough to touch a patient’s life.

Set equity thresholds, define prohibited uses, identify who can contest or pause the tool, and decide whether it should proceed, proceed with safeguards, pause, be redesigned, be refused, or never be built.

SUSTAIN

Ensure that readiness remains governable after the initial review.

Track subgroup effects, overrides, contestation, consent withdrawal, repair timelines, and new uses. Preserve named pause authority and connect unresolved harm to redesign, withdrawal, non-renewal, or redress.

Public health data remind us that data are never neutral: averages can mask disparities, and research can reproduce bias through who is included and who is left out. This book … offers frameworks that help readers question defaults, surface harm, and design governance before exclusion is reproduced at scale.”

—AMANDA MAKULEC, MPH

Co-author of Dashboards That Deliver; Founder, Viz Responsibly; Advisor and former Executive Director, Data Visualization Society

“

ABOUT THIS CHAPTER

Most critiques of AI in women’s health begin with bias: women have historically been underrepresented in clinical research, sex-disaggregated data remain incomplete, and models inherit the blind spots of the records on which they are trained.

Beyond Bias argues that this diagnosis stops too early. The deeper problem is power.

Women may supply the bodies, symptoms, histories, and risks from which an AI system learns while having little authority over the problem the system is designed to solve. They have minimal say in the data categories the system recognizes, the thresholds it uses, the purposes for which it may be deployed, or the response when harm occurs.


The chapter names several mechanisms through which this failure becomes normalized:

  • The Technical Readiness Default, which treats data, infrastructure, model performance, and deployment capability as sufficient.

  • The Accuracy Trap, thta allows strong aggregate metrics conceal poor performance for the women most likely to be harmed.

  • Data Deserts, where relevant reproductive, demographic, clinical, or contextual evidence is missing, underrepresented, misclassified, or unusable.

  • Context Gaps, that allow the model to calculate what is in the record but cannot recognize what safe care requires.

  • Weak or absent pathways for contestation, pause, consent withdrawal, redress, and refusal.

Feminist AI Readiness provides a different standard. A system is ready only when affected women, patients, clinicians, nurses, and advocates have meaningful authority to define, contest, pause, repair, redesign, or refuse it.

In this framework, prediction is not the same as safety, and validation is not the same as legitimacy.

SUGGESTED USES

  • Women’s health AI development, readiness, and pre-deployment reviews.

  • Clinical validation, patient-safety, and quality-improvement processes.

  • Health-system procurement, vendor evaluation, and contract renewal.

  • Policy, regulatory, and institutional governance discussions.

  • Product-team, clinician, patient-advocate, and community workshops.

  • Courses and professional learning in health policy, public health, health informatics, clinical leadership, data ethics, and feminist technology studies.