Race and Bias in Medical Technology

Chilmark Research’s Juneteenth Edition

Key Takeaways

Racial bias in medical hardware and software continues to be a significant problem, even decades after the problem is identified. Technology like the spirometer, which was designed to measure lung function, dates to the pre-Civil War era and was an instrument of oppression used to demonstrate the inferiority of African populations or slaves. Despite awareness of the flawed science this technology is based on, these problems persist.

Hardware such as pulse oximeters have the potential to introduce bias into data collection as well as post-data entry with the potential for significant adverse public health outcomes. Green LEDs do not work as well on dark skin; if the error rate is not addressed, there is the potential for darker-skinned populations to receive more acute care later, possibly jeopardizing outcomes.

Medical bodies and academic researchers will need to deepen their work on rooting out methods that conflate race and biology where this is not warranted.  A substantial number of algorithms in use today are erroneously utilizing race-adjusted measures that have the potential to harm Black patients. The growth of responsible AI in healthcare will require that devices and data sources containing legacy forms of bias be addressed before training models on these data sources.

Introduction

In early June 2023, the medical newswires were covering a new study highlighting the fact that Black men were experiencing bias in medical software/hardware for lung function, ultimately receiving less care than non-Black populations. The study published in JAMA Open Network was conducted by researchers at the University of Pennsylvania and found that approximately 40% more African American men would have been diagnosed with lung problems had the software not been biased. Sounds shocking, but even a cursory view of the history of race and lung function in medicine reveals that the most shocking issue is that the problem persists after decades of studies highlighting racial bias in lung function in medicine.

The spirometer utilized for assessing lung function has a long history connected to racialized science of the 19^th century. Notably, it was even used by lawyers representing Dupont Chemical in the 1980s asbestos lawsuits, arguing for lower payouts to Black shipyard workers due to the “inferior lung function of Blacks.” Even Thomas Jefferson had a perspective on this issue with the following observation, “[there is] a difference of structure in the pulmonary apparatus” between slaves and White Americans. Still today, racially adjusted readings for spirometry are employed in much of the US and Europe. Race adjustment of lung function during COVID-19 may have contributed to poor outcomes of African Americans, according to some studies. In this case, the culprit is in how pulse oximeters are designed and utilized.

The bias found in green LED wearables and pulse oximeters may also have contributed to delayed care, hypoxemia and poor outcomes for darker skinned patients. The problem has been known for 30 years; yet little has changed. The pulse oximeter has been recognized as one of the most important technological advances in medicine for monitoring patients during anaesthesia, recovery, and intensive care. By the 1980s, the devices had achieved widespread adoption. Yet by 1990, one of the first studies to observe racial differences in readings was published.[1]  In 2005 and 2007, two studies found pulse oximeters over-estimated arterial oxygen levels in darker skin individuals.[2] More recent studies over a longer duration found Black patients at higher risk of occult hypoxemia due to pulse oximeters not working as well on darker skin.

Pulmonary function testing suffers from a different issue than blood oxygenation; here the issue is less an issue of flawed hardware and software, but rather an interpretive bias. For much of the history of American medicine—and supported by the early inventor of the spirometer’s racial bias—correction factors that assume a 10-15% less lung function in Black patients has been the norm. A 2022 study found that equations without race-based correction performed better than the other measures utilizing race-based corrections. This leads to underdiagnosis and undertreatment of Black patients.

The problem of racial bias exists at multiple levels—from devices, to software, to foundational studies—and their applications in clinical medicine. The flawed science developed over the decades has been deployed in areas beyond lung function, including the recent NFL attempt to justify lower legal settlements for traumatic brain injuries for Black players, on the obviously flawed assumption that Black players have lower cognitive function. In 2022, 646 Black players won a settlement to have their tests for dementia re-scored, after being race adjusted inappropriately by the NFL and not receiving financial payouts; they now stand to gain substantial sums for the injuries they incurred while playing in the NFL. The effects of racialized medicine can be felt across society.

Conclusion: How white supremacist-informed pseudo-science became a fact and what we can do to address it

Lundy Braun, professor of medical science and Africana Studies at Brown University has studied the history of the spirometer and the science of lung function. She charts how white supremacy-inflected beliefs about Blacks in the mid-19^th century informed the underlying scientific development of the science of lung function. This is an area where ‘progress’ failed to rid medical practice of flawed science into our present.  For example, the National Health and Nutrition Examination Survey (NHANES III), she cautions, needs to be viewed with caution for use in studies due to the standardization process for signifying race and ethnicity. The standardization process has continued the tendency to conflate race and biology in problematic ways in census data as well. The shortcomings she highlights get worked into health studies if researchers are not careful. One outcome is the use of race adjustment as a quick fix which ultimately ends up as bad science. We have seen reversals of this practice in the past 6-7 months, as both National Kidney Foundation and the Maternal-Fetal Medicine Units Network have begun addressing the issue for algorithms relevant to their work.

Addressing this problem requires solutions at multiple levels:

• Hardware development
• Software and AI model design that integrates appropriate uses of biology and race/ethnicity where relevant, but removes race-adjusted calculations where not justified
• Review of surveys and large national data collection efforts where framings of race, ethnicity, and sexuality may rest on outdated social constructions
• Education of data scientists and bioinformatics professionals on the inherent bias in some data sources
• Greater involvement of medical societies in analyzing clinical decision support and other tools for bias based on historically and scientifically flawed measures

The differences in lung function have far more to do with social determinants of health, such as a patient’s environment. A better approach to understanding racial disparities in health comes through analysis of the social context’s influence on respiratory physiology. The growing number of insights from studies of bias in medicine and algorithms is highlighting additional challenges beyond technology alone. This is an area where government could devote more resources to interdisciplinary study of racial (and other) forms of bias in algorithms. As the adoption of AI accelerates, it is of utmost concern that health researchers have more tools and information to address the problem in study designs.

The American Heart Association, The Society of Thoracic Surgeons, and other medical bodies should conduct reviews of algorithms used by their members where race and biology may be conflated and erroneously introduce race adjustments that lead to bias. More academic work, focused on the methodologies of national survey instruments, could contribute to insights on how data scientists constructing models that utilize these surveys should adjust models for bias in the original data collection.

There are non-optical sensors in development for monitoring cardiopulmonary function. More investment is needed to bring these sensors to market so that the data collection at the bedside is less biased for patients with darker skin. In the meantime, algorithmic adjustments could possibly be developed to adjust for the error rate in the optical sensors used currently.

[1] See https://www.annualreviews.org/doi/abs/10.1146/annurev-med-043021-024004

[2] Ibid.