“We set out to create a custom AI predictive model that could be evaluated, tracked, improved and implemented to pave the way for more inclusive and effective population health strategies.”—Reshma Gupta

Creating the BE-FAIR model

To create the system-wide BE-FAIR model, UC Davis Health brought together a team of experts from the health system’s population health, information technology and equity teams.

Over a two-year period, the team created a nine-step framework that provided care managers with predicted probabilities of potential future hospitalizations or emergency department visits for individual patients.

Patients above a threshold percentile of risk were identified, and, with primary care clinician guidance, determined if they could benefit from program enrollment. If appropriate, staff proactively contacted patients, provided needs assessments and began pre-defined care management workflows.

Responsible use of AI

After a 12-month period, the team evaluated the model’s performance. They found the predictive model underpredicted the probability of hospitalizations and emergency department visits for African American and Hispanic groups. The team identified the ideal threshold percentile to reduce this underprediction by evaluating predictive model calibration.

“As healthcare providers we are responsible for ensuring our practices are most effective and help as many patients as possible,” said Gupta. “By analyzing our model and making small adjustments to improve our data collection, we were able to implement more effective population health strategies.”

Studies have shown that systematic evaluation of AI models by health systems is necessary to determine the value for the patient populations they serve. 

“The BE-FAIR framework ensures that equity is embedded at every stage to prevent predictive models from reinforcing health disparities.”—Hendry Ton

“AI models should not only help us to use our resources efficiently — they can also help us to be more just,” added Hendry Ton, associate vice chancellor for health equity, diversity, and inclusion. “The BE-FAIR framework ensures that equity is embedded at every stage to prevent predictive models from reinforcing health disparities.”

Sharing the framework

The use of AI systems has been adopted by health care organizations across the United States to optimize patient care.
About 65% of hospitals use AI predictive models created by electronic health record software developers or third-party vendors, according to data from the 2023 American Hospital Association Annual Survey Information Technology Supplement.

“It is well known that AI models perform as well as the data you put in it — if you are taking a model that was not built for your specific patient population, some people are going to be missed.”—Jason Adams

“It is well known that AI models perform as well as the data you put in it — if you are taking a model that was not built for your specific patient population, some people are going to be missed,” explained Jason Adams, director of data and analytics strategy. “Unfortunately, not all health systems have the personnel to create their own custom population health AI predictive model, so we created a framework healthcare leaders can use to walk through and develop their own.”

The nine step framework of the BE-FAIR model is outlined here.