Interest in predictive analytics in healthcare is on the rise. How long will someone be in the hospital? What are their chances of being readmitted? Which treatment will work the best?
On the surface, these seem like straightforward questions, but it turns out that prediction is not a simple topic. Questions about what data and effort goes into the prediction, how accurate it is, who gets to see it, and how actionable it is can all make a huge difference in its value.
As we start to develop our major Market Trends Report for Analytics, scheduled for release in early 2019, I thought I would unpack these questions a bit.
How Much Data?
To start, it makes sense that the more data and the better data you can include in making a prediction, the more accurate it will be. However, you don’t always know what factors will matter the most, or at all.
If you over-collect data to ensure you factor in most everything, you unnecessarily raise the cost, time, and complexity, as data warehouses are not inexpensive and interoperability is still a struggle. If you under-collect, you might miss something important, and clinicians will continue to complain about the burden of documentation.
The recent focus on social determinants of health is promising, but it requires much more data collection and management, along with more sophisticated analytic models. Here, we welcome the contributions of AI and Machine Learning (ML) to help determine which variables to focus on.
Prediction is not a simple topic. Questions about what data and effort goes into the prediction, how accurate it is, who gets to see it, and how actionable it is can all make a huge difference in its value.
How Accurate?
Accuracy has all kinds of complications. You’ve likely come across the difference between precision and recall.
With precision, you measure how right you were that something was going to happen when it did, in relation to how often you thought it was but didn’t. With recall, you measure how often you thought something was going to happen, and you were right, but you missed some that did happen. In the first situation, false alarms could be costly, leading to additional tests and much worry. In the second situation, missing an important prediction (such as a cancer diagnosis) could also be costly.
Fortune tellers make many predictions and eventually get some stuff right, but most of us don’t take them seriously; Chicken Little got into trouble when he cried out too many times. Conversely, someone who makes too many conservative predictions might have a lot of credibility but also miss too many opportunities.
Can You Handle the Truth?
Consider who wants to know or gets to see the predictions. Most clinicians, healthcare workers, and leadership teams want do the best they can, but negative predictions about their approaches or performance may not be met with welcome arms. Exposing their problems can lead to potential unwanted oversight by others, reduced funding, etc.
In other words, expect pushback about the quality of the data, the accuracy of the prediction, the motivations of the prediction team, and who is really responsible for the bad trends. Think about how predictions are supported and presented. Also consider the flip side: Positive predictions, if not accurate, may lull an organization into a false sense of security.
Is It Actionable?
This leads to one of the most important questions: How actionable is the prediction, and what can be done about it? (This also brings prescriptive analytics into the discussion.) If the answer is “not much,” there may still be value in knowing it – but you have to brace for impact, as it could also lead to helplessness and negativity; consider the effect of telling a patient they have an incurable disease.
Better in most circumstances to focus on those predictions where positive action can be taken. Some actions might be obvious: You’re running out of medication x, so you’d better reorder now. Others might be more complex: What will be the effect of changing our portfolio mix?
Complexity of action can arise when dealing with interdependent systems, multiple parties, and lots of variables. (Welcome to healthcare!) Like the old game Whack-a-Mole, addressing one trend may lead to unanticipated or unwanted changes in others. Image how well you could play if you knew what was going on under the table. To properly act on predictions, you have to employ systems thinking.
The Future of Prediction
As our data sources grow, our ability to analyze them using AI and ML advances. Given regulatory and market pressure to provide higher-quality care while reducing care costs, healthcare’s need to manage to value also continues to increase.
In the coming years, we’ll see a lot more use of predictive analytics in healthcare, and a lot more tools and vendors to support it. As we analyze these predictive tools, we will focus on how healthcare will employ them to deal with the complexities of data selection, model accuracy, user perception, and actionability.
0 Comments