Over the course of the last 18 months artificial intelligence (AI) has matured to the point where there are several viable vendor options for nearly every use case.
AI dominated every aspect of the annual gathering of the Radiological Society of North America (RSNA18) in Chicago. Self-described ‘machine learning’ vendors with a presence on the conference floor more than doubled from 49 in 2017 to over 100 in 2018, 25 of which were first-time presenters.
I moderated a panel hosted by Life Image on practical uses cases of imaging AI and was blown away by the conversation that ensued, particularly what I learned about how the veteran radiologists feel about being “replaced.” During the question period, a senior radiologist approached the microphone to address a comment made by a more junior radiologist on the panel which he interpreted to be too pessimistic about the potential for AI. To paraphrase the elder, “Listen here sonny, you are too young to fully appreciate what you don’t know, and you don’t know how many mistakes you are truly making on a day to day basis. 1-2 percent error rate due to fatigue alone. WE NEED AI to save us from ourselves.”
Not all old school radiologists are so optimistic: “When you’re going up the ride, you get excited,” noted University of Chicago radiologist Paul Chang said during his workshop on AI. “But then right at the top, before you are about to go down, you have that moment of clarity—‘What am I getting myself into?’—and that’s where we are now. We are upon that crest of magical hype and we are about to get the trench of disillusionment… It is worth the rollercoaster of hype. But I’m here to tell you that it’s going to take longer than you think.”
Last year, the major cloud vendors each had a significant footprint at RSNA, but this year the two largest, Amazon and Microsoft, were nowhere to be found. Only Google Cloud had a significant, if smaller than last year’s, presence. Donny Cheung, one of the Google Cloud team leaders, was on the panel I moderated and his message to the imaging community could be boiled down to two words: storage and compute. No dashboards or toolkits or tensorflowing, just storage and compute, a smart and refreshing strategy amidst the obvious feature creep many other vendors suffer from.
Over the course of the last 18 months artificial intelligence has matured to the point where there are several viable vendor options for nearly every use case.
While it was surprising that Amazon had no noticeable presence, it was even more surprising to find Facebook making news on the conference floor. Facebook AI Research (FAIR) has partnered with the Center for Advanced Imaging Innovation and Research (CAI2R) in the Department of Radiology at NYU School of Medicine and NYU Langone Health to release the fastMRI, an open source dataset for training and testing machine learning algorithms to reconstruct MRI images.
This offering is roughly equivalent to similar X-Ray and CT datasets released by NIH. Given that algorithms ALWAYS significantly outperform on all metrics against the data used to train them versus new data, the industry needs independent validation of AI claims so it is unlikely that Facebook moves the needle with this offering.
PACS vendors want to get in on the AI action by positioning their existing products as AI marketplaces or platforms (Philips HealthSuite Insights, PureWeb, LifeImage, GE Edison, FujiFilm REiLI, Nuance AI Marketplace, Blackford Analysis). Nuance has shown there is a viable market for these platforms, counting 40 startups and health systems among user groups for its marketplace. There is no shortage of startups taking this approach (MDW, Envoy.ai, Medimsight, Lify, Fovia). Imaging hardware vendors refused to be left out too, with many partnering with AI vendors to embed their algorithms on the “edge.”
International AI startups, particularly from Israel, China, and South Korea, stood out from the crowd in terms of their approach to product design, but only the companies from Israel have been able to break into the US market so far. One Korean company voiced frustration with the FDA, saying it couldn’t understand what was wrong with their application. I wonder if it underestimates the importance of using data from US patients to validate their algorithms?
Not everything we learned about AI at RSNA was positive. A paper presented at the conference showed that neural networks could be used to insert malignant features into mammograms giving a false positives, and then reverse the alterations without detection. Even scarier, it took about 680 images to train the algorithm that executed the adversarial attack. Cyberattacks have been increasing in healthcare over the last couple years, but mostly just for taking data hostage and demanding ransom to get it unencrypted. This type of attack would represent a frightening new paradigm in cyber-vulnerability, and it is certainly not difficult to imagine ways this could be exploited to make money. It could be used for a different sort of ransom, with every image appearing to show cancer until a ransom is paid and the adversarial attack is reversed. Another conceivable way this type of attack could be exploited would be falsifying data for clinical trials.
Matt Guldin · 2 years ago
Chilmark Team · 1 month ago
Chilmark Team · 2 months ago
Brian Edwards · 2 weeks ago
Revisiting Our 2018 Predictions
As is our custom here, we like to look back on our predictions for the closing year and see just how well we did. Some years we do amazingly well, others we over-reach and miss on quite a few. For 2018, we got seven of our 13 predictions spot-on, two were mixed results and four predictions failed to materialize. If we were a batter in the MLB we would have gotten the MVP award with a .538 batting average. But we are not and have to accept that some years our prediction average may hover just above the midpoint as it did this year.
Stay tuned, 2019 predictions will be released in about one week and it is our hope that they will inspire both rumination and conversation.
(Note: the bigger and plain text are the original predictions we made in 2017, while the italic text is our review of 2018).
Major mergers and acquisitions that mark the end of 2017 (CVS-Aetna, Dignity Health-CHI and rumored Ascension-Providence) will spill over into 2018. Both Humana and Cigna will be in play, and one of them will be acquired or merged in 2018.
MISS – neither happened. However, Cigna did pick-up PBM service Express Scripts and rumors continue to swirl about a possible Humana-Walmart deal or more recently, even a Walgreens-Humana deal.
Hot on the health heels of CVS’ acquisition of Aetna, growth in retail health reignites, albeit off a low overall footprint. By end of 2018, retail health clinic locations will exceed 3,000 and account for ~5% of all primary care encounters; up from 1,800 and ~2%, respectively, in 2015.
MISS – Modest growth in 2018 for retail health clinics with an estimate of around ~2,100 by year’s end. Telehealth, which is seeing rapid growth and on-site clinics may be partially to blame.
In a bid to one-up Samsung’s partnership with American Well, and in a bid to establish itself as the first tech giant to disrupt healthcare delivery, Apple will acquire a DTC telehealth vendor in 2018.
MISS – Apple continues to work on the periphery of care with a focus on driving adoption of its Health Records service in the near-term with a long-term goal of patient-directed and curated longitudinal health records.
Despite investments in population health management (PHM) solutions, payers still struggle with legacy back-end systems that hinder timely delivery of actionable claims data to provider organizations. The best intentions for value-based care will flounder and 60% of ACOs will struggle to break even. ACO formation will continue to grow, albeit more slowly, to mid-single digits in 2018 to just under 1,100 nationwide (up from 923 as of March 2017).
HIT – MSSP performance data showed only 34% earned shared savings in 2017 (up from 31% in 2016) and by year’s end it is estimated there will be ~1,025 ACOs in operation.
While some of the major EHR vendors have announced support for write access sometime this year and will definitely deliver this support to their most sophisticated customers, broad-based use of write APIs will happen after 2018. HCOs will be wary about willy-nilly changes to the patient record until they see how the pioneers fare.
HIT – FHIR-based read APIs are available from all of the major EHR vendors. Write APIs are still hard to find. To be fair, HCOs as a group are not loudly demanding write APIs.
True cloud-based deployments from name brand vendors such as AWS and Azure are in the minority today. But their price-performance advantages are undeniable to HIT vendors. Cerner will begin to incent its HealtheIntent customers to cloud host on AWS. Even Epic will dip its toes in the public cloud sometime in 2018, probably with some combination of Healthy Planet, Caboodle, and/or Kit.
HIT – adoption of cloud computing platforms is accelerating quickly across the healthcare landscape for virtually all applications. Cloud-hosted analytics is seeing particularly robust growth.
Providers will continue to lag behind payers and self-insured employers in adopting condition management solutions. There are two key reasons why: In particular, CMS’s reluctance to reimburse virtual Diabetes Prevention Programs, and in general, the less than 5% uptake for the CMS chronic care management billing code. In doing so, providers risk further isolation from value-based efforts to improve outcomes while controlling costs.
HIT – Awareness of the CCM billing code (CPT code 99490) remains moderate among providers and adoption is still estimated at a paltry less than 15%.
Mobile accessibility is critical for dynamic care management, especially across the ambulatory sector. More than 75% of provider-focused care management vendors will have an integrated, proprietary mobile application for patients and caregivers by end of 2018. These mobile-enabled solutions will also facilitate collection of patient-reported outcome measures, with 50% of solutions offering this capability in 2018.
MIXED – While the majority of provider-focused care management vendors do have an integrated mobile application (proprietary or partnership), collecting PROMs is still a functionality that remains limited through an integrated mobile solution.
A wide range of engagement, PHM, EHR, and care management solutions will make progress on documenting social determinants of health, but no more than 15% of solutions in 2018 will be able to automatically alter care plan interventions based on SDoH in 2018.
HIT – despite all the hoopla in the market about the need to address SDoH in care delivery, little has been done to date to directly affect dynamic care plans.
The hard, iron core of this issue is uncertainty about its real impact. No one knows what percentage of patients or encounters are impacted when available data is rendered unavailable – intentionally or unintentionally. Data blocking definitely happens but most HCOs will rightly wonder about the federal government’s willingness to go after the blockers. The Office of the National Coordinator might actually make some rules, but there will be zero enforcement in 2018.
MIXED – Last December we said, “The hard iron core of this issue is uncertainty about its real impact.” Still true. Supposedly, rulemaking on information blocking is complete but held up in the OMB. The current administration does not believe in regulation. So “data blocking” may be defined but there was and will be no enforcement or fines this year.
Providers will pull back on aggressive plans to broadly adopt and deploy PHM solution suites, leading to lackluster growth in the PHM market of 5% to 7% in 2018. Instead, the focus will be on more narrow, specific, business-driven use cases, such as standing up an ACO. In response, provider-centric vendors will pivot to the payer market, which has a ready appetite for PHM solutions, especially those with robust clinical data management capabilities.
HIT – PHM remains a challenging market from both payment (at-risk value-based care still represents less than 5% of payments nationwide) and value (lack of clear metrics for return on investment) perspectives. All PHM vendors are now pursuing opportunities in the payer market, including EHR vendors.
This is a case where the threat of alert fatigue is preferable to the reality of report fatigue. Gaps are important, and most clinicians want to address them, but not at the cost of voluminous dashboards or reports. A single care gap that is obvious to the clinician opening a chart is worth a thousand reports or dashboards. By the end of 2018, reports and dashboards will no longer be delivered to front-line clinicians except upon request.
MISS – Reports and dashboards are alive and well across the industry and remain the primary way to inform front-line clinicians about care gaps.
Arterys, Quantitative Insights, Butterfly Network, Zebra Medical Vision, EnsoData, and iCAD all received FDA approval for their AI-based solutions in 2017. This is just the start of AI’s future impact in radiology. Pioneer approvals in 2017 — such as Quantitative Insights’ QuantX Advanced breast CADx software and Arterys’s medical imaging platform — will be joined by many more in 2018 as vendors look to leverage the powerful abilities of AI/ML to reduce labor costs and improve outcomes dependent on digital image analysis.
HIT – With about a month left in 2018 the count of FDA approved algorithms year to date is approaching 30 and could potentially hit three dozen by year end. This is a significant ramp up in the regulatory pipeline, but more is needed in the way of clear guidance on how they plan to review continuously learning systems and best practices for leveraging real-world evidence in algorithm training and validation.
What do you think of 2018 for health IT?
Predictions in Healthcare: Unpacking the Complications
Interest in predictive analytics in healthcare is on the rise. How long will someone be in the hospital? What are their chances of being readmitted? Which treatment will work the best?
On the surface, these seem like straightforward questions, but it turns out that prediction is not a simple topic. Questions about what data and effort goes into the prediction, how accurate it is, who gets to see it, and how actionable it is can all make a huge difference in its value.
As we start to develop our major Market Trends Report for Advanced Analytics, scheduled for release later this year, I thought I would unpack these questions a bit.
To start, it makes sense that the more data and the better data you can include in making a prediction, the more accurate it will be. However, you don’t always know what factors will matter the most, or at all.
If you over-collect data to ensure you factor in most everything, you unnecessarily raise the cost, time, and complexity, as data warehouses are not inexpensive and interoperability is still a struggle. If you under-collect, you might miss something important, and clinicians will continue to complain about the burden of documentation.
The recent focus on social determinants of health is promising, but it requires much more data collection and management, along with more sophisticated analytic models. Here, we welcome the contributions of AI and Machine Learning (ML) to help determine which variables to focus on.
Prediction is not a simple topic. Questions about what data and effort goes into the prediction, how accurate it is, who gets to see it, and how actionable it is can all make a huge difference in its value.
Accuracy has all kinds of complications. You’ve likely come across the difference between precision and recall.
With precision, you measure how right you were that something was going to happen when it did, in relation to how often you thought it was but didn’t. With recall, you measure how often you thought something was going to happen, and you were right, but you missed some that did happen. In the first situation, false alarms could be costly, leading to additional tests and much worry. In the second situation, missing an important prediction (such as a cancer diagnosis) could also be costly.
Fortune tellers make many predictions and eventually get some stuff right, but most of us don’t take them seriously; Chicken Little got into trouble when he cried out too many times. Conversely, someone who makes too many conservative predictions might have a lot of credibility but also miss too many opportunities.
Consider who wants to know or gets to see the predictions. Most clinicians, healthcare workers, and leadership teams want do the best they can, but negative predictions about their approaches or performance may not be met with welcome arms. Exposing their problems can lead to potential unwanted oversight by others, reduced funding, etc.
In other words, expect pushback about the quality of the data, the accuracy of the prediction, the motivations of the prediction team, and who is really responsible for the bad trends. Think about how predictions are supported and presented. Also consider the flip side: Positive predictions, if not accurate, may lull an organization into a false sense of security.
This leads to one of the most important questions: How actionable is the prediction, and what can be done about it? (This also brings prescriptive analytics into the discussion.) If the answer is “not much,” there may still be value in knowing it – but you have to brace for impact, as it could also lead to helplessness and negativity; consider the effect of telling a patient they have an incurable disease.
Better in most circumstances to focus on those predictions where positive action can be taken. Some actions might be obvious: You’re running out of medication x, so you’d better reorder now. Others might be more complex: What will be the effect of changing our portfolio mix?
Complexity of action can arise when dealing with interdependent systems, multiple parties, and lots of variables. (Welcome to healthcare!) Like the old game Whack-a-Mole, addressing one trend may lead to unanticipated or unwanted changes in others. Image how well you could play if you knew what was going on under the table. To properly act on predictions, you have to employ systems thinking.
As our data sources grow, our ability to analyze them using AI and ML advances. Given regulatory and market pressure to provide higher-quality care while reducing care costs, healthcare’s need to manage to value also continues to increase.
In the coming years, we’ll see a lot more use of predictive analytics in healthcare, and a lot more tools and vendors to support it. As we analyze these predictive tools, we will focus on how healthcare will employ them to deal with the complexities of data selection, model accuracy, user perception, and actionability.