Love the Machine (Learning)
Since 2022, hundreds of billions of dollars have been spent on making sure the public associates AI with LLMs and not with the real workhorse—Machine Learning. It’s the kind of intelligence that’s been reading radiology scans and flagging irregular heartbeats for well over a decade. It’s narrow, task-specific, lacks the ability to hallucinate, and is reasonably well-validated.
The problem is generative AI: large language models that predict the next plausible word in a sequence and call it knowledge. The same tools that have convinced teenagers to kill themselves, can’t consistently count the number of Rs in “Strawberry”, and are designed for sycophancy and maximum engagement, are now being handed the keys to clinical documentation, patient triage, diagnostic assistance, and drug discovery.
Eighty-one percent of physicians now use AI professionally, according to the American Medical Association’s 2026 survey — more than double the rate from 2023. Meanwhile, consumer trust in AI for healthcare has plummeted, from 52% to 44% over the same window. Generative AI’s obvious failings threaten the reputation of even the most basic of “dumb” machine learning tasks, because how is the public meant to discern between generative AI and ML when their doctor tells them they used “AI” to assist with a diagnosis? Hell, even the doctor might not be able to tell the difference. They get pitched on dozens of tools at any given time, and most don’t understand their inner workings. After all, why should they? They’re a doctor, Jim, not a software engineer.
Excuse Me, Your Privates Are Showing
HIPAA was written in 1996. It covers providers, insurers, and clearinghouses. It does not cover your Oura ring, your Fitbit, your ChatGPT conversation about that weird mole, or the genetic data 23andMe auctioned off when it went bankrupt. A booming market of data brokers now pays hospitals for “de-identified” patient records to train AI models — a market valued at nearly $9 billion — and researchers have already shown that LLMs can re-identify those supposedly anonymous records. One NYU study pulled a specific pregnant patient’s identity from a clinical note that mentioned nothing more identifying than horseback riding.
Google, meanwhile, is assembling the most comprehensive health-data pipeline on Earth, though you probably already assumed that. Android owns roughly 70% of the global smartphone market, and Google also owns Fitbit. They made a promise in 2019 to keep health data out of advertising, which sounds reassuring until you read the actual privacy policy, which permits using data to “improve services” — a phrase capacious enough to drive Sam Altman’s ego through.
Large Language Modus Operandi
The opt-in model (those papers you glanced over at your last doctor’s appointment before signing anyways) is already eroding. AI scribes record your doctor’s visit and generate notes — and while you can technically decline, good luck navigating that conversation mid-appointment when your ass is finally seated after a three hour wait. One patient in Australia had a $1,300 appointment cancelled for refusing. You see where this is going: today’s opt-in becomes tomorrow’s default, becomes next year’s invisible infrastructure, just like browser cookies, just like location tracking, just like every other privacy trade-off we’ve clucked our tongues at while striding full bore into.
With all of that out of the way, this tunnel may be shorter than it seems. World models, an alternative architecture (or the next evolution after LLMs) learn how systems evolve over time rather than predicting the next word. They don’t hallucinate in the same way because they aren’t generating text — they’re modeling states.
Early research is applying them to tumor progression, disease forecasting, and surgical robotics. They’re immature, unstable, and at least a year or more away from any real product. But they represent the only credible path toward AI that a clinician could somewhat-reliably trust with a diagnosis. Where chatGPT takes a stab at statistical correlation, a world model is purpose-built for prediction, rather than prediction as (and I’m being slightly reductive here) an afterthought.
Half-Baked
The health industry is adopting AI faster than its trust infrastructure, its privacy law, and its cybersecurity posture can support. That’s straightforwardly a governance problem, and governance moves at the speed of committee hearings and election cycles while technology moves at the speed of venture capital.
So what does the health market look like when the keys get handed to generative AI? Probably a lot like the internet looked when we handed the keys to ad tech: useful, ubiquitous, and corrosive to the things we said we cared about most. The difference is that this time the thing being corroded isn’t your attention, it’s your medical records, your literal genetic code, and your trust in the person across the exam table telling you those cells probably aren’t cancerous.
As always, zack.wall@icloud.com for feedback and suggestions.