Part 3: Why Your Health Data is a Mess

Working in healthcare can feel like navigating a maze in a fog. Doctors are skilled problem solvers, but their maps are often 100 page faxes pieced together from four different electronic health record systems. While the tech world warns that not using AI means falling behind, doctors need to help patients now, with only about 12 minutes for each visit.

The real world is messy. Decisions are made with limited data and time. In an era of highly targeted mobile advertising and wearable supercomputers – the dissidence between data in healthcare and data used to sell you shoes is LOUD.

This post explains specific healthcare data challenges to show a core problem with value based care. By partnering with insurance companies that have poor information, value based care has followed money instead of data. I believe building a foundation on reliable ground truth data, not lagging insurance claims, offers a path to higher quality, data informed care at a lower cost.

What is a health record?

People talk a lot about protecting health records, but what is a health record? Unclear.

At a minimum, it is your name connected to a health facility. That simple combination is protected by laws like HIPAA and security frameworks like HITRUST & SOC 2. A complete health record would ideally contain all your health information like demographics, a full medical history, a list of medications and allergies, and notes from your care team. Hopefully it would also have detailed documents like test results, treatment plans, immunization records, and consent forms.

However, that document doesn’t exist.

Health records in the real world

Your actual 'health record' is more like Marvel's Infinity Stones, scattered across the galaxy, and guarded by different villains.

Insurance company asteroid field: Where your past health services go to live, but only for the time you were with them. Think of it as a series of tiny, isolated islands but if you leave the island, the island is destroyed.
Primary care planet: Full of notes and forms, but good luck if you ever switch doctors!
Prescriptions planet: An actually high functioning planet since Surescripts has an up to date and comprehensive list of your medications. However, the planet is protected by the CVS space force which seems to be dispensing medications for the first time every time you go.
Hospital planet: Robust data on surgeries and test results run by that hospital but with strong gravitational pull so that data can come in but never leave.
Self-care planet: Where your personal wellbeing battles it out with your Strava KOMs (for all you fitness buffs out there!)

Getting data to travel between these planets is like trying to send a fax to the moon – expensive, slow, and…probably impossible. The financial cost of this lack of interoperability is immense and a whole separate issue from this post.

I personally love watching tortured healthcare engineers try to help each other on the r/healthIT subreddit. Highly recommend!

Your insurance company doesn't have access to the notes from your primary care doctor's visit. If your primary care doctor refers you to the hospital, the hospital will get your records by requesting a fax. None of these people have any idea about your mental health, data from your Apple Watch, or your ability to pay for different treatments.

The real gap in your health record

The thing I think is missing the most about your health is the GOOD in your life. It's like your health record is only interested in your bad days, not your triumphs. This makes it very difficult for providers to connect with patients — they are medical fire fighters rather than trusted partners.

A close family member of mine recently completed a substance recovery program. They are doing incredibly well. They are invested in their health in a way that would inspire any provider. They are hitting sobriety milestones, exercising regularly, making real progress on diet and wellness. Their transformation is one of the most positive things happening in our family right now.

But they have not had a physical or gotten their blood checked in over a decade. There could be a lot missing.

The healthcare system has no idea any of this is happening. Their medical record, if a provider pulled it up today, would reflect their worst days and miss the best. No provider is reaching out to say "this is an incredible time to get a full checkup, to establish a baseline for this new chapter, to make sure nothing was missed during the hard years." The system only knows you when things go wrong. It has no mechanism for knowing when things go right.

This is the gap that frustrates me most. We built technology at Cityblock to identify people who needed help the most but "help" was always defined by what was broken. We never had a way to find people at their most receptive, their most motivated, their most ready to invest in their health. The best moment to intervene is often the moment after someone succeeds, not the moment after they fail.

My own (inaccurate) health record

It is illuminating trying to build your own accurate medical record.

While living in New York City, I have visited four different health systems. As I wrote this, I requested my medical records from all four. The results were very odd!

A surgery I had years ago was missing from every single system. Each facility had a completely different list of medications. None had documented my family medical history or a complete list of my vaccinations. There was no data on my mental health, living environment, or my current physical therapy.

There were no hallucinations which is good! No invented conditions or fabricated visits. The records were simply incomplete in different ways, each system holding its own shard of an inaccurate whole. If a new doctor looked at any one of them, they would get a wrong picture of my health. If they somehow looked at all four, they would get four wrong pictures that didn't agree with each other.

A third group, the clinical data exchange networks, also has pieces of my health data. These include Health Information Exchanges that send event notifications, Surescripts for medications, and networks like Commonwell and Carequality that allow providers to request records from eachother. Epic's Care Everywhere network allows sharing between providers, but only if the patient has linked those providers in MyChart. Unfortunately, I wasn’t able to request my records from any of these sources.

Even with all the talk about making health data work together, my health record is still wrong. The data we're sharing is garbage. This was my experience at companies like Firsthand and Cityblock. We would find out that patients were taking completely different medications and had different health conditions than we thought. A big part of what we did was just correcting their records.

The missing data

In 15 years in NYC, I’ve been to 5 primary care appointments which — is low! — but isn’t far from average. Data on checkups varies, but studies suggest that a most people do not see a primary care physician every year. This creates huge gaps in time with no data!

Even when you do see a doctor, a lot of information gets left out. I told my doctor about my knee pain and some of the vitamins I take, but I didn't mention everything. This is normal. Studies show that 60-80% of people don't tell their doctors everything. They might forget, be embarrassed, or just not want to be a difficult patient.

My doctor seemed rushed. After the appointment, I opened Instagram and saw an ad for something I wanted to buy.

Do you know who has more real time data about how I’m doing? Google, Facebook, Amazon.

Sell me healthcare

It sounds crazy, but combining advertising data with medical records could transform healthcare.

Advertisers know your age, where you live, your job, and your interests. They have your location history and know what you like based on what you read and follow online.

Unlike health data, which is scattered and incomplete, the advertising industry has detailed profiles on most Americans, sometimes going back decades. Google knows what you're interested in, Meta knows your social connections, and Amazon knows what you buy.

Your doctor and health researchers don't have access to any of this information. Combining these datasets could help us understand if medications and surgeries are actually working from how people’s behavior online changes after treatment.

While there are major privacy concerns for more centralized systems, our current system is doing a terrible job of protecting our health data. In 2024 alone, patient records for 276 million Americans were breached, roughly four out of every five people in the country. The dark web may have a more complete and accurate health record for me than any of my four doctors.

The fragmentation of our health system is itself a security risk. Regular business operations require transmitting data between countless parties using data security standards customized for each one. It is a complex issue perhaps for another blog post.

In a perfect world

Success in value-based care depends on good data. The goal for a VBC company is to give the right treatment to the right person at the right time. Value-based care companies try to do this by specializing in certain conditions, like diabetes.

The process is supposed to be simple:

Partner with an insurance plan.
Select a group of people with a specific health condition.
Group them by location.
Assign them to local care teams.
Prioritize them based on need.
Engage them and provide care.

But it's almost impossible to get accurate, up-to-date health data. Just identifying people with diabetes is a huge challenge.

The problem with claims data

It is impossible to find the patients who need help the most when they need it.

At Cityblock and firsthand, we experienced every failure mode insurance claims data has to offer:

30-50% of the identified population had missing or minimal claims data.
Cost information was stripped: insurance companies redacted paid amounts from the files they sent us. How can we compute savings?
Three months is the minimum lag. That's how long it takes for claims to be adjudicated and transmitted. After that lag, the data is generally comprehensive for that one insurance plan.

But the real problem isn't lag. It's amnesia.

When a patient changes insurance plans, their claims history is zeroed out. The new insurer starts from nothing. In Medicaid populations who are exactly the people value-based care companies need to serve the most, have the highest churn. Thirty percent or more of members change plans each year. This means the insurance company that hired us to improve outcomes for their members often knew nothing about a large portion of that membership. No history, no medications, no diagnoses. Just a name and an address that might be wrong.

We were being asked to deliver personalized, preventive care to people whose medical histories had been wiped clean every time they changed jobs, moved, or had their benefits reassigned. The data we were building care plans on wasn't just old, it simply didn't exist.

Even when claims data is good, it only shows what happened under that one insurance plan. Details about your mental health, housing, or food situation are not included.

A value-based care company can get more records if they meet the patient and get them to sign a consent form. But by then, they've already committed to caring for that person, even if they've moved, are sicker than expected, or are in jail.

We once had an insurance partner tell us that two different patients were actually the same person — and that one of them had died.

One of our community health partners called the home to offer their condolences. The member's partner answered the phone. They were…lets say…extremely confused.

That same day, the health worker alerted our data team to the problem. We contacted the insurance company and told them their source data was wrong (ie: two distinct people had been merged into a single record). What followed was a week of back and forth: identifying which data belonged to which person, reprocessing our records, and eventually splitting them apart. It took significant additional time to harden our data systems against this class of error. We had been told that member IDs were stable and unique. They are not. Member IDs change. A single person can have multiple active IDs simultaneously. And old member IDs can be reused for entirely new patients.

This wasn't a one-off bug. It was a structural feature of the data we were building a care model on. In 2025, value-based care companies are still being assigned patients who are dead, in jail, or — as we discovered — two people at once.

Who does this well?

E-commerce companies have spent billions of dollars building systems to track customers and their purchases. With AI, they're getting even better at predicting what people will buy, how much they will pay and when they will buy it.

I experienced this gap firsthand when I built Kelp, a contextual recommendation tool. Kelp needed two things: good information and good signals for when to surface it. Without deep, OS-level access to user data, we could only get mediocre information and mediocre signals. Mediocre times mediocre is still mediocre — no amount of clever UI or AI makes up for a weak foundation.

Healthcare has the same problem at a much higher cost. We have mediocre data (lagging, fragmented claims) combined with mediocre signals (no real-time patient engagement data). The industry keeps trying to build better AI on top of this foundation. But you cannot tell whether the AI is right or just hallucinating when the underlying data is this flawed. Fixing the data must come first.

The healthcare industry never paid those big data billions, so we don't know what happens to patients after they leave the doctor's office. Our health data is missing basic information like interests, fitness data, and the patient's own view of their health. The data that does exist is often both inaccurate and incomplete as it was for me.

No one in healthcare can answer these simple questions:

Did you go to the specialist I referred you to?
Do you feel like your health is getting better or worse?
Did you take your medication today?
Are you in jail?
Are you still eligible for your health insurance?
What are your parents' medical conditions?
Do you have any mental health or substance use issues?
Do you have a stable income and place to live?
Have you had any big life events recently?
Have you filled out our consent form?

Here's the wild part: Someone trying to sell you shoes could answer more of those questions than your own doctor!

Gravity always wins

Today, value based care companies primarily partner with insurance companies. Gravity is dollar savings in the form of reduced medical spend & increased reimbursement. It pulls all value based care into its orbit.

In an AI world, ‘data is the new gravity’ and will drive the next wave of healthcare innovation. Healthcare has spent many billions trying to squeeze water out of the rock that is insurance claims data.

It turns out, we've been looking for the princess in the wrong castle all along. She's not with the insurance companies; she's chilling out at Epic (or maybe even Google or Facebook). The future of value based care will be built around companies with rich data and a real information asymmetry to existing players and to insurance companies.

Healthcare should keep patients at the center. Most data sharing efforts have put companies before consumers, focusing on reducing the risk of getting sued or fined instead of improving the customer experience.

Contacting a mother to offer condolences for their dead son when their son is not actually dead is not a great patient experience! If the U.S. wants to control healthcare costs, it needs a universal system with accurate, complete information.

Main takeaways

Current health data is often inaccurate and was created for billing, not for creating a complete patient record.
Health records are missing key information about a person's life that could improve their care. The missing data is from inconsistent PCP visits, lack of social determinants & mental health data as well as not tracking positive factors of someone’s well-being.
Data from sources like someone’s online presence and advertisers could provide better and more timely information about a person's well-being.

I am hopeful about the future

For value-based care to finally follow the data, a few things need to change. While pressure can come from different directions, meaningful progress likely starts with a new foundation built on better policy.

Meaningful change starts with policy. If the government creates a unified patient ID and enforces modern data-sharing agreements like TEFCA, could push toward a single source of truth for every patient. This foundational layer would make it possible for real innovation to happen, rather than creating even more competing data sources.

Once that foundation is in place, other players can start to build on it:

Patients, empowered by their own data, could use new AI tools to navigate the health system. I am particularly optimistic here since consumers could be motivated to improve the accuracy of their health record by the promise of higher quality, most integrated care.
Big Tech companies like Apple or Google could re-enter the space, using this new data infrastructure to create a truly useful health record for their users. By combining clinical data with information from their own platforms (like wearable data), they could sell access to value-based care companies, creating a new market based on rich, real-time information.
Hospitals and Accountable Care Organizations (ACOs) could more easily partner with value-based care companies. With access to better data from EHRs, VBC companies could more effectively deploy preventative care and care management. Financial pressures, like reduced Medicaid eligibility, may motivate Hospitals and ACOs to find proactive solutions for keeping people healthy and out of the emergency room. Data standards could lower the integration cost with the myriad of partners required.

My main worry is that the healthcare industry is rushing to use AI without first fixing the data. This could lead to people being denied the care they need because of an incomplete and inaccurate health record.

While this is a lot! It is just one of post my attempt to share some things I’ve learned.

In the next post, we'll dive into value-based contracting, so stay tuned!