The "AI-Powered" Tool You're About to Buy Might Just Be a $20 API Key in a Trench Coat
If you run a practice, you've probably gotten the email. Or the LinkedIn DM. Or the cold call. Some version of: "We built an AI-powered tool that will transform your [intake / scheduling / documentation / billing]." The website looks great. The demo looks great. And somewhere in the back of your mind, a small voice is asking: is this actually a company, or is this two guys and a laptop?
That instinct is worth listening to. Here's why.
What "vibe coding" actually means
There's a term that's taken over tech Twitter, TikTok, and every startup meme account in the last year or so: vibe coding. It refers to building software by basically describing what you want in plain English to an AI model and letting it write the code, no traditional engineering background required. You've probably seen the reels: someone builds a whole app in a weekend, narrates the whole thing like it's a magic trick, and by Monday they've got a live product with a Stripe checkout button.
And to be clear, that's genuinely impressive as a shift in how fast software can get built. The problem isn't the technique. The problem is what it means for an industry that runs on protected health information.
If you want to see it for yourself, try it. Go to a vibe coding platform like Lovable or v0 (Vercel's version), make a free account, and just describe an app you want in plain English. No code, no engineering background, just typing what you want. Give it ten minutes. I'd bet you'll be genuinely surprised at how far you get, and your first instinct afterward will probably be "wait, could I actually sell this?" That instinct is exactly the one a lot of these AI wrapper founders are acting on. It's not a knock on them. It's just worth feeling firsthand how low the bar has gotten, so the rest of this post lands the way it's supposed to.
Because here's the thing nobody in those reels talks about: building something that looks like a real product has never been easier. A clean landing page, a slick dashboard, a chatbot that sounds confident, all of that is now table stakes, achievable by literally anyone in a weekend. What used to take a real engineering team and signal real investment now signals almost nothing. The polish isn't proof of anything anymore.
Why that's a real problem in healthcare specifically
Outside of healthcare, a thin wrapper around ChatGPT is low stakes. Worst case, your AI-generated marketing copy is mediocre.
In healthcare, that same wrapper might be touching PHI, and "looks legit" has zero relationship to "is actually compliant." A tool can have a beautiful UI and still have no signed BAA that covers how the AI itself handles data, no real audit trail, and no idea what happens to your patients' information once it hits the model.
This isn't hypothetical risk. Civil penalties for HIPAA violations now reach $2,190,294 at the top tier, and 2025 set a record for large healthcare data breaches. Meanwhile, industry surveys are finding that even inside healthcare organizations that are trying to do this right, most staff don't have clear guidance on what's actually safe to use, in one recent physician survey, only 8% said their organization's AI policies were clear enough to follow. If the people building AI compliance policy full-time are struggling to keep up, a solo founder who shipped a wrapper last month almost certainly hasn't thought it through either.
And there's a reason there are suddenly so many of these companies to sort through. According to Bessemer Venture Partners' 2026 healthcare AI report, AI companies captured 55% of all health-tech venture funding in 2025, up from just 29% in 2022. That's not a gradual trend, that's the entire investment landscape reshaping itself in about three years. A lot of capital chasing a hot category means a lot of new entrants, and not all of them are built the way you'd assume.
None of this means avoid AI tools. It means know how to tell the difference between a real one and a wrapper wearing a nice outfit.
Here's who's actually on the hook if it goes wrong
This is the part most practices don't think through until it's too late: if a vendor you've hired has a breach and your patients' PHI is exposed, you don't get to point at them and walk away clean.
Since 2009, business associates (that's the vendor) have generally been directly liable to federal regulators for their own HIPAA violations. That's real, and it matters. But as a general rule, it doesn't shift your liability as the covered entity. Practices that skip real due diligence before signing, or that know (or reasonably should know) a vendor isn't handling PHI properly and keep working with them anyway, can generally face liability of their own, including civil penalties. A signed BAA doesn't indemnify you against that. It's a contract that sets expectations, not a shield.
On top of that, breach notification is typically yours to deal with regardless of fault. If a vendor's infrastructure is what gets breached, they generally have to notify you within a set window (60 days is the standard under federal rules). Then the clock is on you to notify your own patients and report to HHS. Your patients don't have a relationship with your vendor. They have a relationship with you. So the calls, the trust repair, and a real share of the financial and reputational fallout tend to land on your practice, even when the vendor's server was the one that got breached.
That's the actual reason the checklist below matters so much. This isn't paperwork for paperwork's sake. Choosing a vendor casually doesn't just put their reputation at risk. It puts yours on the line too.
(General information, not legal advice. If you're evaluating a specific vendor or want to understand your exact exposure, talk to a healthcare attorney or compliance professional who can look at your actual contracts and situation.)
A 5-step checklist before you sign anything
Ask for the actual BAA, and read what it covers. A generic Business Associate Agreement isn't the same as one that specifically addresses how their AI component handles PHI. If they can't explain how their AI model touches patient data, that's your answer.
Ask for their SOC 2 report, not the badge. Anyone can put a "SOC 2 Compliant" badge on a landing page. Ask to actually see the report (or the equivalent audit for their scale). A real vendor will have one ready to send. A hesitant, vague answer tells you what you need to know.
Ask what happens to the data after it's processed. Is it used to train their model? Where is it hosted? Can you export a real audit log of who accessed what, and when? "We take security seriously" is not an answer. A specific technical answer is.
Talk to their existing clients, ideally other healthcare practices, not just logos on a slide. A reference call takes fifteen minutes and tells you more than any sales deck will.
Ask how long they've been around and who's actually behind the product. This is the quiet way to find out if you're looking at a real company or a very good weekend project. And here's the part worth knowing: if you are their first healthcare client, that's not necessarily a red flag, it's leverage. This space is getting commoditized fast, and new vendors chasing their first real customer will often negotiate far more on price and terms than you'd expect. You can either walk, or you can use the position you're actually in.
The bottom line
You don't need to be scared of AI tools. If anything, the practices that use them well are going to have a real edge. What you need is to stop assuming "it looks professional" means "it's safe," because right now, those two things have less to do with each other than they ever have. Ask the boring questions before you sign. The vendors worth working with will have good answers ready. The ones that don't are telling you something too.
Vetting a vendor pitch right now and want a second opinion?