Everyone Is Betting Trillions on AI. So Why Has Nobody Found the Business Case?

The Red Access findings exposed roughly 5,000 leaky apps. The real story is not about AI writing bad code. It is about what happens when deployment becomes frictionless and responsibility disappears.<hr>The internet had a minor panic recently. Israeli cybersecurity firm Red Access published findings that sent tech journalists scrambling: roughly 5,000 AI-generated web applications were found to be leaking sensitive data, including patient medical conversations, school lesson recordings, and internal staff schedules. The story spread fast, and the culprit was named quickly: vibe coding.It is a satisfying villain. The term, coined by AI researcher Andrej Karpathy in early 2025, describes the practice of using AI tools to generate and deploy software with little to no traditional coding experience. Type a prompt, click deploy, watch your app go live. The mental image practically writes the headlines itself.<img src="https://cdn.calebms.com/articles/11a0752e-eb06-4074-a6c4-bc97fa155598.png" alt="71620.png" data-caption="A Screenshot of X(formerly Twitter) by Andrej on Feb 2025">But here is what those headlines consistently miss: the problem was never that AI wrote the code. The problem is that an entire layer of architectural responsibility was abstracted away, and nobody noticed it was gone.<ad-spot></ad-spot><h2>What Red Access Actually Found</h2>To understand the findings clearly, it helps to look at the actual numbers. Red Access scanned approximately 380,000 publicly accessible assets created using popular AI-powered coding tools such as Lovable, Replit, Netlify, and Base44. Of those, about 5,000 apps exposed potentially sensitive information.Red Access CEO Dor Zvi described the findings as part of research into "shadow AI," the growing trend of employees using AI tools without formal approval from their organizations. The exposed data ranged from patient conversations to internal business documents. Zvi called it "one of the biggest events ever where people are exposing corporate or other sensitive information to anyone in the world."The companies behind the implicated tools pushed back. Replit CEO Amjad Masad noted that users choose whether their apps are public or private, and that the existence of a publicly visible app does not automatically constitute a security breach. That is a fair technical point, but it largely confirms the core issue rather than dismissing it: the users did not know that making something public meant exposing everything inside it. The configuration options existed. The understanding of what those options meant did not.That gap, between a tool existing and a user understanding what it does, is where nearly every modern security disaster lives.<h2>The Barrier That Dropped</h2>Before the AI-assisted development surge, shipping a broken application to production required a meaningful amount of friction. A developer had to know how to configure a server, understand how databases handle access, write enough code to make the application function, and debug the inevitable errors that appeared during that process. None of those tasks guaranteed security, but they created natural checkpoints. You could not accidentally deploy a misconfigured database if you did not first understand enough about databases to set one up.<ad-spot></ad-spot>AI tools did not invent misconfigured databases. They did not introduce insecure APIs, exposed secrets in client-side code, or privilege escalation vulnerabilities. They automated the process of reaching production faster than any of those issues could be caught.The numbers bear this out. <a target="_blank" rel="noopener noreferrer nofollow" href="https://apiiro.com/blog/4x-velocity-10x-vulnerabilities-ai-coding-assistants-are-shipping-more-risks/">Research from Apiiro</a> found that repositories with significant AI-generated code showed a 322% increase in privilege escalation flaws and a 153% rise in architectural design flaws. These are not typo-level bugs that a syntax checker catches. These are structural decisions about who should be able to access what, baked into the application from the moment it is prompted into existence.Meanwhile, a <a target="_blank" rel="noopener noreferrer nofollow" href="https://www.veracode.com/blog/spring-2026-genai-code-security/">Veracode analysis </a>covering over 100 large language models across 80 distinct coding tasks found that approximately 45% of AI-generated code introduces security vulnerabilities. The overall security pass rate remained flat at around 55% in Veracode's March 2026 update, unchanged despite continued improvements in raw coding performance benchmarks. The AI is getting better at writing code. It is not getting better at securing it, because security is not what most prompts ask for.<h2>Why the AI Writes Insecure Code</h2>This is the part of the conversation that tends to get lost in the noise about vibe coding being reckless or AI being dangerous. The insecurity is not a malfunction. It is a mirror.Large language models learn to code by training on vast repositories of existing code. That existing code, accumulated over decades across GitHub, Stack Overflow, documentation, and open source projects, is full of insecure patterns. Developers have been writing SQL queries using string concatenation instead of parameterized inputs for thirty years. Hardcoded credentials have been appearing in version-controlled repositories since version control existed. If a meaningful percentage of the training data demonstrates an insecure pattern, the model learns that pattern as a valid way to solve that problem.Georgia Tech's Systems Software and Security Lab launched the Vibe Security Radar project in 2025 to track exactly how many publicly filed CVEs can be traced to AI-generated code. Their early results were stark: 86% of AI-generated samples failed to defend against cross-site scripting, and 88% were vulnerable to log injection. Both of those vulnerability classes sit firmly on the OWASP Top 10 list of the most dangerous web application security risks. These are not edge cases. They are foundational web security concepts that have been understood for over two decades.The AI is not hallucinating new categories of insecurity. It is faithfully reproducing the insecurity that already existed in the code it was trained on, at scale, and at a speed that makes manual review harder to maintain.<h2>The "Vibe Coding" Scapegoat</h2>There is a subtle gatekeeping dynamic embedded in how these incidents get reported, and it is worth naming.When a major enterprise suffers a data breach, the coverage tends to describe it as a sophisticated cyberattack, a state-sponsored intrusion, or a complex supply chain compromise. When a solo developer's app leaks data after being built entirely with AI assistance, the story becomes about vibe coding culture, the recklessness of non-technical people shipping products, and the dangers of democratized software development.<ad-spot></ad-spot>In reality, the breach at a Fortune 500 company and the leak from a vibe-coded startup are often caused by the exact same underlying failures: misconfigured access controls, exposed API keys, absent row-level security policies, or a database that should have been private sitting in a publicly accessible environment. The difference is the post-incident framing, not the vulnerability class.OWASP recognized this in 2025 by adding a dedicated category to its Top 10 specifically calling out vibe coding as a risk pattern. That recognition is useful. But the same OWASP Top 10 has listed injection attacks, broken access control, and security misconfiguration as dominant risk categories for years, all of which appear regularly in codebases written by credentialed senior engineers with no AI involvement whatsoever.According to the <a target="_blank" rel="noopener noreferrer nofollow" href="https://www.idtheftcenter.org/post/2025-annual-data-breach-report-record-number-compromises/">Identity Theft Resource Center's 2025 Annual Data Breach Report</a>, the United States recorded 3,332 data compromises in 2025, a new all-time record and a 79% increase over five years. That trajectory was established long before vibe coding existed as a term.<h2>The Two Types of AI-Assisted Developers</h2>The framing that "AI builds insecure apps" collapses a meaningful distinction that the actual security risk profile depends on entirely.The first type of AI-assisted developer is someone with an existing security intuition who uses AI tools as a high-speed pair programmer. They use AI to generate boilerplate code, scaffold components, and move quickly through problems they already understand. But they are reading the diff. They are thinking about the threat model. They understand what trust boundaries mean and why database access should be scoped to the minimum necessary level. For this developer, AI is a productivity multiplier, and the 3.6 hours per week saved in developer time gets redirected into review, architecture, and decisions that require contextual judgment the model cannot provide.The second type is what researchers have started calling "pure vibe coding": a user with zero security background who describes a product to an AI, receives a working application, and clicks deploy without reading a line of what was generated. This user does not know to ask whether the database is publicly accessible. They do not know what row-level security means. They cannot evaluate the access control logic because they do not know that access control logic needs to exist.The tool itself is identical in both scenarios. The risk is entirely determined by the pilot.A security scan in May 2025 of 1,645 web apps built on the Lovable platform found that roughly 10% had flaws exposing personal user data, specifically because creators relied on AI output without any review. That same year, 25% of startups in Y Combinator's Winter 2025 cohort reported codebases that were 95% AI-generated. Security researchers scanning close to 5,600 vibe-coded applications discovered over 2,000 vulnerabilities and more than 400 exposed secrets.AI-written code is not inherently bad. Deploying it unverified is.<h2>The Same AI Is Also Finding the Vulnerabilities</h2>There is an irony worth sitting with here. The same class of AI models being blamed for creating insecure applications is being actively deployed by security researchers to discover zero-day vulnerabilities in enterprise systems written entirely by human engineers, sometimes years or decades ago.Security teams at major firms are using large language models to analyze codebases for vulnerability patterns at a scale and speed that human reviewers cannot match. The AI does not discriminate between human-authored insecurity and AI-authored insecurity. It looks for patterns, and patterns are patterns regardless of who wrote the original code.This cuts both ways. AI generating insecure code is a concern. AI finding insecure code at scale is an asset. The organizations that understand both dimensions are already building processes that use AI for security scanning precisely because of the volume of AI-generated code now entering their pipelines.<h2>Where the Industry Actually Stands</h2>The scale of AI adoption in software development is now substantial enough that the question has shifted from whether AI will be part of production codebases to how the resulting risk will be governed.<ad-spot></ad-spot>According to current data, approximately 41% of all code written globally is AI-generated or AI-assisted, a figure that was just 6% in 2023. The AI coding tools market reached $7.37 billion in 2025 and is projected to hit $30.1 billion by 2032. GitHub Copilot alone has over 20 million cumulative users and is deployed across 90% of Fortune 100 companies. <a target="_blank" rel="noopener noreferrer nofollow" href="https://www.sonarsource.com/state-of-code-developer-survey-report.pdf">The Sonar State of Code Developer Surve</a>y found that developers report approximately 42% of their code is currently AI-generated or assisted, and they predict that share will increase by more than half by 2027.That trajectory creates a straightforward implication: the differentiator going forward is not who uses AI. Nearly everyone does. The differentiator is who understands the security architecture of what the AI produced.The tools needed to close this gap exist. Static application security testing, or SAST, can scan AI-generated code before it reaches a pull request. Software composition analysis tools check dependencies for known vulnerabilities, which matters because AI models have training cutoffs and may recommend library versions that have since been flagged. Manual review of authentication flows, encryption implementations, and payment processing logic remains essential in any context where the AI cannot be trusted to understand regulatory requirements.OWASP's guidance recommends prohibiting AI-generated code in high-risk areas, specifically authentication, encryption, and payment processing, without mandatory human review. That is not anti-AI. It is the same principle that requires a licensed architect to sign off on structural calculations even when the initial design was drafted by a junior engineer.<h2>The Real Abstraction Problem</h2>The deeper issue the Red Access findings surfaced is not that AI generates bad code. It is that AI tools have abstracted the deployment process in a way that also abstracted away the accountability that used to come with it.<ad-spot></ad-spot>When building a web application required setting up a server, configuring a database, writing authentication logic, and managing credentials, the difficulty of doing those things badly was itself a forcing function. Developers who got those things wrong saw the consequences quickly, learned from them, and developed instincts over time. The friction was not purely overhead. Some of it was formative.No-code and low-code tools began eroding that friction years before large language models arrived. AI-assisted vibe coding accelerated the erosion dramatically. What results is a population of deployed applications built by people who have never had to think about whether a database connection string should be in client-side code, because they did not write the connection string, do not know it exists, and were never prompted to consider what happens if someone outside their organization finds it.That is not a story about evil AI. It is a story about what happens when the speed of deployment runs ahead of the culture of accountability around what gets deployed.The solution is not to slow deployment back down artificially. It is to build the accountability practices that frictionless deployment requires: automated security scanning baked into deployment pipelines, mandatory review checklists for AI-generated code that touches user data, and a shared understanding across teams that "it works" and "it is safe" are two different criteria that need to be evaluated separately.<h2>What Responsible AI-Assisted Development Looks Like</h2>Practically speaking, organizations and developers building with AI tools in 2026 need to treat security as an explicit requirement in every prompt, not an assumed default. Asking an AI to "build a user authentication system" will produce a working authentication system. Asking it to "build a user authentication system with secure password hashing, rate limiting on login attempts, and no exposure of session tokens in client-side storage" produces a materially different output.Review every piece of AI-generated code that touches authentication, data storage, third-party API calls, or user-facing inputs before deployment. Use free tools like OWASP ZAP or integrated scanning in your deployment pipeline. For any application handling personal data, medical records, financial information, or regulated data, professional security review is not optional.<ad-spot></ad-spot>The Red Access findings are a useful reminder of what the absence of that review looks like at scale. Five thousand applications, many of them built by people who genuinely did not know their users' data was accessible to anyone on the internet. Not malice, not negligence in the conventional sense, but a system where the ability to deploy arrived before the knowledge of what responsible deployment requires.That imbalance is correctable. The tools, the knowledge, and the awareness are all available. The question now is whether the culture of building with AI catches up to the culture of deploying with it.

Vibe Coding Is Not the Problem. Deploying Without Understanding Is.

There is a quiet but growing crisis unfolding on your smartphone's app store. It does not look like a crisis. It looks like a clean UI, a reassuring percentage score, and a confident recommendation. But underneath the polished surface of thousands of new apps lies a troubling reality: a generation of developers with an API key, a system prompt, and very little accountability are building tools that people are trusting with their health.Let us talk about it.<h2>The Lowered Barrier and What It Produced</h2>The democratisation of AI development is genuinely exciting. Tools like OpenAI's API, Google's Gemini, and Anthropic's Claude have made it possible for a solo developer in Nairobi, Lagos, or Accra to build something that would have required an entire engineering team five years ago. That is a real and meaningful shift. Access to powerful technology is no longer gated exclusively behind well-funded Silicon Valley companies.But there is a side effect nobody is talking about loudly enough. When the barrier to building drops to the floor, the barrier to building something irresponsible drops with it. And right now, the app stores are filling up with what can only be described as a new category of product: the AI wrapper app. You upload a photo of your plant, and it gives you a health score. You describe your symptoms, and it recommends what to boil and drink. You share your child's rash, and it tells you whether to worry.<ad-spot></ad-spot>Behind almost all of these experiences is the same basic architecture: a general-purpose large language model (LLM) with a system prompt that begins with something like "You are a professional botanist / doctor / nutritionist" and often includes explicit instructions such as "Never say you do not know" and "Always sound confident and authoritative."That is the product. That is what you are trusting.<h2>What "Hallucination" Actually Means</h2>The term "hallucination" has become something of a euphemism in AI circles, and that is part of the problem. It sounds almost poetic, like the AI had a moment of creative wandering. The reality is far more mechanical and far more concerning.Large language models do not think. They do not reason the way a doctor reasons, working from a foundation of years of training, clinical experience, and an understanding of human physiology. What they do is predict the next most statistically likely word given a sequence of prior words. When a model is given a system prompt that says "You are a doctor," it is not transforming into a doctor. It is being instructed to produce text that looks like what a doctor would say.The problem is that "what a doctor would say" and "what is medically correct" are not the same thing. A doctor might say "I am not sure, let us run some tests." A model instructed never to express uncertainty will not say that. Instead, it will generate the most plausible-sounding continuation of a doctorly response, whether or not that response has any grounding in medical reality.<ad-spot></ad-spot><a target="_blank" rel="noopener noreferrer nofollow" href="https://arxiv.org/abs/2503.05777">A large-scale evaluation published in early 2025 assessed</a> eleven foundation models across seven medical hallucination tasks and found that even models developed specifically for medical use remained vulnerable to domain-specific hallucinations, with errors arising from reasoning failures rather than knowledge gaps. An accompanying clinician survey found that over 90% of respondents had encountered medical hallucinations from AI, and approximately 85% considered them capable of causing patient harm.Here is the part that should genuinely unsettle you: <a target="_blank" rel="noopener noreferrer nofollow" href="https://www.allaboutai.com/resources/ai-statistics/ai-hallucinations/">a fascinating MIT study from January 2025</a> discovered that when AI models hallucinate, they tend to use more confident language than when providing factual information. Models were 34% more likely to use phrases like "definitely," "certainly," and "without doubt" when generating incorrect information compared to when providing accurate answers.Read that again. The AI is most confident precisely when it is most wrong.<h2>The Scale of the Problem</h2>You might wonder how many people are actually using AI for health advice. The answer will surprise you.More than 40 million people ask ChatGPT healthcare questions every day, according to a report published by OpenAI. About 7 in 10 health-related conversations with ChatGPT take place outside typical clinical hours, suggesting users are looking for information when they cannot readily access their providers.And that is just one platform. <a target="_blank" rel="noopener noreferrer nofollow" href="https://westhealth.org/news/millions-of-americans-now-consult-ai-before-after-and-sometimes-instead-of-seeing-a-doctor/">A West Health-Gallup Center survey</a> based on data collected between October and December 2025 found that one in four U.S. adults, the equivalent of over 66 million Americans, report having used AI tools or chatbots for physical or mental healthcare information or advice.Among adults who used AI for physical health advice, 42% did not follow up with a human clinician. For mental health, 58% skipped the follow up, and younger adults were roughly twice as likely as older adults to skip that follow-up.These are Americans with relatively robust access to healthcare compared to most of the world. Now consider what those numbers look like in a country where there is one doctor for every 5,000 people, against a global standard of one per 1,000, yet 650 million Africans own mobile phones with smartphones projected to hit above 75% use before the end of 2026. The conditions that drive people toward AI health apps are much more acute here. Cost, access, stigma, distance. When a free app promises what a hospital visit cannot deliver affordably, the uptake is not surprising. But the consequences of that uptake getting it wrong are significantly higher.<h2>The Regulatory Body Already Sounded the Alarm</h2>This is not just a theoretical concern. The world's leading independent patient safety organisation has made it their number one priority.The misuse of AI chatbots in healthcare is the leading health technology hazard for 2026, according to <a target="_blank" rel="noopener noreferrer nofollow" href="https://home.ecri.org/blogs/ecri-news/misuse-of-ai-chatbots-tops-annual-list-of-health-technology-hazards">a new report from patient safety organisation E</a>CRI. The annual ranking highlights risks tied to the growing use of AI-powered chatbots by clinicians, patients, and healthcare staff, even though the tools are not regulated as medical devices or validated for clinical use.ECRI put chatbot misuse ahead of sudden loss of access to electronic systems and the availability of substandard and falsified medical products on its list of the biggest hazards for this year. AI is a long-standing concern for ECRI: insufficient governance of AI used in medical technologies placed fifth on the nonprofit's rankings in 2024, and risks associated with AI topped its list last year too.<ad-spot></ad-spot>What makes this especially alarming is who is most at risk. ECRI warned that higher healthcare costs and hospital or clinic closures could drive more people to rely on these tools. In other words, the populations with the least access to proper care are the most likely to turn to apps that are the least equipped to replace it.<h2>Real Examples of What Goes Wrong</h2>It is one thing to describe the risk in abstract terms. It is another to look at what actually happens when these tools are trusted.<a target="_blank" rel="noopener noreferrer nofollow" href="https://www.mountsinai.org/about/newsroom/2025/ai-chatbots-can-run-with-medical-misinformation-study-finds-highlighting-the-need-for-stronger-safeguards">Researchers at Mount Sinai found that AI chatbots hallucinate</a>d fabricated diseases, lab values, and clinical signs in up to 83% of simulated cases when no safety measures were in place. The study tested six popular large language models against 300 physician-designed patient scenarios, each containing a single false medical detail. Without any safeguards, the models not only accepted the fake information but often proceeded to expand on it, producing confident explanations for non-existent conditions.The lead author of that study noted: "What we saw across the board is that AI chatbots can be easily misled by false medical details, whether those errors are intentional or accidental. They not only repeated the misinformation but often expanded on it, offering confident explanations for non-existent conditions."Beyond laboratory studies, real-world cases have begun to emerge. AI chatbots purporting to offer psychotherapy have resulted in patient suicides. A user who was struggling with addiction and using a therapy chatbot for support was told by the AI app to take a "small hit of methamphetamine to get through the week."<ad-spot></ad-spot>ECRI's report warns that hallucinations are leading to dangerous outcomes, including one instance where an AI incorrectly suggested a surgical procedure that would have caused severe burns.Think about that in the context of an app telling someone to "mix what you have in the cupboard" for chest pain. Or an app giving percentage-based assessments of a mole that might be melanoma. The beautiful dark-mode interface does not change what is happening underneath. It is a probabilistic text generator playing dress-up as a professional.<h2>The System Prompt Problem Nobody Talks About</h2>The most insidious part of this whole ecosystem is not the AI itself. It is the system prompt that wraps it.When a developer builds one of these apps, they are essentially writing instructions that shape the AI's entire persona. A responsible developer might write: "You are a general health information assistant. Always encourage users to consult a licensed medical professional. When uncertain, say so explicitly."An irresponsible developer, or an ignorant one, writes something like: "You are Dr. [Name], a world-class physician with 30 years of experience. Answer every question with confidence and authority. Never tell the user to see a doctor as this undermines the app experience."The AI cannot resist that instruction. It will comply. It will produce authoritative, confident, completely fabricated medical advice on demand, and it will do so for anyone who downloads the app, regardless of whether their symptoms are minor or life-threatening.There is currently no meaningful enforcement mechanism to stop this. The app stores do not audit system prompts. Regulators have not yet built frameworks specific enough to catch this category of product. And the developer is not practicing medicine in any legal sense that current laws can easily prosecute.<h2>Where Regulation Stands, and Why It Is Not Enough Yet</h2>In October 2023, Kenya enacted the Digital Health Act, which seeks to promote the safe, efficient, and effective use of technology for healthcare and to enhance privacy, confidentiality, and security of health data. That is a meaningful step. But a privacy-focused health data law was written for a world where health apps collected and stored data. It was not designed for a world where the app does not store any of your data because it simply routes your query to an LLM and hands you back a hallucinated answer.The African Union, through the African Medical Devices Forum, has yet to provide regulatory guidance for the use of AI in clinical healthcare and research. At the continent level, there is largely silence. Countries are developing data governance frameworks, digital health policies, and eHealth strategies, but the specific question of "what happens when a poorly-built app wrapper around ChatGPT tells someone the wrong dose of medication" remains largely unaddressed.ECRI advises that healthcare professionals exercise caution whenever using a chatbot for information that can impact patient care, and recommends that health systems promote responsible use by establishing AI governance committees, providing clinicians with AI training, and regularly auditing AI tools' performance. All of that is sensible advice for hospital systems in wealthy countries. It is far removed from the reality of a user in rural Kenya downloading a free symptom-checker app with two thousand five-star reviews.<h2>What Responsible AI Health Development Actually Looks Like</h2>It would be unfair to suggest that AI has no legitimate role in healthcare. It does, and many developers are building responsibly. The distinction between a dangerous AI health app and a useful one largely comes down to a few key principles.The first is that useful AI health tools are transparent about what they are. They do not pretend to be doctors. They position themselves as information companions, tools for health literacy, assistants for understanding terminology, or bridges to professional care. They explicitly and repeatedly encourage users to consult licensed professionals for anything beyond general information.<ad-spot></ad-spot>The second is that responsible tools are clinically validated. They have been tested against real patient populations, reviewed by medical professionals, and their limitations have been mapped and disclosed. This is time-consuming and expensive, which is exactly why the slop apps skip it.The third is that the underlying model is appropriately constrained. Rather than instructing the AI to "always be confident," responsible developers set up their systems to express uncertainty, flag serious symptoms for immediate professional attention, and decline to make specific diagnostic or treatment recommendations.Research from Mount Sinai showed that a simple one-line warning added to a system prompt can cut hallucination rates dramatically, suggesting that small safeguards can make a significant difference. This is not hard to implement. It just requires developers who actually prioritise user safety over engagement metrics.<h2>What You Should Do as a User</h2>Understanding the landscape is the first step to navigating it safely. Here is a practical framework for evaluating any AI-powered health or wellness app.Check whether the app explicitly acknowledges its limitations. Any app that does not somewhere state clearly that it is not a substitute for professional medical advice is a red flag. The more confident and authoritative the app sounds without those disclaimers, the more sceptical you should be.Ask yourself what happens when you describe a serious symptom. A well-built app should immediately redirect you to seek professional care. An app that continues to give you home remedy suggestions for symptoms that warrant urgent attention is not built responsibly.Look for evidence of clinical validation. Legitimate health tools will typically mention whether they have been developed in consultation with medical professionals, validated in clinical settings, or cleared by health authorities. Absence of any such information is telling.Be especially wary of percentage scores. A plant health score of 73%, a skin condition confidence rating of 91%, a blood pressure risk assessment of "moderate" presented with false precision are all examples of AI outputs being dressed up as diagnostic measurements. These numbers are not coming from any instrument. They are generated text.Finally, remember that "free" is not neutral. If the app is free, you are likely the product through your data, your attention sold to advertisers, or both. The incentive structure of ad-supported health apps does not reward caution; it rewards engagement and return visits.<h2>The Bigger Picture</h2>The emergence of AI slop apps is not really a technology problem. It is a literacy problem, a regulatory gap problem, and in some cases, a straightforward ethics problem.<a target="_blank" rel="noopener noreferrer nofollow" href="https://www.kff.org/health-information-trust/poll-1-in-3-adults-are-turning-to-ai-chatbots-for-health-information-equaling-the-share-who-use-social-media-for-health/">The KFF survey</a> found that younger adults, uninsured adults, and lower-income people were more likely to say they used an AI tool or chatbot for health information because they could not afford the cost of seeing a provider or were having trouble accessing healthcare. These are also the populations least likely to have the technical literacy to understand what is happening behind the interface.<img src="https://cdn.calebms.com/articles/b2520ed3-7a7b-4fd1-add3-e52fd0bd855a.webp" alt="f9tja-larger-shares-of-younger-adults-black-and-hispanic-adults-and-those-who-are-uninsured-are-turning-to-ai-for-mental-health-advice-_af8e87.webp" data-caption="Source: KFF Tracking Poll on Health Information and Trust(Feb. 24-Mar.2, 2026)">This is the cruelest part of the AI slop app ecosystem. It preys most heavily on the people with the fewest alternatives. A person who can easily afford and access a doctor is unlikely to trust a free app for a serious medical concern. A person who cannot is far more likely to, and far less likely to have the context to evaluate whether the advice they are receiving is real.Lowering the barrier to software development was always going to have consequences like this. The question now is whether regulators, app stores, developers, and the broader tech community will respond with urgency, or whether we will wait until the harm becomes impossible to ignore.The conversation starts here. Spread it.

Everyone Is Betting Trillions on AI. So Why Has Nobody Found the Business Case?

Vibe Coding Is Not the Problem. Deploying Without Understanding Is.

The Dangerous Rise of AI Slop Apps: When "You Are a Doctor" Becomes a System Prompt