ChatGPT Health: The Opportunities, Risks, and Where I Draw the Line

There’s no question that generative AI is reshaping healthcare communications. We see it in content workflows, research synthesis, engagement modeling, and now, more visibly than ever, in consumer-facing health tools.

But just because something can be done with AI doesn’t mean it should be done without serious guardrails.

And this is where I want to be very clear about my own perspective.

I work in AI every day. I train teams on how to use it responsibly. I help brands operationalize it across marketing, communications, and customer engagement.

I would not give OpenAI, or any AI platform, my medical records.

Not now. Not “with safeguards.” Not because a privacy policy says it’s safe.

That line matters. And it matters even more for healthcare marketers.

The Risk No One Wants to Say Out Loud: AI-Generated Diagnoses

One of the most concerning directions we’re heading in is the normalization of AI as a quasi-diagnostic tool. Consider that OpenAI says 40 million people already use ChatGPT for healthcare.

More than 5% of all ChatGPT messages globally are about healthcare, averaging billions of messages each week, according to an OpenAI report. In fact, more than 5% of all ChatGPT messages globally are about healthcare, averaging billions of messages each week.

But, even when platforms say healthcare engagements are “for informational purposes only,” the reality is this:

People already trust AI more than they trust search results, and sometimes more than they trust doctors.

That creates a significant risk.

A survey found that most Americans think AI-generated health information is “somewhat” or “very” reliable, with about 63% reporting reliability, and many respondents saying AI responses often or sometimes give them the answers they need.

In the same survey, AI-generated health information was considered more reliable than social media and influencers, though less trusted than doctors and friends.

But, it’s important to remember: AI systems are exceptionally good at pattern recognition and language generation. They are not accountable for outcomes. They do not understand nuance, lived experience, or the consequences of being wrong in a healthcare context.

When consumers start using AI tools to interpret symptoms, lab results, or treatment options, we edge dangerously close to AI-generated diagnoses by proxy, whether brands intend that or not.

For healthcare marketers, this raises hard questions:

• What responsibility do brands have when their content is surfaced in AI-generated health answers?
• How do we prevent educational content from being interpreted as medical advice?
• Where does brand engagement end and clinical decision-making begin?

These are not theoretical issues. They are already happening.

Why “Safe” AI Still Makes Me Uncomfortable With Health Data

Even with HIPAA-compliant environments, anonymization, and enterprise assurances, there’s a deeper issue that often gets overlooked:

AI systems learn. And learning systems change over time.

Health data is uniquely personal, deeply contextual, and often emotionally charged. Once it enters an AI ecosystem, even in a controlled way, it becomes part of a broader probabilistic system that evolves.

For me, that’s not a technical concern. It’s a trust concern.

Healthcare brands should take note: If someone like me, who understands how these systems work, has reservations, imagine how patients feel once they fully understand what’s happening behind the scenes.

So What Does This Mean for Healthcare Marketers?

Despite these concerns, the answer is not “don’t use AI.” It’s use it differently.

The real opportunity for healthcare marketing lies not in diagnosis or prediction, but in communication, clarity, and connection.

1. AI Will Redefine Digital Health Engagement

AI is already changing how people discover, consume, and interact with health information. Traditional funnels are being replaced by conversational journeys, often mediated by AI systems.

Healthcare marketers must now think about:

• How brand information appears in AI-generated responses
• Whether messaging holds up when summarized, remixed, or paraphrased by models
• How trust is conveyed when there is no direct brand touchpoint

This is not SEO. It’s not content marketing as we’ve known it. It’s AI-mediated engagement.

I’ve conducted many GEO audits for pharma and healthcare brands and you may be surprised how much of your brand’s old, inaccurate and outdated information appears authoritatively in LLM responses to healthcare prompts.

It’s more important now than ever to audit your brand’s presence in these AI models and optimize the right content so your brand’s visibility is accurate and useful, especially as ChatGPT Health and other LLMs focus on healthcare content.

2. Education Will Matter More Than Ever

As AI becomes the front door to health information, brands have an opportunity, and an obligation, to elevate the quality of what enters the system.

That means:

• Clear educational boundaries
• Plain-language explanations without oversimplification
• Strong differentiation between awareness, education, and treatment

The brands that win trust will be the ones that help people understand, not just click.

3. Governance Is Now a Marketing Issue

AI governance is no longer just an IT or legal concern. It directly affects brand reputation, credibility, and risk exposure.

Healthcare marketing teams should be asking:

• What datasets are our AI tools trained on?
• Who approves AI-assisted content, and how?
• How do we document intent, limitations, and review processes?

Well-governed AI doesn’t slow teams down. It protects them.

The Line Between Engagement and Ethics

AI gives healthcare marketers extraordinary leverage, but also extraordinary responsibility.

The goal should never be to replace clinicians, shortcut diagnosis, or simulate medical authority. The goal should be to:

• Reduce confusion
• Improve access to understandable information
• Support informed conversations between patients and providers

When AI stays in its lane, it can be transformative. When it crosses into decision-making without accountability, it becomes dangerous.

My Bottom Line

AI will absolutely shape the future of healthcare marketing and communications. There’s no avoiding that.

But trust, not technology, will determine which brands thrive.

We need to be honest about risks.

We need to draw clear ethical boundaries.

And we need to design AI-powered engagement that respects the gravity of health decisions.

Because in healthcare, being “innovative” is meaningless if you’re not also being responsible.

---

Remember, AI won’t take your job. Someone who knows how to use AI will. Upskilling your team today, ensures success tomorrow. Custom in-person and virtual trainings are available. If you’re looking for something more top-level to jump start your team’s interst in AI, we offer one-hour Lunch-and-Learns. If you’re planning your next company offsite, our half-day workshops are as fun as they are informational. And, of course, we offer AI consulting and support with custom prompt libraries, or AISO/GEO strategies. Whatever your needs, we are your partner in AI success.