As large language models (LLMs) rapidly enter the healthcare sector, artificial intelligence is evolving beyond providing general health information and is beginning to interact directly with patients’ most sensitive personal health data.<\/p>\n\n\n\n
A recent Viewpoint article in JAMA, “When Patients Share Everything With an AI Chatbot: Risks and Opportunities of Large Language Models<\/a>,”<\/em> examines the opportunities and challenges posed by a new generation of health-focused AI platforms such as ChatGPT Health. The authors argue that as patients gain the ability to synchronize their complete electronic health records (EHRs) with AI systems, healthcare is moving from the democratization of medical knowledge toward the democratization of medical data.<\/p>\n\n\n\n While this transformation has the potential to advance personalized medicine, it also raises unprecedented concerns regarding privacy, bias, and regulatory oversight.<\/p>\n\n\n\n In theory, if AI systems can integrate patients’ medical histories, health monitoring data, wearable device information, and lifestyle records, they may provide several important benefits:<\/p>\n\n\n\n *More personalized health recommendations<\/strong><\/p>\n\n\n\n *Earlier identification of rare diseases<\/strong><\/p>\n\n\n\n *Enhanced epidemic and public health surveillance<\/strong><\/p>\n\n\n\n *Greater utilization of Real-World Data (RWD)<\/strong><\/p>\n\n\n\n *Accelerated clinical research and drug development<\/strong><\/p>\n\n\n\n For researchers, large-scale and real-time integration of health data may become a critical foundation for the future of precision medicine.<\/p>\n\n\n\n However, the authors highlight an important reality: <\/p>\n\n\n\n Electronic health records are not neutral repositories of facts.<\/p>\n\n\n\n In addition to objective laboratory and diagnostic findings, medical records often contain subjective assessments and interpretations made by healthcare professionals.<\/p>\n\n\n\n Examples include:<\/p>\n\n\n\n 1. Behavioral descriptions of patients<\/strong><\/p>\n\n\n\n 2. Preliminary diagnostic impressions<\/strong><\/p>\n\n\n\n 3. Unconfirmed clinical assumptions<\/strong><\/p>\n\n\n\n 4. Potentially biased language or documentation<\/strong><\/p>\n\n\n\n If AI systems learn directly from these records, they may reproduce\u2014or even amplify\u2014existing biases.<\/p>\n\n\n\n For example, a patient suffering from chronic pain may have previously been labeled as “drug-seeking.” Even if subsequent evaluations confirm a legitimate physiological cause for the pain, an AI system may still be influenced by earlier documentation and provide less appropriate recommendations.<\/p>\n\n\n\n In other words, AI systems may learn not only medical knowledge but also the biases embedded within healthcare systems.<\/p>\n\n\n\n The authors further note that healthcare systems already face significant health disparities.<\/p>\n\n\n\n Patients from different racial, ethnic, socioeconomic, and demographic backgrounds may experience unequal diagnosis and treatment.<\/p>\n\n\n\n If such disparities are reflected in EHR data and AI systems treat these records as objective truth, future applications may generate:<\/p>\n\n\n\n *Biased diagnoses<\/strong><\/p>\n\n\n\n *Biased recommendations<\/strong><\/p>\n\n\n\n *Biased risk assessments<\/strong><\/p>\n\n\n\n As a result, existing healthcare inequities could become further entrenched.<\/p>\n\n\n\n Therefore, the risks associated with AI may stem not only from the model itself but also from the data used to train and inform it.<\/p>\n\n\n\n Another key issue discussed in the article is data privacy.<\/p>\n\n\n\n Many patients assume that their medical information remains protected under the Health Insurance Portability and Accountability Act (HIPAA).<\/p>\n\n\n\n However, the authors point out that once patients voluntarily upload their medical records to a commercial AI platform, those data may no longer be fully protected under HIPAA.<\/p>\n\n\n\n The reason is that most AI platforms are not considered HIPAA-covered entities.<\/p>\n\n\n\n Consequently:<\/p>\n\n\n\n 1. HIPAA restrictions on data use may not apply.<\/strong><\/p>\n\n\n\n 2. HIPAA security requirements may not apply.<\/strong><\/p>\n\n\n\n 3. HIPAA breach notification obligations may not apply.<\/strong><\/p>\n\n\n\n Although AI companies may promise strong privacy protections, corporate privacy policies are fundamentally different from legally enforceable regulatory safeguards.<\/p>\n\n\n\n According to the authors, the most significant challenge may not be data breaches, but rather the lack of transparency.<\/p>\n\n\n\n Currently, independent researchers have limited ability to evaluate:<\/p>\n\n\n\n *Whether AI systems exhibit bias against specific populations<\/strong><\/p>\n\n\n\n *Whether safety incidents have occurred<\/strong><\/p>\n\n\n\n *Whether inappropriate medical recommendations are being generated<\/strong><\/p>\n\n\n\n *Whether AI is influencing patients’ healthcare-seeking behavior<\/strong><\/p>\n\n\n\n Because these data remain under the control of platform developers, external validation is often impossible.<\/p>\n\n\n\n As a result, even when AI companies report strong performance, there may be insufficient independent evidence to verify such claims.<\/p>\n\n\n\n Health-focused AI platforms such as ChatGPT Health represent an important milestone in the evolution of AI-powered healthcare. For the first time, patients may be able to provide AI systems with comprehensive health records for personalized analysis.<\/p>\n\n\n\n However, when AI begins reading entire medical records, it receives not only information about diseases and treatments but also decades of accumulated clinical assumptions, documentation biases, and systemic healthcare challenges.<\/p>\n\n\n\n For this reason, the future development of healthcare AI should not focus solely on technological innovation. Equal attention must be given to data governance, clinical evidence generation, regulatory oversight, and ongoing performance monitoring.<\/p>\n\n\n\n Only through robust safeguards can AI become a tool for improving healthcare outcomes rather than amplifying existing inequities and risks.<\/p>\n\n\n\n Reference: When Patients Share Everything With an AI Chatbot\u2013Risks and Opportunities of Large Language Models<\/a><\/p>\n\n\n\n Reviewer: PI-Union Medical Science Ltd.<\/a><\/strong><\/p>\n\n\n\n * E-mail: piunion@pi-union.com<\/p>\n\n\n\n * Official Website: https:\/\/pi-union.com\/<\/a><\/p>\n\n\n\n * Facebook: www.facebook.com\/piunion2020<\/a><\/p>\n\n\n\n * Youtube: www.youtube.com\/@pi-union<\/a><\/p>\n\n\n\nWhat Is the Potential Value of AI Access to Complete Medical Records?<\/h4>\n\n\n\n
Medical Records Are Not Entirely Objective<\/h4>\n\n\n\n
AI May Reinforce Existing Health Disparities<\/h4>\n\n\n\n
Can HIPAA Protect Patient Data Uploaded to AI Platforms?<\/h4>\n\n\n\n
Lack of Transparency May Be the Greater Concern<\/h4>\n\n\n\n
\n\n\n\nPI-Union Medical Science Commentary<\/h3>\n\n\n\n
\n\n\n\n