{"id":6805,"date":"2025-05-15T18:31:31","date_gmt":"2025-05-15T18:31:31","guid":{"rendered":"https:\/\/nationalgunowner.org\/index.php\/2025\/05\/15\/chatgpt-used-for-medical-advice-openai-launches-healthbench-tool\/"},"modified":"2025-05-15T18:31:42","modified_gmt":"2025-05-15T18:31:42","slug":"chatgpt-used-for-medical-advice-openai-launches-healthbench-tool","status":"publish","type":"post","link":"https:\/\/nationalgunowner.org\/index.php\/2025\/05\/15\/chatgpt-used-for-medical-advice-openai-launches-healthbench-tool\/","title":{"rendered":"ChatGPT Used for Medical Advice, OpenAI Launches HealthBench Tool"},"content":{"rendered":"
\n
Are you asking ChatGPT for medical advice? Ahmed\/Unsplash<\/span><\/figcaption><\/figure>\n

More people around the world are turning to ChatGPT for medical advice when feeling unwell or anxious\u2014sometimes in place of seeing a doctor.\u00a0A study released in August 2024 by the University of California found that patients often consult A.I. models like ChatGPT<\/a> either before or after speaking with a physician. Similarly, a February study from the University of Sydney, which surveyed more than 2,000 adults, reported that nearly six in ten respondents<\/a> had asked ChatGPT at least one high-risk health question\u2014queries that would typically require professional clinical input.<\/p>\n

\n<\/section>\n

ChatGPT\u2019s appeal is obvious: it\u2019s free, easy to access and provides answers in seconds. But how safe is this instant feedback? That\u2019s the question OpenAI<\/a> hopes to answer with HealthBench, a new evaluation tool designed to test how accurate and reliable A.I. models are when responding to health-related questions.<\/p>\n

HealthBench was developed with the help of 262 licensed physicians from 60 countries and evaluates model performance across 5,000 realistic medical conversations using doctor-written criteria. It simulates back-and-forth chats between users and A.I. assistants on a wide range of health concerns, from symptom checks to determining when emergency care is needed. Each A.I. response is assessed against detailed guidelines outlining what a good medical answer should contain, what it should avoid and how it should be communicated. OpenAI\u2019s GPT-4.1 model is used to score the responses.<\/p>\n

Dr. Ran D. Anbar<\/a>, a pediatric pulmonologist and clinical hypnotist at Center Point Medicine in California, says it\u2019s no surprise that patients turn to A.I. when health care access feels out of reach. \u201cPatients may defer consultation with a health care provider because they\u2019re satisfied with the answers they get from ChatGPT, or because they want to save money,\u201d he told Observer.<\/p>\n

However, he cautions that this convenience comes with serious risks. \u201cUnfortunately, it is completely predictable that patients will be harmed and perhaps even die because of delays in seeking appropriate medical treatment, thinking ChatGPT\u2019s guidance is sufficient,\u201d he warned.<\/p>\n

Which A.I. model is the best at answering health care questions?<\/h3>\n

According to OpenAI\u2019s evaluation, its newest model, o3, performed best on HealthBench, scoring 60 percent. It was followed by xAI<\/a>\u2019s Grok at 54 percent and Google\u2019s Gemini 2.5 Pro at 52 percent.<\/p>\n

GPT-3.5 Turbo\u2014the model that powers the free version of ChatGPT\u2014scored only 16 percent. Notably, OpenAI\u2019s GPT-4.1 Nano model outperformed older large models while being about 25 times cheaper to run, suggesting that newer A.I. tools are becoming both smarter and more efficient.<\/p>\n

Still, Dr. Anbar notes that even as the health care system remains overburdened and often slow to respond in urgent situations, A.I. tools like ChatGPT are unreliable. \u201cThe apparent certainty with which it provides its responses can be misleadingly reassuring,\u201d he said.<\/p>\n

OpenAI acknowledged these limitations in its blog post announcing HealthBench, noting that its models can still make critical errors, particularly when handling vague or high-stakes queries. In medicine, even one incorrect answer can outweigh dozens of accurate ones. Given ChatGPT\u2019s widespread use for health-related questions, OpenAI\u2019s latest safeguards may be a step forward\u2014but whether they\u2019re enough remains an open question.<\/p>\n

\t\t\t\t\"ChatGPT<\/p><\/div>\n