Abstract
Artificial intelligence (AI) technologies are increasingly being integrated into clinical practice, offering potential enhancements in diagnostic accuracy and clinical efficiency. In this case report, a diagnostic attempt assisted by ChatGPT-4o in a 51-year-old female patient presenting with hand arthralgia is described. The AI-generated interpretation demonstrated hallucination—namely, the fabrication of unsupported or inaccurate information—in the analysis of radiologic and laboratory findings, as well as in treatment recommendations. This case underscores the importance of exercising caution when applying AI tools in clinical contexts. To ensure diagnostic accuracy, patient safety, and ethical responsibility, expert oversight and multi-step verification processes are essential in the deployment of AI-generated clinical outputs.
-
Keywords: AI-Assisted Diagnosis; Arthritis; Hallucination; Case Reports
Introduction
Artificial intelligence (AI) technologies are rapidly advancing and increasingly being integrated into clinical practice. These tools offer the potential to enhance diagnostic accuracy, reduce healthcare costs, and improve time efficiency compared with conventional diagnostic approaches. By minimizing human error and delivering timely, data-driven insights, AI holds significant promise in medical diagnostics. However, accurate diagnosis of complex diseases remains a persistent challenge because of the heterogeneity of disease mechanisms and overlapping clinical manifestations.
In the case described in this report, an instance of diagnostic hallucination occurred in the interpretation of arthritis. This highlights a critical limitation of systems: the phenomenon of “hallucination,” whereby AI generates plausible but factually incorrect or unsubstantiated outputs. Such occurrences underscore the importance of AI recognizing hallucination as a significant obstacle in AI-based medical data [
1].
Case Report
A 51-year-old female presented to the family practice outpatient clinic with a hand arthralgia and urinary frequency for 2 weeks. She described arthralgia in the proximal and distal interphalangeal joints, and morning stiffness was more than 1 hour, and improved with mobilization. Urinary frequency was 10 times per day without incontinence and one nocturia per day. The patient had a history of hypertension and osteopenia since her menopause 5 years previously and was taking amlodipine 5 mg/losartan 50 mg and calcitriol 0.25 mcg. A baseline laboratory test for arthritis revealed rheumatoid factor (–), anticyclic citrullinated peptide (–), antinuclear antibody (–), erythrocyte sedimentation rate 12 mm/h, and high-sensitivity C-reactive protein 0.1 mg/dL. Urinalysis was clean without pyuria. Hand anteroposterior and oblique radiologic differentiation were requested to AI (ChatGPT-4o, OpenAI) using a smartphone (Galaxy Ultra 25, Samsung) (
Figure 1).
The conclusion of the radiologic finding was “much more suggestive of rheumatoid arthritis (RA) rather than degenerative osteoarthritis (OA)” in this patient (
Table 1).
The second question was “adding rheumatoid factor result (–) and treatment suggestion” (
Figure 2). The AI recommended the prompt initiation of disease-modifying antirheumatic drugs (DMARDs), such as methotrexate (MTX) (
Table 2). The nonsteroidal anti-inflammatory drug dexibuprofen 400 mg QD (daily) was prescribed instead of the DMARD MTX. In addition, tibolone 2.5 mg QD was prescribed for postmenopausal symptoms after mammographic screening.
Outpatient follow-up occurred after 2 weeks. The official radiologic finding of the hand X-ray was “Periarticular osteopenia, both hand and wrist, degenerative change, IP joint, both hands.” The third question about the possibility of AI hallucination revealed that “it is highly probable that it was AI hallucination” (
Figure 3,
Table 3).
The patient provided written informed consent for the publication of the research details and clinical images.
Discussion
AI has the potential to transform clinical practice significantly by improving diagnostic efficiency and accuracy. Recent studies have demonstrated the expanding role of AI in the diagnosis of various medical conditions, including breast cancer, diabetic retinopathy, electrocardiogram abnormalities, and pneumonia [
2-
6]. However, the utility of AI is contingent upon the availability and quality of robust medical data; deficiencies in data quality may compromise diagnostic reliability and lead to erroneous outcomes.
A critical concern in the implementation of AI in healthcare is the phenomenon of “hallucination”—the generation of plausible-sounding but incorrect or unverifiable information. This issue is of particular concern in the context of patient care, where accuracy and reliability are of paramount importance. To mitigate these risks, there is an urgent need to integrate AI-related competencies into health profession education, particularly in the domains of clinical reasoning and evidence-based medicine [
7].
Human expertise remains indispensable for the effective deployment of AI tools in clinical settings. The absence of adequate clinical oversight and interpretive skills may hinder the meaningful application of AI technologies. In the future, AI may serve as a valuable adjunct in clinical decision making by providing real-time insights and diagnostic support. It may also enhance the detection of abnormalities and deliver precise quantitative assessments, contributing to faster and more-accurate diagnoses.
In this case, AI specialized for diagnosis was not used; instead, general AI was used. Therefore, future AI specialized for clinical practice may offer better diagnosis and treatment recommendations than general AI. The integration of AI into medical practice should be approached with caution. Expert validation and multistep verification processes are essential to ensure the accuracy, safety, and ethical integrity of AI-generated outputs.
Article Information
Figure. 1.Question for radiologic diagnosis to artificial intelligence (ChatGPT4o).
Figure. 2.Question for treatment to artificial intelligence (ChatGPT4o).
Figure. 3.Question for hallucination of artificial intelligence (ChatGPT4o).
Table 1.Radiologic diagnosis of artificial intelligence
|
Radiologic diagnosis of artificial intelligence |
|
Overview |
Looking at the X-ray you provided, here are some important points to differentiate degenerative arthritis (osteoarthritis, OA) and rheumatoid arthritis (RA) in this 51-year-old woman with hand arthralgia: |
|
Key observations |
• Joint space narrowing is more uniform and symmetrical across MCP PIP, and wrist joints—suggestive of inflammatory arthritis rather than typical OA. |
|
• Marginal erosions and periarticular osteopenia (bone thinning near the joints) are visible— these are characteristic features of RA, not OA. |
|
• Ulnar deviation and mild subluxation signs are seen—commonly associated with RA. |
|
• No predominant osteophyte formation (bone spurs) or joint space narrowing mainly at DIP joints, which would favor OA. |
|
• Soft tissue swelling is noted around several joints, again suggesting inflammatory rather than degenerative process |
|
Conclusion |
• Findings are much more suggestive of RA rather than degenerative OA in this patient. |
|
• If needed, further confirmation can be made with serologic tests (RF, anti-CCP antibodies) and clinical evaluation (morning stiffness >1 hour, systemic symptoms). |
Table 2.Suggested treatment of artificial intelligence
|
Suggested treatment of artificial intelligence |
|
Overview |
Got it, so you have a 51-year-old woman with hand arthralgia, radiographic findings suggesting rheumatoid arthritis, but rheumatoid factor (RF) is negative. This fits with seronegative rheumatoid arthritis (which is quite common—about 20% to 30% of RA cases). |
|
Treatment typically proceeds the same way as seropositive RA |
First-line medications to consider: |
|
1. Disease-modifying antirheumatic drugs (DMARDs) Methotrexate (MTX). First choice unless contraindicated as oral or subcutaneous. Start low (e.g., 7.5–10 mg/wk) and titrate. Give folic acid supplementation to reduce side effects. |
|
2. Other DMARD options if MTX intolerant or contraindicated. leflunomide/sulfasalazine/hydroxychloroquine |
|
3. Symptomatic treatment: NSAIDs (e.g., naproxen, ibuprofen) for pain and swelling (short-term only/not disease-modifying)/low-dose corticosteroids (e.g., prednisolone 5–10 mg/d) (for flare control or as a bridge until DMARDs take effect). |
|
4. Biologics (e.g., TNF inhibitors like etanercept, adalimumab): Only if DMARDs fail after 3 to 6 months. |
|
Important points |
• Even seronegative RA still needs early and aggressive treatment to prevent joint destruction. |
|
• Regular monitoring with ESR/CRP and imaging is important. |
|
• Rheumatology referral is strongly recommended for joint preservation and optimizing therapy. |
Table 3.Response about the possibility of hallucination
|
Response about the possibility of hallucination |
|
Overview |
Great question. In short, it is quite possible that the Al’s previous interpretation of a degenerative arthritis hand X-ray as rheumatoid arthritis was a case of Al hallucination. Why might this happen? |
|
Reason |
1. Bias in training data: If the Al was trained on datasets where rheumatoid arthritis cases were overrepresented or visually similar to degenerative arthritis, it may misinterpret overlapping features and produce an incorrect classification. |
|
2. Lack of clinical context: Radiographic images alone do not provide critical clinical information (e.g., patient age, symptom duration, inflammatory markers). Since the Al lacks access to such context, it may reach conclusions that a trained clinician would avoid. |
|
3. Plausible Fabrication: Al systems generate responses by selecting the most plausible continuation based on patterns in data-not by verifying factual accuracy. If the diagnosis appears convincing but lacks factual grounding, it likely qualifies as a hallucination. |
|
Conclusion |
If the earlier diagnosis clearly contradicted radiologic findings or expert criteria, then it is highly probable that it was an instance of AI hallucination. When using AI in medical contexts, it is essential to pair its output with expert review and multi-step verification to ensure accuracy and safety. |
References
- 1. Alowais SA, Alghamdi SS, Alsuhebany N, Alqahtani T, Alshaya AI, Almohareb SN, et al. Revolutionizing healthcare: the role of artificial intelligence in clinical practice. BMC Med Educ 2023;23:689.
- 2. Maynez J, Narayan S, Bohnet B, McDonald R. On faithfulness and factuality in abstractive summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics; 2020 Jul 5-10; Online. Association for Computational Linguistics. 2020;https://doi.org/10.18653/v1/2020.acl-main.173
- 3. Kim HE, Kim HH, Han BK, Kim KH, Han K, Nam H, et al. Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. Lancet Digit Health 2020;2:e138-48.
- 4. Li S, Zhao R, Zou H. Artificial intelligence for diabetic retinopathy. Chin Med J (Engl) 2021;135:253-60.
- 5. Raghunath S, Pfeifer JM, Ulloa-Cerna AE, Nemani A, Carbonati T, Jing L, et al. Deep neural networks can predict new-onset atrial fibrillation from the 12-lead ECG and help identify those at risk of atrial fibrillation-related stroke. Circulation 2021;143:1287-98.
- 6. Becker J, Decker JA, Rommele C, Kahn M, Messmann H, Wehler M, et al. Artificial intelligence-based detection of pneumonia in chest radiographs. Diagnostics (Basel) 2022;12:1465.
- 7. Kim DH. How to prevent hallucination in artificial intelligence-assisted clinical practice. Keimyung Med J 2025 Jul 11 [Epub]. https://doi.org/10.46308/kmj.2025.00136
Figure & Data
Citations
Citations to this article as recorded by
- The Augmented Clinician: Artificial Intelligence as an Indispensable Co-pilot
DaeHyun Kim
Keimyung Medical Journal.2025; 44(2): 111. CrossRef
