Open AI’s ChatGPT has reportedly enabled to detection of the symptoms correctly, medical conditions are diagnosed with 49% of complex cases correctly, however, the remaining 61% of cases still lack the result to be sure. This has come as a limitation of ChatGPT, this reveals that the famous AI language model does answer medical questions, but struggles in diagnosing complex medical cases.
Therefore, the study is to be focused on enhancing the effectiveness of ChatGPT as a diagnostic tool for complex medical cases. The main motive aims at increasing the accuracy to diagnose conditions and responding with relevant treatment options. The researchers tested ChatGPT150 Medscape Clinical Challenges published after August 2021, ensuring the AI had no prior knowledge of multiple health-related issues, these cases include detailed patient history, examination findings, and diagnostic tests. ChatGPT’s responses were compared to the correct answers and the choices made by medical professionals using the same cases.
Furthermore, ChatGPT is to be revealed as coming with significant shortcomings in AI’s diagnostic capabilities. The overall accuracy it holds of 74% with a precision of 49%. This states the AI to struggle in correctly identify diagnoses. However, ChatGPT can eliminate wrong answers effectively but lacks the reliability to pinpoint the correct diagnosis consistently.
ChaatGPT’s response towards the quality of the information provided has covered (52%) half of the answers considered to have a low cognitive load, easy to understand, the following 49% required moderate cognitive effort, and 7% were deemed highly complex. The study reveals that ChatGPT can generate grammatically correct answers, for critical ones necessary details to be needed for accurate diagnosis.
Moreover, the major issue highlighted the diagnosing performance such as its training data, which may lack some specialized medical knowledge, it shows the advancement level of ChatGPT to come up with the latest medical upgrades. This unnecessary inaccuracy will lead to missed diagnoses and false treatments. AI “hallucinations” generate model sounds with incorrect information.
Meanwhile, ChatGPT shows potential as a supplementary tool for medical learners, however, its current limitations make it unsuitable as a standalone diagnostic resource. Dealing with the complexity of real-world medical cases requires a lot of AI ability to ensure relevant and significant information is provided.