Dapatkah Large Language Models Melakukan Triase Kasus Layanan Primer di Indonesia Dengan Aman?

Muhammad Sobri Maulana; Dwitia Pratiwi; Arditya Prayogi

Authors

Muhammad Sobri Maulana Rumah Sakit Angkatan Udara Prof. dr. Abdulrachman Saleh Jakarta Author
Dwitia Pratiwi Rumah Sakit Angkatan Udara Prof. dr. Abdulrachman Saleh Jakarta Author
Arditya Prayogi Universitas Islam Negeri K.H. Abdurrahman Wahid Pekalongan Author

Keywords:

Large Language Models, Triase, Layanan Primer, Keselamatan Pasien, Bahasa Indonesia

Abstract

Model bahasa besar atau large language models (LLMs) berpotensi mendukung triase yang berhadapan langsung dengan pasien, tetapi bukti keselamatannya masih didominasi konteks emergency department berbahasa Inggris dan belum menjawab kebutuhan layanan primer Indonesia. Penelitian ini mengusulkan benchmark empat tingkat untuk self-care, konsultasi rutin, penilaian urgent pada hari yang sama, dan rujukan emergency menggunakan 240 vignette klinis berbahasa Indonesia. Empat LLM anonim dievaluasi melalui exact triage accuracy, macro F1, weighted kappa, high-acuity sensitivity, under-triage, unsafe under-triage, serta robustness terhadap variasi redaksi. Untuk mendemonstrasikan pipeline analitik sebelum eksperimen prospektif, seluruh hasil numerik pada artikel ini disimulasikan. Pada simulasi, exact accuracy berkisar 60,4%–78,3%; model terbaik mencapai macro F1 78,4%, weighted kappa 0,886, high-acuity sensitivity 92,5%, dan unsafe under-triage 0,8%. Performa menurun ketika ambiguitas vignette meningkat dan kegagalan penting terkonsentrasi pada presentasi atipikal atau tidak lengkap. Temuan demonstratif ini menunjukkan bahwa benchmark triase perlu memprioritaskan metrik sensitif terhadap bahaya, bukan accuracy semata, dan memerlukan validasi Indonesia dengan versi model terkunci, adjudikasi klinisi, serta ambang keselamatan yang diprespesifikasi sebelum deployment patient-facing.

Downloads

Download data is not yet available.

References

Alomari, L. M., et al. (2025). Safety and accuracy of AI in triaging patients in the emergency department: A prospective observational study. [PubMed PMID: 41266963].

Arslan, B., et al. (2025). A comparative study of ChatGPT Plus, Copilot Pro, and nurses in emergency triage. [PubMed PMID: 39731895].

Atmaji, W. A., Kartika, A. P. T., Akbar, R., & Dwina, E. (2026). Analisis keakuratan Emergency Severity Index (ESI) berdasarkan kecerdasan buatan ChatGPT dan Gemini: Studi retrospektif pada pasien IGD di Indonesia. Jurnal Ners, 10(2), 5500–5508.

Cahyawijaya, S., Winata, G. I., Wilie, B., Vincentio, K., Li, X., Kuncoro, A., Ruder, S., Lim, Z. Y., Bahar, S., Khodra, M. L., Purwarianti, A., & Fung, P. (2021). IndoNLG: Benchmark and resources for evaluating Indonesian natural language generation. Proceedings of EMNLP 2021. https://arxiv.org/abs/2104.08200

Goh, E., Gallo, R., Hom, J., Strong, E., Weng, Y., Kerman, H., Cool, J. A., Kanjee, Z., Parsons, A. S., Ahuja, N., Horvitz, E., Yang, D., Milstein, A., Olson, A. P. J., Rodman, A., & Chen, J. H. (2024). Large language model influence on diagnostic reasoning: A randomized clinical trial. JAMA Network Open, 7(10), e2440969. https://doi.org/10.1001/jamanetworkopen.2024.40969

Hager, P., Jungmann, F., Holland, R., et al. (2024). Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature Medicine, 30(9), 2613–2622. https://doi.org/10.1038/s41591-024-03097-1

Haim, G. B., Saban, M., Barash, Y., Cirulnik, D., Shaham, A., Eisenman, B. Z., Burshtein, L., Mymon, O., & Klang, E. (2024). Evaluating large language model-assisted emergency triage: A comparison of acuity assessments by GPT-4 and medical experts. Journal of Clinical Nursing. https://doi.org/10.1111/jocn.17490

Levine, D. M., et al. (2024). The diagnostic and triage accuracy of the GPT-3 family of large language models compared with physicians and laypeople. [PubMed PMID: 39059888].

Liu, S., Wright, A. P., McCoy, A. B., Huang, S. S., Steitz, B., & Wright, A. (2025). Detecting emergencies in patient portal messages using large language models and knowledge graph-based retrieval-augmented generation. Journal of the American Medical Informatics Association, 32(6), 1032–1039. https://doi.org/10.1093/jamia/ocaf059

Masanneck, L., Schmidt, L., Seifert, A., Kölsche, T., Huntemann, N., Jansen, R., Mehsin, M., Bernhard, M., Meuth, S. G., Böhm, L., & Pawlitzki, M. (2024). Triage performance across large language models, ChatGPT, and untrained doctors in emergency medicine: Comparative study. Journal of Medical Internet Research, 26, e53297. https://doi.org/10.2196/53297

McDuff, D., Schaekermann, M., Tu, T., Palepu, A., Wang, A., Garrison, J., et al. (2025). Towards accurate differential diagnosis with large language models. Nature, 642(8067), 451–457. https://doi.org/10.1038/s41586-025-08869-4

Maulana, M. S., Prayogi, A., & Pratiwi, D. (2026). AI-DRIVEN VOCABULARY PERSONALIZATION: FILLING THE GAP IN TRANSPARENCY AND LEARNER TRUST IN ADAPTIVE RECOMMENDER SYSTEMS FOR LANGUAGE LEARNING. Mandailing Journal of Education and Sciences, 1(2), 62-67.

Maulana, M. S., Pratiwi, D., & Prayogi, A. (2026). RadOnco-Priority: Machine Learning Decision Support for Radiotherapy Queue Prioritization Using Real-World Retrospective Radiotherapy Referral Data. SAGA: Journal of Technology and Information System, 4(2), 607-615.

Maulana, M. S., Pratiwi, D., & Prayogi, A. (2026). Vaccination programs and infectious disease burden among military personnel: a systematic review with pathogen-specific quantitative synthesis. The ASEAN Journal of Military and Preventive Medicine, 3(2).

Omar, M., et al. (2025). Sociodemographic biases in medical decision making by large language models: Evaluation across 1,000 emergency department cases. [PubMed PMID: 40195448].

Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., Payne, P., Seneviratne, M., Gamble, P., Kelly, C., Babiker, A., Schärli, N., Chowdhery, A., Mansfield, P., Demner-Fushman, D., Agüera y Arcas, B., Webster, D., Corrado, G. S., Matias, Y., Chou, K., Gottweis, J., Tomasev, N., Liu, Y., Rajkomar, A., Barral, J., Semturs, C., Karthikesalingam, A., & Natarajan, V. (2023). Large language models encode clinical knowledge. Nature, 620(7972), 172–180. https://doi.org/10.1038/s41586-023-06291-2

Wibowo, B. D. P., & Anggraini, S. N. P. (2026). Pengembangan sistem pendukung keputusan cerdas untuk seleksi karyawan berbasis multi-kriteria dan machine learning. JINTEN: Journal of Intelligent Systems and Digital Science, 1(1), 37–50.

Williams, C. Y. K., Miao, B. Y., Kornblith, A. E., & Butte, A. J. (2024a). Use of a large language model to assess clinical acuity of adults in the emergency department. JAMA Network Open, 7(5), e248895. https://doi.org/10.1001/jamanetworkopen.2024.8895

Williams, C. Y. K., Miao, B. Y., Kornblith, A. E., & Butte, A. J. (2024b). Evaluating the use of large language models to provide clinical recommendations in the emergency department. Nature Communications, 15, 8236. [PubMed PMID: 39379357].

Xu, L., Zhao, W., & Huang, X. (2025). Diagnosis and triage performance of contemporary large language models on short clinical vignettes. Journal of Medical Systems, 49, 141. https://doi.org/10.1007/s10916-025-02284-y

Yazaki, M., Maki, S., Furuya, T., Nakada, T., & Ohtori, S. (2025). Emergency patient triage improvement through a retrieval-augmented generation-enhanced large language model. [PubMed PMID: 38950135].

Yudhistira, P. C. Y., Malik, D. R., & Yudistira, N. (2026). Does language shift break medical vision-language models? Indonesian radiology visual question answering case study. arXiv:2606.03693.