Explainable Clinical Text Mining for Treatment Response Extraction in Lung Cancer Notes
Main Article Content
Abstract
The extraction of treatment response information from unstructured clinical notes is a critical task in oncology research and clinical practice, particularly for lung cancer, where therapeutic paradigms evolve rapidly. Despite the promising capabilities of advanced natural language processing models, their opaque nature significantly limits clinical adoption due to a lack of interpretability and trustworthiness. This paper investigates the development and implementation of an explainable clinical text mining framework explicitly designed to extract treatment response trajectories from unstructured electronic health records pertaining to lung cancer cohorts. By integrating transformer-based language models with robust explainability techniques, the proposed methodology aims to bridge the gap between high predictive performance and clinical transparency. The analysis encompasses an exhaustive review of current natural language processing applications in oncology, a detailed explication of the architectural design prioritizing attention-based interpretability, and a rigorous evaluation against traditional black-box methodologies. The findings demonstrate that augmenting extraction models with explanatory mechanisms not only maintains state-of-the-art accuracy but also significantly enhances clinician trust by providing verifiable linguistic evidence for every automated extraction. This research underscores the necessity of transparent artificial intelligence in high-stakes medical domains and provides a scalable blueprint for deploying explainable text mining systems in real-world oncology workflows.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in the Journal of the Artificial Intelligence Engineers Consortium (JAIEC) are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
This license permits unrestricted use, distribution, reproduction, and adaptation in any medium, provided the original author(s) and source are properly credited.
Authors retain copyright of their work while granting the journal the right of first publication.
References
1. Ren, Y., Wu, D., Tong, Y., López-DeFede, A., & Gareau, S. (2023). Issue of data imbalance on low birthweight baby outcomes prediction and associated risk factors identification: establishment of benchmarking key machine learning models with data rebalancing strategies. Journal of medical Internet research, 25, e44081.
2. Dong, J., Qu, X., Zhang, C., Rong, S. Q., Thai, N. D., Pan, W., ... & Ong, Y. S. (2026). Tug-of-war no more: Harmonizing accuracy and robustness in vision-language models via stability-aware task vector merging. In The Fourteenth International Conference on Learning Representations.
3. Chu, Y. B., Li, J., Jia, P., Cui, J., Zhang, R., Kang, X., ... & Zhang, S. (2021). Irf1-and Egr1-activated transcription plays a key role in macrophage polarization: A multiomics sequencing study with partial validation. International Immunopharmacology, 99, 108072.
4. Ren, Y. (2024). Using Machine Learning and Deep Learning Algorithms to Improve Low Birthweight Prediction.
5. Yao, Y., Li, Y., Zheng, S., & Zhu, T. (2025). Deep Learning Models for Multi-class Pneumonia Detection in Chest X-rays: A Comparative Study of VGG16, MobileNet, and ResNet152. Computer Simulation in Application, 3(1), 66-73.
6. Yu, X., Li, J., Wang, J., Wang, Y., Li, S., Ma, X., ... & Wang, H. (2026). Prediction of risk of hearing loss by industry noise from cross-sectional and longitudinal data. Communications Medicine.
7. Chetry, M., Feng, R., Babar, S., Sun, H., Zafar, I., Mohany, M., ... & Khan, S. (2025). Early detection and analysis of accurate breast cancer for improved diagnosis using deep supervised learning for enhanced patient outcomes. PeerJ Computer Science, 11, e2784.
8. Dong, J., Koniusz, P., Zhang, Y., Zhu, H., Liu, W., Qu, X., & Ong, Y. S. (2025, October). Improving zero-shot adversarial robustness in vision-language models by closed-form alignment of adversarial path simplices. In Forty-second International Conference on Machine Learning.
9. Yao, Y., Li, Y., Zheng, S., & Zhu, T. (2025). MSA-TransUNet: A Multi-Scale Attention Enhanced Transformer-UNet Architecture for Accurate Vessel Segmentation and Visualization in Medical Imaging. Journal of Medicine and Life Sciences, 1(4), 48-57.
10. Lv, Qi, et al. "F1: A vision-language-action model bridging understanding and generation to actions." arXiv preprint arXiv:2509.06951 (2025).
11. Zhao, Y., Zhu, Z., Yang, S., & Li, W. (2026). YeastSAM: A Deep Learning Model for Accurate Segmentation of Yeast Cells. Molecular Biology of the Cell, mbc-E25.
12. Qu, W., Wang, J., Gong, Y., Huang, X., & Xiao, L. (2025). An end-to-end robust point cloud semantic segmentation network with single-step conditional diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 27325-27335).
13. Tu, P., Xie, X., Ai, G., Li, Y., Huang, Y., & Zheng, Y. (2023). FemtoDet: An object detection baseline for energy versus performance tradeoffs. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 13318-13327).
14. Zhang, W., Zhu, J., Zhang, Y., Sun, L., Wang, K., Dong, Y., ... & Shang, A. (2025). Personalized prediction of lymph node metastasis in papillary thyroid microcarcinoma: a nomogram and web calculator. Scientific Reports.
15. Zhang, P., Ge, Z., Song, L., & Lam, E. Y. (2023). Neuromorphic imaging with density-based spatiotemporal denoising. IEEE Transactions on Computational Imaging, 9, 530-541.
16. Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., et al. (2016). Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 316(22), 2402–2410.
17. Dong, J., Chen, J., Xie, X., Lai, J., & Chen, H. (2024). Survey on adversarial attack and defense for medical image analysis: Methods and challenges. ACM Computing Surveys, 57(3), 1-38.
18. Xu, H., Ge, J., & Ou, Y. (2026). DentaScope-AI: A Real-Time k-mer–Driven CNN-Attention Framework for Rapid Pathogen Identification and Virulence Prediction in Endodontic Infections Using Nanopore Sequencing. Frontiers in Healthcare Technology, 3(1), 112-122.
19. Song, S., Tang, Y., & Qin, R. (2025). Synthetic Data Matters: Re-training with Geo-typical Synthetic Labels for Building Detection. IEEE Transactions on Geoscience and Remote Sensing.
20. Wang, J., & Zhang, M. (2020). Deepflash: An efficient network for learning-based medical image registration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4444-4452).
21. Cai, Lie, et al. "Deep learning model for breast shear wave elastography to improve breast cancer diagnosis (INSPiRED 006): An international, multicenter analysis." Journal of Clinical Oncology 43.32 (2025): 3482-3493.
22. Bagheri, Mahdi, et al. "An MRI Atlas of the Human Fetal Brain: Reference and Segmentation Tools for Fetal Brain MRI Analysis." ArXiv (2025): arXiv-2508.
23. Xie, C., Zhu, D., Wang, Z., Zhang, H., & Wei, Z. (2026). Explainable AI for Mental Health Detection from Social Media: A Comparative Study of Traditional Machine Learning and a Large Language Model. Available at SSRN 6429778.
24. Dong, J., Liu, J., Qu, X., & Ong, Y. S. (2025). Confound from All Sides, Distill with Resilience: Multi-Objective Adversarial Paths to Zero-Shot Robustness. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 624-634).
25. Ding, S., Yu, X., Wang, Q., Luo, P., Li, H., Li, Z., ... & Zhang, C. (2025). Multifunctional hydrogel sensors with dynamic covalent networks for machine learning-assisted Parkinson's disease diagnosis and encrypted human-computer interaction. Materials Today Bio, 102524.
26. Xie, C., Zhu, D., Wang, Z., Zhang, H., & Wei, Z. (2026). Compliance-Aware Discharge Agent for Auditable ICU Discharge Planning: A Pilot Feasibility Study Using Structured eICU Records. Available at SSRN 6429758.
27. Wang, Y., Ma, X., Huang, X., Li, S., Yu, X., Huang, S., ... & Yin, S. (2026). Integrating environmental factors and genetic variants in machine learning to assess occupational noise impact on health. Ecotoxicology and Environmental Safety, 311, 119805.