A Comprehensive Framework for Quality Assurance in Artificial Intelligence: Methodologies, Standards, and Best Practices
DOI:
https://doi.org/10.63282/3050-922X.IJERET-V4I2P105Keywords:
Artificial Intelligence, Quality Assurance, AI Testing, Bias Mitigation, FrameworkAbstract
AI is undeniably one of the technologies that has evolved significantly in recent years and has infiltrated sectors like healthcare, finance, production, and autonomous vehicles. However, AI's quality, reliability, and ethical standards remain a major issue of concern. This paper also offers a theoretical framework for the methodologies, current standards used, and recommendations for practice in AI quality assurance. A survey of AI testing, verification, and validation approaches is presented, along with the different global AI quality standards and guidelines and a framework to improve the AI system's resilience. Moreover, we also discuss various issues related to AI quality, such as biases, interpretability, and regulation. Finally, utilizing case studies, we show possible specific applications of the developed framework based on real practices. These results from the current research prompt the need to incorporate sound quality assurance approaches into an AI development process to implement dependable and responsible AI systems
References
[1] Wang, C., Yang, Z., Li, Z. S., Damian, D., & Lo, D. (2024). Quality assurance for artificial intelligence: A study of industrial concerns, challenges, and best practices. arXiv preprint arXiv:2402.16391.
[2] Fujii, G., Hamada, K., Ishikawa, F., Masuda, S., Matsuya, M., Myojin, T., ... & Ujita, Y. (2020). Guidelines for quality assurance of machine learning-based artificial intelligence. International journal of software engineering and knowledge engineering, 30(11n12), 1589-1606.
[3] Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014). Power to the people: The role of humans in interactive machine learning. AI magazine, 35(4), 105-120.
[4] Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2020). Machine learning testing: Survey, landscapes, and horizons. IEEE Transactions on Software Engineering, 48(1), 1-36.
[5] Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., ... & Herrera, F. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities, and challenges toward responsible AI. Information fusion, 58, 82-115.
[6] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016, August). " Why should I trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1135-1144).
[7] Gehr, T., Mirman, M., Drachsler-Cohen, D., Tsankov, P., Chaudhuri, S., & Vechev, M. (2018, May). Ai2: Safety and robustness certification of neural networks with abstract interpretation. In 2018 IEEE symposium on security and privacy (SP) (pp. 3-18). IEEE.
[8] Galhotra, S., Brun, Y., & Meliou, A. (2017, August). Fairness testing: testing software for discrimination. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (pp. 498-510).
[9] Raji, I. D., & Buolamwini, J. (2019, January). Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial AI products. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (pp. 429-435).
[10] Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., ... & Young, M. (2014, December). Machine learning: The high-interest credit card of technical debt. In SE4ML: software engineering for machine learning (NIPS 2014 Workshop) (Vol. 8).
[11] Holstein, K., Wortman Vaughan, J., Daumé III, H., Dudik, M., & Wallach, H. (2019, May). Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1-16).
[12] Ma, L., Juefei-Xu, F., Zhang, F., Sun, J., Xue, M., Li, B., ... & Wang, Y. (2018, September). Deepgauge: Multi-granularity testing criteria for deep learning systems. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (pp. 120-131).
[13] QA Best Practices for Developing and Testing AI and Machine Learning Systems, quality, online. https://qualitrix.com/qa-best-practices-for-developing-and-testing-ai-and-ml-systems/
[14] Manabe, K. (2024). Quality Assurance-Artificial Intelligence. In Encyclopedia of Computer Graphics and Games (pp. 1507-1514). Cham: Springer International Publishing.
[15] Keill, P., & Johnson, T. (1994). Optimizing performance through process improvement. Journal of Nursing Care Quality, 9(1), 1-9.
[16] Karam, M., Fares, H., & Al-Majeed, S. (2021). Quality assurance framework for the design and delivery of virtual, real-time courses. Information, 12(2), 93.
[17] Ouyang, T., Isobe, Y., Marco, V. S., Ogata, J., Seo, Y., & Oiwa, Y. (2021, August). AI robustness analysis with consideration of corner cases. In 2021 IEEE International Conference on Artificial Intelligence Testing (AITest) (pp. 29-36). IEEE.
[18] Lima, R., da Cruz, A. M. R., & Ribeiro, J. (2020, June). Artificial intelligence applied to software testing: A literature review. In 2020 15th Iberian Conference on Information Systems and Technologies (CISTI) (pp. 1-6). IEEE.
[19] Felderer, M., Enoiu, E. P., & Tahvili, S. (2023). Artificial intelligence techniques in system testing. In Optimising the Software Development Process with Artificial Intelligence (pp. 221-240). Singapore: Springer Nature Singapore.
[20] Chang, C. L., Hung, J. L., Tien, C. W., Tien, C. W., & Kuo, S. Y. (2020, October). Evaluating the robustness of AI models against adversarial attacks. In Proceedings of the 1st ACM Workshop on Security and Privacy on Artificial Intelligence (pp. 47-54).
