Bias Detection in AI Models Using Word Embedding Association Tests (WEAT)
DOI:
https://doi.org/10.63282/3050-922X.IJERET-V5I4P109Keywords:
Word Embedding Association Test (WEAT), bias detection, natural language processing (NLP), contextual word embeddings, algorithmic fairness, ethical AI governanceAbstract
Bias in artificial intelligence (AI) systems has emerged as a critical concern, particularly in natural language processing (NLP), where pretrained word embeddings often encode and amplify societal stereotypes. The Word Embedding Association Test (WEAT) has become a foundational method for detecting such biases by quantifying the associative relationships between conceptually relevant word groups. This review provides a comprehensive synthesis of research and applications related to WEAT and its methodological extensions, covering developments. It begins with a technical overview of static and contextual word embeddings and outlines how various forms of bias manifest within them. We explore the origins and mathematical framework of WEAT, followed by its adaptations such as SEAT and ROME that address limitations in modern transformer-based models. The article then examines practical applications of WEAT in sentiment analysis, recommendation systems, healthcare diagnostics, and legal risk assessments, highlighting its role in AI governance and ethical auditing. A comparative analysis of prominent WEAT-supporting tools and empirical case studies further illustrate its effectiveness and limitations. Finally, we discuss unresolved challenges related to model contextuality, definitional ambiguity, and cross-disciplinary integration. Through this review, we underscore the importance of embedding-level bias audits and advocate for the evolution of WEAT into a more context-aware and policy-aligned framework for responsible AI
References
[1] M. Kuziemski and G. Misuraca, "AI governance in the public sector: Three tales from the frontiers of automated decision-making in democratic settings," Telecommunications Policy, vol. 44, no. 6, p. 101976, 2020.
[2] J. Whittlestone et al., "Ethical and societal implications of algorithms, data, and artificial intelligence: A roadmap for research," Nuffield Foundation, London, 2019, pp. 1–59
[3] R. Patil et al., "A survey of text representation and embedding techniques in NLP," IEEE Access, vol. 11, pp. 36120–36146, 2023.
[4] T. Bolukbasi et al., "Man is to computer programmer as woman is to homemaker? Debiasing word embeddings," in Advances in Neural Information Processing Systems, vol. 29, 2016.
[5] K. Ethayarajh, D. Duvenaud, and G. Hirst, "Understanding undesirable word embedding associations," arXiv preprint arXiv:1908.06361, 2019.
[6] J. Jangid and S. Dixit, The AI Renaissance: Innovations, Ethics, and the Future of Intelligent Systems, vol. 1, Technoscience Academy, 2023.
[7] F. Ahmed, "Cloud Security Posture Management (CSPM): Automating Security Policy Enforcement in Cloud Environments," ESP International Journal of Advancements in Computational Technology (ESP-IJACT), vol. 1, no. 3, pp. 157–166, 2023.
[8] O. Papakyriakopoulos et al., "Bias in word embeddings," in Proc. 2020 Conf. on Fairness, Accountability, and Transparency (FAT), 2020.
[9] P. P. Liang et al., "Towards understanding and mitigating social biases in language models," in Proc. Int. Conf. Machine Learning (ICML), PMLR, 2021.
[10] A. Van Loon and J. Freese, "Word embeddings reveal how fundamental sentiments structure natural language," American Behavioral Scientist, vol. 67, no. 2, pp. 175–200, 2023.
[11] K. Durrheim et al., "Using word embeddings to investigate cultural biases," British Journal of Social Psychology, vol. 62, no. 1, pp. 617–629, 2023.
[12] K. K. Singh et al., "Don't judge an object by its context: Learning to overcome contextual bias," in Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2020.
[13] R. Xu et al., "Word embedding composition for data imbalances in sentiment and emotion classification," Cognitive Computation, vol. 7, pp. 226–240, 2015.
[14] S. Dixit, "AI-powered risk modeling in quantum finance: Redefining enterprise decision systems," Int. J. Scientific Research in Science, Engineering and Technology, vol. 9, no. 4, pp. 547–572, 2022. doi: 10.32628/IJSRSET221656
[15] F. Yashu et al., "Thread mitigation in cloud native application development," Webology, vol. 18, no. 6, pp. 10160–10161, 2021. [Online]. Available: https://www.webology.org/abstract.php?id=5338s
[16] N. Swinger et al., "What are the biases in my word embedding?," in Proc. 2019 AAAI/ACM Conf. on AI, Ethics, and Society, 2019.
[17] D. Rozado, "Wide range screening of algorithmic bias in word embedding models using large sentiment lexicons reveals underreported bias types," PloS One, vol. 15, no. 4, p. e0231189, 2020.
[18] F. Ahmed, "Quantum-Resistant Cryptography for National Security: A Policy and Implementation Roadmap," Int. J. Multidisciplinary on Science and Management, vol. 1, no. 4, pp. 54–65, 2024.
[19] U. Naseem et al., "A comprehensive survey on word representation models: From classical to state-of-the-art word representation language models," Trans. Asian and Low-Resource Language Information Processing, vol. 20, no. 5, pp. 1–35, 2021.
[20] S. Husse and A. Spitz, "Mind your bias: A critical review of bias detection methods for contextual language models," arXiv preprint arXiv:2211.08461, 2022.
[21] J. Jangid, "Efficient Training Data Caching for Deep Learning in Edge Computing Networks," Int. J. Scientific Research in Computer Science, Engineering and Information Technology, vol. 7, no. 5, pp. 337–362, 2020. doi: 10.32628/CSEIT20631113
[22] C. May et al., "On measuring social biases in sentence encoders," arXiv preprint arXiv:1903.10561, 2019.
[23] J. M. Alvarez and S. Ruggieri, "Counterfactual situation testing: Uncovering discrimination under fairness given the difference," in Proc. 3rd ACM Conf. on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), 2023.
[24] I. Rabiu et al., "Modeling sentimental bias and temporal dynamics for adaptive deep recommendation system," Expert Systems with Applications, vol. 191, p. 116262, 2022.
[25] R. K. E. Bellamy et al., "AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias," IBM Journal of Research and Development, vol. 63, no. 4/5, pp. 4:1–4:15, 2019.
