A Scalable Master Data Management Architecture for Enterprise Data Integration and Governance in Full-Stack Application Environments
DOI:
https://doi.org/10.63282/3050-922X.IJERET-V2I1P111Keywords:
Master Data Management (MDM), Data Governance, Enterprise Integration, Full-Stack Architecture, Data Quality, Metadata Management, MicroservicesAbstract
Enterprise computing is becoming more dependent on heterogeneous data sources throughout full-stack environments, which pose a challenge in data consistency, data quality and data management. The legacy Master Data Management (MDM) technologies have difficulty in scaling to microservice-based, cloud-native and real-time models which results in the presence of data silos, data duplication, approximately delayed existence, and weak governance. The paper will present a scalable and modular MDM architecture to the contemporary enterprise systems. The design is based on the microservices architecture in which the most important element of the design is centralized and includes the master data hub as well as the distributed data pipelines, metadata management and governance layer. It uses API-based communication, synchronization principles based on events, and cloud-native to guarantee the smoothness of the interoperability between application layers. The methodology is based on a layered architecture in which data ingestion, standardization, entity resolution, and distribution are used. The high quality and consistency of data is guaranteed by complex matching algorithms and governance mechanisms on the basis of policies, and scalability and fault tolerance are made possible by containerized and distributed technologies. Latency reduction and throughput in experimental results demonstrate a significant improvement after experimentation of pinpointing improvements in how traditional MDM systems used to work. The architecture also exhibits high levels of scalability and workload when in high load and improved levels of accuracy, compliance and traceability of the data. The key contributions are: (i) scalability cloud native MDM architecture, (ii) embedded real time data processing and governance, (iii) high-performance data integration design, and (iv) overall evaluation with findings that revealed the improvement in data quality, scalability and governance efficiency.
References
[1] Seetala, S. R. (2021). Master Data Management as a Strategic Foundation for Enterprise Consistency: Frameworks, Architectures, and Governance Practices. International Journal of Computer Technology and Electronics Communication, 4(1), 3230-3240.
[2] Balalaie, A., Heydarnoori, A., & Jamshidi, P. (2015). Migrating to cloud-native architectures using microservices: An experience report. arXiv. https://arxiv.org/abs/1507.08217
[3] Paik, H. Y., Xu, X., Bandara, H. D., Lee, S. U., & Lo, S. K. (2019). Analysis of data management in blockchain-based systems: From architecture to governance. Ieee Access, 7, 186091-186107.
[4] Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., & Brandic, I. (2009). Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation computer systems, 25(6), 599-616.
[5] Medvidovic, N., & Taylor, R. N. (2010, May). Software architecture: foundations, theory, and practice. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering-Volume 2 (pp. 471-472).
[6] Newman, S. (2015). Building microservices: Designing fine-grained systems. O’Reilly Media.
[7] Bai, X. (2012). A mathematical framework for data quality management in enterprise systems. INFORMS Journal on Computing, 24(4), 648-664.
[8] Yu, S., Liu, M., Dou, W., Liu, X., & Zhou, S. (2016). Networking for big data: A survey. IEEE Communications Surveys & Tutorials, 19(1), 531-549.
[9] Gade, K. R. (2021). Data-driven decision making in a complex world. Journal of computational innovation, 1(1).
[10] Shaykhian, G. A., Khairi, M. A., & Ziade, J. (2016, June). Architectural Evaluation of Master Data Management (MDM): Literature Review. In 2016 ASEE Annual Conference & Exposition.
[11] Jakšić, D., Jovanović, V., & Poščić, P. (2017, May). Integrating evolving MDM and EDW systems by data vault based system catalog. In 2017 40th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) (pp. 1401-1406). IEEE.
[12] Loshin, D. (2010). Master data management. Morgan Kaufmann.
[13] Dreibelbis, A. (2008). Enterprise master data management: an SOA approach to managing core information. Pearson Education India.
[14] Bonnet, P. (2013). Enterprise data governance: Reference and master data management semantic modeling. John Wiley & Sons.
[15] Vilminko-Heikkinen, R., & Pekkola, S. (2012, November). Organizational Issues in Establishing Master Data Management Function. In ICIQ (pp. 1-13).
[16] Rahman, M. S., & Reza, H. (2020, July). Systematic mapping study of non-functional requirements in big data system. In 2020 IEEE International Conference on Electro Information Technology (EIT) (pp. 025-031). IEEE.
[17] Liu, D., Deters, R., & Zhang, W. J. (2010). Architectural design for resilience. Enterprise Information Systems, 4(2), 137-152.
[18] Ghane, K. (2020, March). Big data pipeline with ML-based and crowd sourced dynamically created and maintained columnar data warehouse for structured and unstructured big data. In 2020 3rd International Conference on Information and Computer Technologies (ICICT) (pp. 60-67). IEEE.
[19] Bakshi, K. (2017, March). Microservices-based software architecture and approaches. In 2017 IEEE aerospace conference (pp. 1-8). IEEE.
[20] Agostinho, C., Ducq, Y., Zacharewicz, G., Sarraipa, J., Lampathaki, F., Poler, R., & Jardim-Goncalves, R. (2016). Towards a sustainable interoperability in networked enterprise information systems: Trends of knowledge and model-driven technology. Computers in industry, 79, 64-76.
[21] Fernando, L. K., & Haddela, P. S. (2017, September). Hybrid framework for master data management. In 2017 seventeenth international conference on advances in ICT for emerging regions (ICTer) (pp. 1-7). IEEE.
