EquivTrace: AI-Driven Behavioural Equivalence Testing for Mainframe-to-Cloud Migration Using Production Transaction Traces

Abstract

The largest issue is not about converting code for a mainframe to the cloud, but about proving that the new system behaves identically to its predecessor. In the industry, we refer to this process as achieving behavioural test parity. Most companies are unable to achieve this level of parity, primarily because most teams struggle to migrate their systems. This article describes EquivTrace, a tool that captures trace records of each production transaction performed in a legacy batch processing environment. These traces are then replayed through the newly developed (modernised) system, and the resulting transaction records are compared on a field-by-field basis. Legacy behaviour serves as a "derived oracle" for determining whether or not a difference in the resulting records represents a functional failure (a regression), or simply represents one form of representational divergence relative to another. Most field differences in records generated by migrated systems will be examples of what we refer to as "benign representational divergence," and include differences related to formatting fields (e.g., padded fixed width, packed decimal, etc.), differences in the order of fields within records, differences in date/time formats used, and differences in rules governing collation. EquivTrace employs both metamorphic relationships and a machine learning-based classifier to differentiate between true behavioural failures and representational divergence. Additionally, EquivTrace allows for clustering of those differences that do not result in a behavioural failure, thereby reducing the number of potential root cause analyses required by human engineers from examining individual field-by-field differences to evaluating root causes based upon relatively few groups/clusters of differences.