We use cookies to personalize content, to provide social media features and to analyze our traffic. We also share information about your use of our site with our social media, advertising and analytics partners. For information on how to change your cookie settings, please see our Privacy policy. Otherwise, if you agree to our use of cookies, please continue to use our website.

The Quality Control Process Behind Your Big Data Solution

Big Data. It’s what every marketer needs to establish competitive advantage in today’s digital world. However, there is little insight into how Merkle ensures the quality and accuracy behind each client solution. Let’s take a look inside the "test it" portion of the process to see what's involved in verifying the integrity of a world-class big data solution. Merkle follows a three-phase quality control process: Unit testing, system Integration testing, and user acceptance testing. 

1. Unit Testing 

Unit testing is generally the first set of quality checks on the solution, executed by the developer during his or her development process. The developer will validate that the completed code works properly as a small unit piece on its own. 

2. System Integration Testing

The majority of the testing effort falls under system integration testing, which is planned and executed by the quality assurance team. It can be further described by looking at the most common functional areas that make up a big data solution at Merkle:

  • Customer Data Integration (CDI) Testing: provides validation of a functional, client-specific rule set for the solution. This is critical as much of the down stream processing relies on the accuracy of this data.
  • Staging Testing: validation of the source data and loading into the staging area in the database.
  • Target Testing: validation of transformations, aggregations, derived attributes, besting, and other advanced proceedings as data is moved into the target tables.
  • Database Integration Testing: end-to-end testing that runs all processing and automation on the data, starting with staging of source data and ending with campaign file outputs, aggregations, and reports.
  • Application/Extract Testing: verification of data integrity and integration with the campaign systems, business intelligence reporting systems, extracts to email and direct mail providers, back feeds, portal extracts, and web services.
  • System Testing: tests between the solution under development and any third-party systems (outside vendors, in-house client solutions, etc.).
  • Campaign/Campaign Mart Testing: verification of the data loaded into the campaign management systems and/or custom campaign marts developed for the client.

This diagram describes the general "component view" of the testing process followed at Merkle:

The quality control process with a database install

In all of these component areas and in each step of the testing process, data quality and integrity is analyzed for the smallest of variances, which are then corrected. We’ve seen how small errors in data can become large anomalies as the data is processed and flows through the solution, making this attention to detail a critical hallmark of a world-class quality program.

3. User Acceptance Testing

Once the quality assurance team completes testing, it is time for solution acceptance. The testing component of the solution acceptance process is user acceptance testing (UAT).  The goal of UAT is for the client to validate the solution's functionality as outlined in the business requirements. In most situations, Merkle assists the client in preparing and executing these UATs, ending with the client’s acceptance of the solution.

Finally, a series of automation quality checks are put in place when the solution transitions into the production environment. These quality checks are automated tests that ensure the accuracy of the data during normal solution processing. They alert to any anomalies in the data environment so that it can be corrected before it becomes an issue for the client’s solution.

Join the Discussion