Due to the emergence and growth of data privacy laws, organizations can no longer use real production data for testing, development, quality assurance, or education. Therefore, they need the right tools to generate anonymized and protected data and still deliver needed insights. The need for secured quality test data is increasing to avoid penalties due to non-compliance with data privacy regulations. Refer to my other blog for more information on why secure and compliant test data management is important.
Enterprise requirements on Test Data Management follow similar principles of privacy compliance. Compliance requires data masking or encryption of sensitive production data or personal information while being used in testing environments to preserve privacy and protect production data.
The CyberRes Voltage Secure and Compliant Test Data Management (SC-TDM) solution help in discovering sensitive data, securing them with anonymization or masking of data to be compliant with regulatory requirements to use it as test data. SC-TDM combines two of CyberRes Voltage portfolio products: Voltage Secure Data and Structured Data Manager (SDM).
SC-TDM Solution Architecture and SC-TDM process
Test data creation and consumption are required for most or all the software applications at all levels: applications, database, system integration, etc. Our SC-TDM solution process is in line with global data privacy laws that include GDPR, CCPA, and frameworks such as NIST, and ENISA.
The below architecture diagram highlights details of the components, databases supported, mapping of user personas, and target locations of test data created. Testers can create the test data on-premises or in the cloud, as a file, or directly into supported bases.
Discover & Classify – This stage of the secure test data generation process, permits you to discover what kind of sensitive data are present in your application data stores and classify them whether they are PII, PCI, or PHI data. SDM Discovery provides the data analysis necessary to determine the presence of specific types of data, including personal data and GDPR, CCPA, PCI and more use cases.
The grammar and policies can be customized with the organization-specific internal security requirements and compliance related to your business to find out the sensitive data in your data stores. The source databases can be Oracle, Microsoft SQL Server, Sybase, DB2, or open standards JDBC environments in on-premises, cloud, or hybrid platforms.
The discovery process provides
- A risk score of the application database (can be customized to meet specific customer requirements)
- Review analysis results with individual grammar used to match specific types of data for discovery analysis purposes and create documentation
- Classification of sensitive data based on the policies & grammar
Data Model & Govern- In this stage, you will create data models, and processes for data extraction and deploy a secure data subsetting process, to conduct data modelling activities and customized data extraction. SDM offers
- Data modelling and business process creation to align data extraction with target databases or file locations in on-premises, cloud, or hybrid platforms.
- Automated secure data extraction tasks with built-in recovery and restart features.
- Data subsetting with selected columns and rows of database tables
- Provide group-level access control for privileged users through LDAP and SAML V2 IdP integrations for test data generation and test data validation.
Protect – In this step process you will define protection policies and execute protected extraction of test data. SDM’s data masking and built-in integration of SDM with Voltage SecureData enables you to,
- Apply appropriate data protection mechanisms to sensitive data based on its classification type – masking, tokenization, encryption, etc.
- Inclusion or exclusion of tables or columns to create required test data
- Validate the correctness of the data extraction
- Stateless key management with generation and issuance of encryption keys, and
- Supports a range of data protection methods in applications through SimpleAPI, RestAPI, UDFs, and SDKs
Monitor & Use – in this step you will create OnDemand test data extraction and make data available to Test Engineers by executing the business flow in SDM to create the required amount of test data. Once the modelled business flow is deployed it can be run multiple times with the data size mentioned by the test engineer to create test data.
Execution of the SC-TDM business flow to archive data from a database to a file allows you to relocate or copy the archived data to,
- Source database where it came from
- Another supported active database
- Another file location
- Archive database
- Separate tablespaces inside the active database
For more details regarding the process flow, refer to my white paper on Secure and Compliant Test Data Management.
Extended use cases of SC-TDM Solution
Privacy and protection provided by the SC-TDM solution in creating test data and, its flexible integration with different databases & applications allow organizations to extend this solution for,
- Data Monetization by sharing the secured data with cloud data warehouses to conduct enterprise data analytics and reporting
- Application / Data migration by creating customized test data based to evaluate the applications before migrating them into hybrid or cloud platforms
- Data Minimization by archiving or deleting the obsolete, expired, or unwanted data from operational systems to
- Save the storage cost
- Improve application performance
- Meeting compliance requirements for data retention
