Importance of Data Archival in a GxP-Regulated Environment
Written by Piotr Kuchta and Robin Frey
Introduction
In analytical and industrial scientific research laboratories generation of comprehensive analytical data is a main purpose to serve scientific goals. The generation of this data in chromatographic analysis is generally supported with comprehensive chromatography data systems (CDS) such as Thermo Scientific Chromeleon 7.3.2.
Various manufacturers offer solutions for generating analytical data. Depending on the environment, the data generated in, the need to meet regulatory requirements of different authorities becomes important.
Covering those challenges correct planning, implementation and validation of the environment and the workflow is required. The CDS can help as a tool and serve as operational system fulfilling its part in the business workflow robustly and streamlined. As laboratories generate and accumulate significant volumes of data from various chromatographic analyses, the need for efficient and secure data archival becomes a major concern.
As a negative example of this matter, when a Data Vault in Chromeleon reaches 90% of its maximum capacity of Database Size Limit or Disk Space, the CDS would inform the user about this by displaying a red notification bar in its main window.
This was simulated in Chromeleon 7.3.2 MUb and applies to all actual Chromeleon 7 Version releases.
Chromeleon would recommend the user therefore to move sequences to another Data Vault or permanently delete them.
Unfortunately, this notification can be ignored by the user and only appears again after a new session login.
If this notification message is ignored long enough, the Data Vault may finally reach a point when solving the issue might become cumbersome or simply impossible without deeper understanding of data bases and additional tools only available outside of Chromeleon.
In Chromeleon as the Data Vault reaches 100% capacity, the user eventually will not be able to move any sequences or acquire more data and finally will not be able perform any actions that require Chromeleon to use a Data Transaction with the database. In this case only the database is full, while disk space would still be available, but the limit of the database size is reached.
At this point, the user would see the following error message: “The operation failed because there is not enough space available in the database. If a SQL Server Express Edition, the built in size limit may be reached. You can use the Administration Console to create a new Data Vault.”
In the error details of the message, insight for the real cause of the issue can be found.
“Could not allocate space for object 'dbo.CR_AUDITTRAIL_ITEM'.'PK_CR_AUDITTRAIL_ITEM' in database '
This translates to Chromeleon’s inability to write the preferred action into the audit trial due to unavailable disk capacity. It also proposes a solution by mentioning the deletion of unneeded files in order to make space for additional files in the file group or by choosing to change the auto growth settings for the storage in question.
It is important to remember that Chromeleon creates new audit trail entries for every sequence related action that is performed.
To further explain, even for the deletion of data such as a sequence from Chromeleon’s recycle bin (to at least get some space back) requires storage in the database to save the audit trail log.
Another topic worth mentioning is that Chromeleon Workstations and Instrument Controllers are using MS SQL Express Edition as Database system, since provided by the vendor no additional licensing is needed. The limitation is Microsoft SQL Server Express database maximum capacity limit of 10 GB.
If all above circumstances occur, fixing such a complex Chromeleon issue might seem impossible, but there are ways to handle that.
Importance of Data Archival in a GxP-Regulated Environment
The negative example shows that in Good Practice (GxP) regulated environments, data archival is not just a best practice but a critical requirement ensuring not only the proper storage of the data but also just being able to save it. GxP regulations encompass guidelines and standards that ensure products are safe, meet quality standards, and are produced consistently. In other words, ALCOA+ principles are applied to data archiving to support regulatory compliance.
ALCOA+ Principles
- Complete: All data, including metadata, should be retained. For archived data, this means ensuring that nothing is lost or omitted during the archiving process.
- Consistent: Data should be recorded in a consistent manner across the system. For archives, this means maintaining a consistent format and structure for data storage.
- Enduring: Data should remain intact and accessible throughout its retention period. Archived data must be stored on durable media and regularly monitored for integrity.
- Available: Data should be readily available for review and audit. Archived data must be retrievable in a timely manner, even years after it was originally recorded.
Keeping too much data in the system without proper archiving can lead to several issues
Different issues occur when too much data is stored without proper management such as:
- Slower Performance: Excessive data can slow down system performance, leading to longer processing times for data retrieval, analysis, and reporting.
- Complexity: The more data stored in the system, the more complex data management becomes, making it harder to organize, search, and retrieve specific datasets.
- Audit Challenges: During audits, excessive data can make it difficult to locate and present the required information promptly, potentially leading to negative audit outcomes.
- Backup and Recovery: The more data stored, the longer it takes to perform backups. The recovery of such large amounts of data in the event of system failures or disasters delays the process until operation can be resumed.
- No Data Acquisition: A full database or full hard drives on the servers force the user to act otherwise no more data can be acquired or saved after a modification in the worst-case scenario.
Best Practices to Mitigate These Issues
To mitigate these issues, organizations should implement effective data management and archiving strategies, including:
- Regular Data Archival: Implement a routine archival process to move older, less frequently accessed data to secure and compliant storage solutions such as a different Data Vault.
- Data Retention Policies: Establish clear data retention policies in line with regulatory requirements to ensure data is kept only as long as necessary the use of scheduled tasks in Chromeleon can serve this purpose.
- Data Compression and Optimization: Use data compression and optimization techniques to reduce the storage footprint without compromising data integrity done on a database level.
- Data Lifecycle Management: Employ data lifecycle management practices to ensure data is appropriately archived, retained, and disposed in accordance with its value and regulatory requirements.
- Database Maintenance: Compression of the database by removing empty pages and rebuilding indexes and other actions on a regular basis.
- Backup Strategies: Schedule backups of the database and transaction log files to assure restoration of the database to an earlier working state to ensure business continuity.
By effectively managing the volume of data kept in active systems, organizations can enhance performance, ensure compliance, uphold business continuity and reduce costs.
wega Informatik is happy to assist you in any of the above-mentioned goals as planning, executing, documenting or validating partner. With experience of many years in the pharmaceutical industry and other markets we provide this kind of support and much more.