Digital Long-Term Archiving: How to Ensure Data Remains Readable, Accessible, and Traceable Over the Long Term

An auditor is requesting the assembly records for a vehicle series from 2014. The data still exists, somewhere on a server that no one has touched in years. However, the database version is no longer supported by the manufacturer, the application won’t launch, and the colleague who knew the table schema left the company long ago. So the data is stored, but no one can read it. This is exactly the scenario for which long-term digital archiving is intended, and it reveals whether it is actually implemented in the company or merely mentioned in the procedural documentation.

What is digital long-term archiving?

Digital long-term archiving is the permanent preservation of digital data so that it remains readable, accessible, unchanged, and understandable in terms of content for decades, regardless of the hardware, software, or file formats used in the future. The term originates from the fields of archiving and librarianship. It has become more common in industry since quality records, process data, and design data are now almost exclusively generated digitally.

A database in active use is designed for speed and flexibility, while a long-term archive is designed for stability: data is imported once, never altered, and maintained throughout the entire retention period so that it can be retrieved at any time. The Association of German Business Archivists points out that in IT, “long-term archiving” is often understood to mean a period of no more than ten years, whereas archivists generally plan for an indefinite period. Manufacturing companies usually fall somewhere in between, as their retention periods are determined by law, standards, and product liability risks.

A long-term archive must fulfill three requirements simultaneously. The data must be readable—that is, available in a format that can be opened and interpreted even without the original application. It must be accessible so that authorized individuals can locate it within a reasonable amount of time without having to revive a legacy system. And it must be traceable: The archive documents that the data has remained complete and unchanged since it was transferred, and who accessed it and when.

A data record created today must, in extreme cases, remain admissible as evidence well into the 2050s. The servers, operating systems, and database versions on which it currently resides will have long since been replaced by then.

Why Digital Data Decays Faster Than Paper

A paper document loses its content gradually and visibly. A digital object often loses its content suddenly and completely: a single missing program, an unknown character encoding, or a data storage medium that is no longer readable is enough to render the information practically lost, even though every bit is still present. The term “digital obsolescence” has become established to describe this phenomenon.

The causes can be categorized into six risk classes. Each requires its own countermeasure, and a traditional backup is of no help in any of them.

For manufacturing data, system dependency and loss of context are particularly significant. A screw-tightening log is only valid evidence if it is clear which torque value, in which unit, was measured against which target value, and with which tool. This information is rarely contained in the file itself, but rather in the application’s data model. Anyone who shuts down the application without documenting and archiving the data model retains numbers that are of no use to anyone during an audit.

The BSI has therefore long recommended open standard formats for archiving documents and images, including PDF/A, XML, TIFF, and JPEG. This principle applies to all data types: The less a format depends on a specific manufacturer, the more likely it is that it can still be opened 15 years from now.

What retention periods apply to production and quality data?

The length of time data must be retained determines the complexity of any archiving strategy. In manufacturing, tax law, commercial law, industry standards, and product liability overlap, and for each data category, the longest applicable retention period applies. A detailed overview is provided in the article “Retention Periods According to the German Commercial Code (HGB), the GDPR, and the GoBD.”

Two details are easy to overlook. First, the tax retention period does not begin until the end of the calendar year in which the document was created, and, pursuant to Section 147(3) of the German Fiscal Code (AO), it does not expire as long as the assessment period for the relevant taxes is still open. Second, IATF 16949 binds the automotive supplier to the product’s service life. If a component is produced for twelve years and then supplied as a replacement part, the retention period can quickly add up to 15 to 20 years.

A similar situation applies to medical technology, as the article “Compliance and Audit-Proof Archiving in Medical Technology” demonstrates. In this context, it is important to note that the retention periods are tied to the last product placed on the market and not to the date the data record was created.

When switching systems, it is worth reviewing Section 147(6) of the German Fiscal Code (AO). According to this provision, in the event of a system migration or migration out of the production system, it is sufficient to retain the tax-relevant data on a machine-readable and analyzable data carrier after the fifth calendar year following the migration. Until then, the tax authorities must retain full access to the data. Without an archive that provides this access, the legacy system may not be shut down at all.

Which file formats are suitable for long-term digital archiving?

The choice of format is the most important technical decision in an archiving project. A good archiving format is openly documented, standardized as much as possible, and widely used. It contains no encryption or copy protection and includes all the information necessary for display. Proprietary formats are not ruled out, but they must provide a way to export data to an open format.

SIARD is of particular interest to manufacturing companies. The acronym stands for Software Independent Archiving of Relational Databases. The format was developed by the Swiss Federal Archives and published as the eCH 0165 standard. It stores the contents of a relational database as a ZIP archive containing XML files, while also describing primary keys, foreign keys, and column types. This ensures that a database from an MES, a testing system, or an ERP system remains analyzable even if the original database software is no longer licensed.

No format guarantees readability forever. Even PDF/A exists in several sub-standards, and validation tools change over time. Therefore, after selecting a format, someone must monitor whether the chosen formats continue to be supported. The OAIS model refers to this task as preservation planning.

How does the OAIS reference model according to ISO 14721 work?

The OAIS (Open Archival Information System) reference model is the international standard for long-term digital archives and is standardized as ISO 14721. It does not specify any particular technology. Instead, it describes the tasks of an archive and how information flows through it. Based on this model, other standards have been developed, such as DIN 31644—which sets criteria for trustworthy digital long-term archives from the German nestor network—and ISO 16363 for auditing such archives.

OAIS distinguishes between three information packages. The Submission Information Package (SIP) originates from the source system. The Archival Information Package (AIP) is stored permanently and contains data plus preservation metadata. The user receives the Dissemination Information Package (DIP) upon request. Six functional units operate between these stages:

An industrial company is not required to obtain OAIS certification, but can still use the model effectively as a checklist. Does each of the six functional units have a designated person in charge and a tool within its own archiving concept? Where this is not the case, there is a gap. A typical gap is preservation planning: The archive is being populated, but no one is responsible for ensuring that the formats will still be supported in ten years.

How can archived data remain traceable and admissible as evidence?

An auditor or a court also wants to know whether the data set presented is the same one that was created twelve years ago. Five components provide this proof:

1. Upon ingestion, the archive generates a cryptographic hash value for each object—for example, using SHA-256—and stores it separately. If the value differs later, the object has been altered.
2. At fixed intervals, the checksums are verified against the inventory. This allows bit rot to be detected before all copies are affected.
3. Archived data is stored on write-protected media (WORM, Write Once Read Many) or in an archive database without write access. This ensures the immutability required by the GoBD.
4. Import, access, export, migration, and deletion are logged along with a timestamp and the user’s name. The article “Maintaining Audit Trails Correctly” describes what a robust audit trail looks like.
5. Electronic signatures lose security over time because the algorithms become weaker. The BSI Technical Guideline TR 03125 (TR ESOR) describes how signatures are renewed in a timely manner using timestamps and evidence records in accordance with RFC 4998.

In addition, the GoBD require procedural documentation that describes how data is created, processed, archived, and deleted. For long-term digital archiving, this documentation serves two purposes: It is a mandatory document for the tax authorities, and it explains the archive even after the individuals involved have left the company. The detailed requirements are outlined in the article “GoBD-Compliant Archiving.”

Backup, legacy system, or archive: What will actually last 15 years?

In many manufacturing companies, three approaches—all referred to as “archiving”—are used side by side: data backups on tape or in the cloud, legacy systems kept running in read-only mode, and true archiving systems. Only the third meets the requirements for long-term digital archiving. The article “Backup vs. Archiving” explains the difference between backup and archiving in detail.

Continuing to operate a legacy system seems cost-effective at first because no migration project is required. Over a period of 10 to 15 years, however, this approach no longer pays off. The hardware ages, the operating system is no longer supported, and eventually the system will no longer start up. This article describes how to properly decommission a legacy system without losing data Shut down the legacy system.

The true value of an archive becomes apparent when, 15 years later, someone asks a question that no one anticipated at the time of the transition.

Guiding principle from CSP archiving projects

7 Steps to Digital Long-Term Archiving

The following seven steps apply the six OAIS functional units to a manufacturing company. They are suitable for replacing a single legacy system as well as for a company-wide archiving strategy.

Step 1: Inventory and Classify Data Sources

Identify all systems that generate data subject to retention requirements: MES, inspection systems, assembly control systems, ERP, CAD, and document repositories. Assign each data type to a class, such as tax documents, quality records, technical documentation, or personal data.

Step 2: Create a retention schedule with retention periods and deletion rules

For each class, define the longest applicable retention period and the triggering event—such as the end of the fiscal year or the last shipment of a product. At the same time, specify when and how the data will be deleted. The article “GDPR-Compliant Deletion Policy for Manufacturing Companies” provides the basics for this.

Step 3: Define a format catalog

For each data type, determine the target format based on the format guide above. Document which formats are accepted, which are converted, and which are additionally retained in their original form.

Step 4: Define metadata and context

Describe the metadata associated with each archive object: serial number, batch, order, plant, time period, source system. Include the data dictionary, units, status codes, and process documentation in the archive.

Step 5: Perform the transfer with proof of integrity

Before the export, count the number of records per table, calculate checksums, and compare the two after the transfer. Log any discrepancies and their resolution. The source system may not be taken offline until a successful comparison has been performed.

Step 6: Manage access and roles

Determine who is authorized to search, export, and delete data. Quality management, the tax department, data protection, and auditors require different views. The article “Archive Access Without an IT Ticket” explains how business units can access the system without going through IT.

Step 7: Organize Long-Term Preservation

Appoint a person responsible for preservation planning. At least once a year, verify the integrity of the collection, the support for archive formats, and the recoverability of a sample. Perform deletion runs after the retention period ends and log each run.

Use the following grid to assess where your company stands today.

How CHRONOS Implements Digital Long-Term Archiving

CHRONOS is the archiving solution in the CSP Manufacturing OS. It transfers structured data from databases and applications into an audit-compliant long-term archive, where it can be searched by business units. This primarily refers to database content from testing, production, and administrative systems, the meaning of which depends on the data model of the source application. It is precisely this data that is the most difficult to keep readable over the long term.

The KLS Martin Group user case study, “KLS Martin Group Secures Application Data with CHRONOS,” demonstrates how this works in practice. You can find more information about the solution on the CHRONOS Data Archiving page.

For the simple storage of Office documents, a document management system with PDF/A support is often sufficient. CHRONOS is designed for situations where structured data from databases must remain analyzable for many years and the source system is to be replaced. The article “Choosing the Right Software for Audit-Compliant Archiving” describes the criteria for selecting a system.

Frequently Asked Questions About Digital Long-Term Archiving

What is digital long-term archiving?

Digital long-term archiving refers to all organizational and technical measures that ensure digital data remains readable, retrievable, unchanged, and interpretable for ten years or longer. It encompasses open archival formats, metadata, integrity checks, media and format migration, and regulated access. The reference model for this is OAIS, as defined by ISO 14721.

How long is “long-term” in the context of digital long-term archiving?

In the technical literature, retention periods of ten years or more are considered long-term archiving. For manufacturing companies, the specific time periods are determined by law and standards: 8 years for accounting documents pursuant to Section 147(3) of the German Fiscal Code (AO), 10 years for annual financial statements, at least 10 years after the last placing on the market for medical devices, and 15 years for implants pursuant to Article 10(8) of the Medical Device Regulation (MDR).

Which file format is best suited for long-term archiving?

There is no single “best” format; rather, there is a suitable format for each data type. For documents, PDF/A conforming to ISO 19005 is the standard; for raster images, TIFF or PNG; for tabular data, CSV with documented character encoding; for structured data, XML with a schema; for relational databases, SIARD; and for 3D design data, STEP in accordance with ISO 10303.

Is a backup the same as long-term archiving?

No. A backup is a copy created for recovery after data loss and is regularly overwritten. It remains tied to the original system and format. Long-term archiving separates the data from the source system, converts it into durable formats, verifies its integrity, and manages search, access, and deletion throughout the entire retention period.

What is the difference between audit-proof archiving and long-term archiving?

Audit-proof archiving ensures that data is stored completely, unchanged, and tamper-proof, as required by GoBD and HGB. Long-term archiving addresses the question of whether this data can still be opened and understood in 15 years. In practice, a manufacturing company needs both: immutability and long-term readability.

How do you archive a database for the long term?

A database is archived for the long term by transferring tables, keys, relationships, and metadata to a vendor-neutral format such as SIARD or to a documented archive database. Before the transfer, data records are counted and checksums are calculated; completeness is then verified. Afterward, the source system can be taken offline without losing access to the data.

Is it permissible to convert archived data into a different format?

Yes, format migration is actually necessary for long-term archiving. However, tax-relevant data must remain machine-readable, and according to the GoBD, both the original and the converted version must be retained following a conversion. Every migration should be documented with checksums, record reconciliation, and a log.

What happens to archived data after the retention period expires?

After the retention period expires, personal data must be deleted in accordance with Art. 5(1)(e) and Art. 17 of the GDPR, unless another legal basis permits continued storage. Tax-related documents are also subject to the suspension of the statute of limitations under Section 147(3) of the German Fiscal Code (AO). A good archiving system deletes data based on rules and logs every deletion.

Conclusion: Storage space alone is not enough for long-term digital archiving. It is crucial that formats, metadata, proof of integrity, and responsibilities be defined today. Then, even 15 years from now, questions from an auditor, a client, or a court can be answered with reliable data.