At CERN, on the border between France and Switzerland, two laboratories—the European Organization for Nuclear Research (CERN) itself, focusing on particle physics, and the GSI Helmholtz Centre for Heavy Ion Research, a nuclear physics lab that collaborates closely with CERN—still exchange much of their analysis code as tar archives. The tar format, first specified in 1979 and widely used since the late 1980s, predates the iPhone, the Arduino, and most of the current workforce. It is not the most elegant tool, nor the fastest, nor the easiest to extend. But it works, and that simple fact has kept it in place for over two decades, quietly shaping how these two labs collaborate.

Why tar Persists at CERN

The tar format, originally designed for tape archives, bundles multiple files into a single archive while preserving directory structure and file permissions. Its simplicity and universality made it a natural fit for distributing analysis code in high-energy physics, a field that prizes stability above novelty. Today, physicists at CERN and GSI routinely create tar archives of reconstruction algorithms, detector simulations, and analysis frameworks, then share them with colleagues across the campus or across continents. The reliability of tar—its ability to be read by virtually any Unix-like system, and its resistance to corruption when properly checksummed—is its greatest asset.

The persistence of tar is not due to technical superiority. Modern alternatives like zip with AES encryption, or the more recent Zstandard compression, offer better compression ratios, parallel extraction, and more robust error detection. Yet tar remains the default because compatibility trumps elegance. Every piece of software written in the last thirty years that reads tar can still read it, and every script that creates tar archives can still create them. Upgrading would require rewriting thousands of lines of glue code, re-testing edge cases, and convincing hundreds of researchers to change habits built over entire careers.

Inertia here is not laziness but rational cost-benefit. A researcher who spends a week migrating to a new format is a researcher who is not analyzing data, and in a field where publication pressure is intense, that week is a luxury few can afford. The result is classic lock-in: the more people use tar, the harder it becomes to abandon it.

How tar Became the Default at CERN

The story of tar's dominance at CERN begins in the late 1980s, when the lab was preparing for the Large Electron-Positron Collider, the predecessor to the Large Hadron Collider. The experiments required massive code bases shared among hundreds of physicists, many of whom were not professional software engineers. Tar's simplicity was a feature: a graduate student could learn it in an afternoon, and a senior physicist could trust it without understanding the details. Early adopters were the experiments themselves. They needed a way to distribute Monte Carlo generators, detector simulations, and analysis frameworks without relying on a central server that might go down. Tar, with its self-contained archives and straightforward structure, fit the bill. Soon, every major experiment at CERN was using it, and when new collaborations formed, they inherited the practice from their predecessors.

Physics experiments demand stability. A detector runs for years, and the software that analyzes its data must remain compatible with data taken at the beginning of the run. Switching formats mid-experiment risks breaking the chain of reproducibility, which is the bedrock of scientific credibility. So tar stayed, not because it was perfect, but because it was good enough and because changing it would introduce risk. Universities followed the lead. Graduate students trained at CERN returned to their home institutions and introduced tar there. Funding agencies saw no reason to interfere. There was no single owner of the standard; it was simply a de facto convention, maintained by a small group of volunteers who answered questions on mailing lists and fixed occasional bugs. That lack of ownership made it resilient, because no one could kill it, but also made it hard to evolve.

The Hidden Costs of Sticking with tar

While tar is convenient, it is not free. The most tangible cost is maintenance. The open-source libraries that implement tar, such as GNU tar and libarchive, are maintained by volunteers, and they are aging. Security patches are slow to arrive, and support for modern operating systems is sometimes spotty. Researchers who encounter a bug must often patch the code themselves, a task that distracts from their actual science. New tools fight old wrappers. Modern analysis frameworks, such as those built around Python and Jupyter notebooks, do not natively read tar archives without additional libraries. As a result, researchers must write conversion scripts, which are often ad hoc and poorly documented. These scripts become a hidden layer of technical debt, and when they break, they break silently.

Reproducibility suffers silently. A key principle of scientific computing is that a published result should be reproducible by others, but if the archive format is not transparently readable by all tools, reproducibility is undermined. A researcher who receives a tar archive may find that the included checksum does not match the files, or that the archive contains a file with a name that is not valid on their operating system. These small frictions add up, and they erode trust in the scientific record. Funding flows to workarounds. Instead of investing in a modern replacement, labs spend money on bridge tools, conversion libraries, and training. These workarounds are often funded by grants that could have been spent on science itself. The opportunity cost is rarely counted, because it is invisible: no one tallies the hours lost to format issues, and no one publishes a paper about them.

Why tar Won—and Why It Stays

Tar won not because it was secure, but because it was documented. The specification was published, and implementations existed for every major language. That transparency allowed it to be embedded in countless tools, from data acquisition systems to visualization packages. It became a lingua franca, and like many lingua francas, it is imperfect but universally understood. Nobody gets fired for using tar. That phrase, borrowed from the business world, applies here. A lab manager who approves a migration to a new format risks a public failure if the new format has a subtle bug. The manager who sticks with tar, however, is seen as prudent, even if the hidden costs accumulate.

Standards bodies move slowly. The organizations that could formally standardize a replacement, such as the International Organization for Standardization or the World Wide Web Consortium, have other priorities. A format that is good enough for physics is not on their radar, and so no official alternative emerges. The result is a classic collective action problem. Any single lab could switch, but the benefits of switching increase with the number of others who also switch. Without a coordinated push, the first mover bears all the risk and gains little. So tar persists, and will likely persist for another decade.

Lessons from Other Fields' Format Wars

CERN is not alone in clinging to an old format. Astronomy, for example, adopted the Flexible Image Transport System (FITS) in the 1980s, and it remains the standard for astronomical data today. FITS has been criticized for its limitations, but it is deeply embedded in the field's software, and switching would be prohibitively expensive. Biologists, meanwhile, standardized on FASTA for sequence data, a format so simple it is almost trivial, yet it has survived because it is easy to parse and universally supported.

Economics offers a contrasting case. Many economists share data as comma-separated values (CSV) files, a format that is even older than tar. CSV is not a formal standard, but it is so simple that it works everywhere. The downside is that CSV lacks metadata, and economists have struggled with reproducibility as a result. The replication crisis in economics, as documented in field experiments, has been exacerbated by this lack of structure.

Each field's path differs, but the pattern is consistent: early adoption, network effects, and high switching costs create lock-in. CERN's case mirrors them all, but with a twist. The format is not just a data format; it is a code format, and code is updated more frequently than data. That means the friction is felt more often, and the need for a solution is more acute.

Counter-Arguments: Why Some Defend tar

Not everyone at CERN sees tar as a problem. Some researchers argue that tar's simplicity is a virtue that modern formats lack. They point out that tar archives are transparent: you can list the contents, extract a single file, and even repair a damaged archive with standard tools. In contrast, newer formats like ZIP or SquashFS often require specialized software to handle edge cases, and their compression algorithms can introduce complexity that makes debugging harder. For a small collaboration, tar is often sufficient, and the overhead of learning a new format outweighs any benefits.

Another defense is that tar is not the bottleneck. The real challenges in scientific computing, they argue, are not format-related but stem from issues like dependency management, environment reproducibility, and the sheer size of datasets. Tar archives are just a delivery mechanism; the content inside is what matters. If the code is well-structured and documented, the format is irrelevant. This perspective has merit, but it overlooks the cumulative friction that format issues create across thousands of interactions.

There is also a cultural argument. For many physicists, tar is part of their identity. It is a tool they learned in graduate school, and mastering its quirks is a rite of passage. Abandoning it would feel like abandoning a piece of history. This emotional attachment is not to be underestimated, as it influences decisions in ways that purely rational cost-benefit analysis cannot capture.

What a Modern Replacement Would Need

If a new format were to succeed, it would need to be backward compatible. That is non-negotiable. Researchers have decades of tar archives, and they will not abandon them. A new format must be able to read and write tar seamlessly, at least for a transition period. Performance must match or beat tar. The tar format is fast enough for most use cases, and a new format that is slower would be rejected out of hand. It must also support modern features, such as parallel compression, streaming, and robust error detection, without adding complexity.

Easy migration is critical. Thousands of users would need to switch, and the process must be automated. A tool that converts existing archives to the new format, with minimal user intervention, would lower the barrier. The tool must be tested on real-world data, not just toy examples. Clear governance and funding are essential. A format without a steward is a format that stagnates. The new format would need a working group, a bug tracker, and a development roadmap. That requires funding, which in turn requires a business case. The business case might be made by showing the hidden costs of the current format, but that argument has not yet won.

Finally, community buy-in from day one is necessary. The format must be designed with input from the people who will use it, not imposed from above. A participatory process, like the one that led to the adoption of tar, would build trust and reduce resistance.

Practical Steps for the Two Labs Now

In the absence of a coordinated replacement, the two labs can take pragmatic steps to mitigate the hidden costs. The first is to audit current usage patterns. How many tar archives are created each month? What tools are used to create and read them? Where are the pain points? This audit would provide data to justify a change, or at least to target workarounds.

A pilot of a parallel toolchain could be run in a single collaboration. The pilot would use a modern format alongside tar, with a conversion tool that runs in the background. The hope is that the new format would prove itself in practice, and that the pilot would generate evidence for a wider rollout. Incentivizing early adopters is another lever. A lab could offer a small grant or a named acknowledgment to researchers who migrate their code to the new format. These incentives would create a cadre of experts who can help others.

Documenting pain points publicly would build a case. A shared wiki or a GitHub issue tracker where researchers report format-related problems would quantify the cost. This documentation could be used to lobby funding agencies for a dedicated migration project. Finally, a five-year transition window is realistic. A sudden switch is impossible, but a gradual phase-out, with tar still supported but no longer the default, could work. The first two years would be for planning and tool development, the next two for parallel running, and the fifth for retirement. This timeline respects the inertia that has kept tar alive, while acknowledging that the status quo is not sustainable forever.

The Broader Implications for Scientific Software

The tar dilemma at CERN is a microcosm of a larger issue in scientific software: the tension between stability and innovation. Many fields rely on tools that were designed decades ago, not because they are ideal, but because they are trusted. This reliance creates a paradox: the very conservatism that ensures reproducibility also hinders progress. In high-energy physics, for example, the use of Fortran persists in some legacy codes, despite the availability of more modern languages, because rewriting would introduce errors. Similarly, the ROOT data analysis framework, developed at CERN in the 1990s, remains central to the field, despite its steep learning curve and idiosyncrasies.

The same dynamics play out in other sciences. Climate modelers still use netCDF, a format developed in the late 1980s, because it is deeply integrated into their workflows. Genomic researchers rely on BAM files, which are compressed and indexed in a specific way, and changing that format would require re-engineering entire pipelines. These examples show that format lock-in is not unique to tar; it is a feature of scientific computing as a whole. The question is not whether to change, but how to manage change without breaking the fragile ecosystem of tools and data that science depends on.

One possible path is the development of format-agnostic layers. Just as the internet uses protocols like HTTP to abstract away the underlying data, scientific computing could benefit from abstraction layers that allow different formats to coexist. Tools like DVC (Data Version Control) and CWL (Common Workflow Language) are steps in this direction, enabling researchers to define workflows that are independent of the underlying file formats. If such tools become widespread, the choice of archive format becomes less critical, and the costs of tar's limitations diminish.

What remains unresolved is whether the scientific community will ever find the will to coordinate such a transition. The tar format will not disappear tomorrow, and its costs will continue to accumulate. But the recognition of those costs is the first step toward a change that, while slow, is not impossible. Whether that change comes from within CERN or from an external push, the question is open: will the next generation of physicists inherit tar, or will they finally build something better?