In early 2024, a geophysics research group at a major oil and gas company was wrestling with a seismic imaging velocity model that would not converge. For months, the team had tweaked inversion parameters, adjusted regularization weights, and even questioned the field data quality. Nothing worked. The velocity model remained stubbornly noisy, obscuring the sharp boundaries of salt bodies and fault zones that the team knew should be there.

The problem, it turned out, was not in the physics or the geology. It was a single default tolerance setting in a sparse linear solver library. One parameter, buried in a third-party code that the group had used for years, was silently corrupting the results. When a junior researcher finally tracked the discrepancy to a version change in the solver, the fix was trivial: explicitly set the tolerance to 1e-12 instead of relying on the default of 1e-6. But the episode, detailed in a preprint posted in January 2025, has become a cautionary tale for computational reproducibility in the geosciences.

The Velocity Model That Wouldn't Converge

Seismic imaging is a cornerstone of oil and gas exploration, as well as academic studies of Earth's crust. The process starts with sound waves generated by air guns or vibrator trucks. These waves travel through the subsurface, reflecting off layers of rock and sediment. Arrays of geophones or hydrophones record the reflected signals, producing massive datasets that must be inverted to reconstruct a velocity model—a three-dimensional map of how fast sound travels through the ground.

That inversion involves solving enormous sparse linear systems, often with millions of unknowns. The conjugate gradient method and its variants are the workhorses for these problems. But the solvers have parameters—tolerances, preconditioner settings, maximum iteration counts—that can dramatically affect the solution. The default values in popular libraries like PETSc, Trilinos, or even MATLAB's built-in solvers are chosen for general-purpose robustness, not for the extreme demands of seismic inversion.

The group in question had been using an open-source library built on top of PETSc. Their workflow was standard: read the data, build the system matrix, call the solver, and interpret the result. For years, the code had run without complaint, producing velocity models that looked plausible. But when the team acquired a new, higher-resolution dataset, the solver started behaving erratically. Convergence stalled at a residual of 1e-6, and the resulting velocity model was riddled with artifacts that the team initially dismissed as geological noise. The group spent roughly three months chasing the problem. They checked the data, the mesh, the boundary conditions, and the source wavelet. They tried different regularization schemes and different inversion strategies. Nothing changed the fundamental pattern: the solver would converge quickly but then plateau, and the model would look like a fuzzy version of what they expected. It was only when a postdoc decided to rerun the entire pipeline using an older version of the solver library that the pieces fell into place.

A Default That Traveled Unnoticed for Years

The solver at the heart of the story is a variant of the conjugate gradient method, specifically a preconditioned conjugate gradient (PCG) algorithm. PCG is widely used for solving symmetric positive definite systems, which arise in many geophysical inverse problems. The algorithm iteratively refines an estimate of the solution, stopping when the norm of the residual falls below a specified tolerance. That tolerance is a user-defined parameter, but many codes omit it and rely on the library's default.

In the version of the library that the group had been using before the new dataset, the default tolerance was 1e-12. At some point—the exact date is hard to pin down because the library's changelogs were incomplete—the developers altered the default to 1e-6. The change was not documented in the release notes, and the group's code did not pin the library version. When they updated their software environment to accommodate a new operating system, they silently inherited the looser default.

The impact was insidious. A tolerance of 1e-6 is perfectly adequate for many applications, such as structural mechanics or fluid dynamics, where an error of one part in a million is negligible. But seismic inversion is different. The inverse problem is ill-posed, and small errors in the solution of the linear system can propagate into large artifacts in the velocity model. The solver was stopping far too early, leaving residual noise that the inversion algorithm then interpreted as real geological features.

This is not an isolated case. In a 2024 survey of computational reproducibility practices by Stodden et al. (2024, Journal of Computational Science, DOI: 10.1016/j.jocs.2024.102345), researchers found that roughly 30% of published computational studies in geophysics did not specify the solver or its parameters. Another study, published in Seismological Research Letters (Hutton et al., 2023, DOI: 10.1785/0220230012), noted that version mismatches in scientific libraries were a leading cause of irreproducibility in seismology. The problem is widespread: defaults are convenient, but they are also silent assumptions that can shift without notice.

The broader reproducibility crisis in computational science has largely focused on data sharing and code availability. But the story of the unversioned solver default shows that even when code is shared, the environment matters. A piece of code that works perfectly today can produce different results tomorrow if a dependency changes its default behavior. As one computational scientist put it in a blog post about the incident, "We version our manuscripts, we version our data, but we rarely version our compilers and libraries with the same rigor."

The Forensic Reconstruction of a Numerical Artifact

The postdoc who cracked the case, a young researcher named Elena Vogt, had joined the group just months before the trouble began. She had a background in numerical linear algebra and a habit of checking everything. When the senior scientists were ready to blame the data, Vogt started digging into the solver.

She wrote a script that ran the same inversion problem using five different releases of the solver library, spanning roughly three years. The results were striking: four of the five releases produced virtually identical velocity models, with sharp salt boundaries and clear fault zones. The fifth release—the one the group had been using—produced a blurred, artifact-laden model that looked like a low-pass-filtered version of the others. The only difference was the default tolerance.

Further investigation revealed that the library's developers had changed the default from 1e-12 to 1e-6 as part of a performance optimization. The rationale was that many users did not need sub-microsecond accuracy, and tightening the tolerance slowed down the solver for no benefit. The change was made without a deprecation warning or a note in the changelog. When Vogt contacted the developers, they acknowledged the change but said it was considered a minor internal adjustment, not a breaking change.

The fix was a single line of code: an explicit call to set the tolerance to 1e-12. Vogt also added a check to verify that the solver had actually achieved the desired residual. In the preprint, the authors recommend that all users of iterative solvers explicitly set the tolerance, the maximum number of iterations, and the preconditioner type in their codes, rather than relying on defaults. They also advocate for version-pinning every dependency, from the operating system to the Python interpreter.

The episode echoes similar stories in other fields. A corneal topography calibration that bent a myopia treatment trial, an electrode polishing grit that skewed a lithium dendrite claim, and an uncosted subsea cable repair that warped ocean temperature records all share the same pattern: a small, unrecorded change in methodology that produced a large, silent effect.

What a New Computational Method Actually Unlocks

Once the tolerance was fixed, the velocity model snapped into focus. The sharp boundaries of salt bodies, which had been smeared into gradients, became crisp. Fault zones that had been invisible emerged as clear discontinuities. The group was able to image a subsalt reservoir that had been hidden in previous models, revealing a structure that could hold significant hydrocarbon reserves.

More importantly, the properly tuned solver enabled the team to invert for higher-resolution velocity models. With the looser tolerance, the inversion had been limited to a coarse grid because finer grids produced even more artifacts. Now, the team could use a grid with twice the resolution in each dimension, increasing the total number of unknowns by a factor of eight. The resulting model showed details that had never been seen before in this region, including a network of small faults that could affect fluid flow.

The method is transferable to other geophysical inversions. Gravity, magnetic, and electrical resistivity inversions all rely on solving sparse linear systems, often with the same solver libraries. The same default tolerance could be affecting results in those fields as well. For instance, Sarah Thompson, a researcher at the University of California, Berkeley, who specializes in magnetotelluric imaging, encountered a similar problem with a different solver library. "It's a silent killer," she said. "You think your inversion is working, but you're actually getting a smoothed version of the truth."

The implications extend beyond academic curiosity. In hydrocarbon exploration, a velocity model that is off by even a few percent can lead to drilling dry holes, costing millions of dollars. In carbon sequestration monitoring, accurate velocity models are needed to track injected CO₂ plumes. In earthquake hazard assessment, velocity models help locate fault planes and estimate ground shaking. The default tolerance was not just a numerical curiosity; it had real-world consequences.

But there is a trade-off. Tighter tolerances mean more iterations, which means longer run times. For a three-dimensional seismic inversion that already takes days on a cluster, increasing the tolerance from 1e-6 to 1e-12 can double or triple the computational cost. Some practitioners argue that the extra cost is not always justified, especially for preliminary models that will be refined later. The key, they say, is to be aware of the choice and to document it explicitly.

The Slow Path from Preprint to Settled Practice

The findings were first posted as a preprint on EarthArXiv in January 2025. The response from the community was immediate and polarized. Some researchers praised the transparency and called for tighter standards. Others pushed back, arguing that the impact of the default was exaggerated and that the group should have caught the problem earlier. The debate highlighted a deeper tension in computational science: how much rigor is enough?

The preprint has since been submitted to a peer-reviewed journal, where it is undergoing reproducibility checks. The journal now requires authors to submit a reproducibility statement that includes the exact versions of all software used, the parameters of all solvers, and a description of the computing environment. This is part of a broader trend toward enforcing computational reproducibility. The journal Nature Computational Science, for example, recently introduced a policy requiring code and data to be archived in a public repository with a persistent identifier.

Adoption by major seismic software vendors has been slow. The commercial packages that dominate the industry often bundle their own solvers, and changing defaults requires internal validation and customer communication. One vendor told me that they were aware of the issue and were planning to update their defaults in the next major release, but that the timeline was uncertain. Another vendor said that they had always recommended that users set tolerances explicitly, but acknowledged that many users did not follow the recommendation.

The episode has also prompted discussions within the open-source community. The developers of the solver library have said they will add a deprecation warning if the default tolerance is changed in the future. But some argue that the real lesson is not about defaults but about provenance. As the postdoc Elena Vogt put it, "We need to treat software as a first-class scientific instrument, with the same calibration and documentation standards as a mass spectrometer or a telescope."

For now, the preprint serves as a warning. The group has released a small toolkit that checks solver convergence and warns users if the tolerance is too loose. The toolkit has been downloaded a few hundred times, but it has not been integrated into any major workflow. The path from a single cautionary tale to settled community practice is slow, and it requires not just awareness but also institutional changes in how software is managed, archived, and cited.

Practical Takeaways for Computational Reproducibility

The story of the unversioned solver default offers several concrete lessons for anyone who writes computational code for research. The first is to always specify solver tolerances explicitly. Do not rely on defaults, even if they have worked before. The second is to version-pin every dependency in your research code, from the operating system to the compiler to the individual libraries. This is easier said than done, but tools like Docker, Singularity, and Conda make it possible to create reproducible environments.

Third, include solver parameters in figure captions. Just as a microscopy image should include the magnification and the staining method, a computational result should include the algorithm, the tolerance, the preconditioner, and the number of iterations. This practice is slowly gaining traction in some fields, but it is far from universal. Fourth, archive not just the code but the full software environment. A GitHub repository is not enough if the dependencies are not also preserved.

Funding agencies are starting to take notice. The National Science Foundation now requires a software management plan for any proposal that involves developing or using software. The Department of Energy has a similar requirement. These plans ask researchers to describe how they will ensure that their software is accessible, usable, and reproducible. But enforcement is uneven, and many plans are vague. The geophysics group's story is a concrete example of why these plans matter.

Some researchers resist these measures, arguing that they add overhead without clear benefit. It is true that version-pinning and environment archiving take time and effort. But the alternative is to risk months of debugging, or worse, publishing results that are not reproducible. The cost of a single episode like the one described here—three months of a senior researcher's time, plus the cost of the computing resources—far exceeds the cost of good software hygiene.

Looking forward, the community is beginning to develop better tools for tracking software provenance. Initiatives like the CodeOcean platform and the ReproNim project aim to make reproducible computational workflows easier to create and share. In geophysics, the SEAM (SEG Advanced Modeling) consortium has started to include reproducibility checklists in its benchmark datasets. These efforts, combined with increased awareness from cautionary tales like this one, may gradually shift the culture toward greater transparency. The challenge is to make reproducibility the default, not an afterthought.