Months after an analysis is delivered the question comes: "could we try this condition as well?" or, worse, "the field result did not match, where does the difference come from?"
At that moment what is valuable is not the charts in the report but the whole chain that produced the result.
What has to be kept
- The input geometry and the list of simplifications made to it. See the geometry preparation note.
- The mesh generation parameters — not the mesh itself but the recipe that produces it. With the recipe the mesh can be regenerated; a mesh file alone is an unmodifiable box.
- Boundary conditions and material properties, with their sources.
- Solver settings: discretisation order, pressure-velocity coupling, convergence criterion.
- The convergence history and the conservation balance table. See the convergence note.
- The raw results of the grid independence study, not just the final table.
- Version information. Which version the computation was done with; default settings can change between versions.
Why the recipe, not the file
Mesh and solution files are large and become unreadable with time. A recipe is small, readable and modifiable. When a parameter change is requested, the answer takes hours with a recipe and weeks without one.
This is the real justification for setting analyses up by script rather than by hand: reproducibility is not a by-product of automation but its purpose.
Naming and versioning
Runs need distinguishable, sortable names; "trial2_final_real" is not a naming system. Each run should change one variable, and what changed should be visible in its name or its log.
In optimisation studies this is mandatory; consistency between runs is the result itself. See the surrogate model note.
The test
A simple check: if the person who did the analysis were off the project, could a colleague reproduce the same result? If the answer is no, what was delivered is not a result but a claim.
References
- ASME (2009, R2021). V&V 20 — Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer. American Society of Mechanical Engineers, New York.
- Celik, I. B., Ghia, U., Roache, P. J., Freitas, C. J., Coleman, H. & Raad, P. E. (2008). Procedure for estimation and reporting of uncertainty due to discretization in CFD applications. Journal of Fluids Engineering, 130(7), 078001. DOI: 10.1115/1.2960953