A signed calculation report has to be reproducible. If someone reopens the project a year from now, or another engineer reviews it, the same model has to give the same numbers, and it has to be clear which version of the program produced them.
When we measured it we found it did not always hold: two solves of the same model in one session could differ in the last digits, and some results came out in a different order. The differences were tiny, but a number that changes while nothing changed is a number that cannot be checked.
Where the difference came from
The engine keeps nodes, members, supports and loads in hash tables, and every table is created with a random seed: the same file read twice is walked in two different orders. Adding floating-point numbers in another order can change the last bit, because computer addition is not associative. The 3D assemblers already sorted members, but the 2D ones, the load and support loops and most other analyses did not.
- Every walk over an input table now goes in a fixed order, the order of its keys: 587 places across the engine.
- The equilibrium sums and the cross terms of inclined supports are walked in a fixed order too.
- Warnings come out sorted by node or member number, not at random.
| What | Count |
|---|---|
| Walks over input tables put in a fixed order | 587 |
| Readings of each random model compared bit for bit | 6 |
A test reads each random model six times, solves it and compares the results as text, bit for bit, in linear, modal, buckling and P-Delta analyses. An unordered walk that changes a result shows up as a difference between the runs.
Every report says where it came from
Every calculation report, PRO report, workbook and CSV now carries a provenance block: version and commit of the app and of the engine, the SHA-256 fingerprint of the WebAssembly module that did the solve, a fingerprint of the exact model, of the settings and of the results shown, and the status of the certificates (a moving-load envelope records less: its model and settings). If the project changed while the export was being prepared, nothing is exported: a report never carries one state’s numbers under another state’s provenance.
With that block the app can reproduce a report: it solves the project again and compares every fingerprint. The verdict is “reproduced” only when all of them match (a different WebAssembly build is reported beside it, not counted as a difference); when one could not be recomputed it says “incomplete”, never “reproduced”. There is also a command-line tool that solves the same input twice and checks the fingerprints.
Old projects say so
Every saved project records the results version it was solved with. When a project solved before a change that concerns its features is opened, the app lists those changes, explained in your language, and marks the results as stale until it is solved again. When in doubt a change counts as applicable: one notice too many costs a minute, one too few can hide a number that moved.
One limit
The app always compares WebAssembly with WebAssembly. The native tool reproduced the browser’s fingerprints bit for bit on the test model, but trigonometric functions come from different libraries on each platform, so a model whose assembly uses them in a way the two round differently could differ in the last bit between the command line and the browser.