Benchmark-Driven Development
Building verification and validation cases first, then developing the code to pass them, so credibility is designed in from the start.
Tests Before Code
In ordinary software, test-driven development writes the test before the code that satisfies it. The same idea applies to scientific codes: define the verification and validation benchmarks a capability must meet before or alongside developing it, then build the code to pass them. Credibility becomes a design goal from the start rather than an audit bolted on after the science is done.
How It Changes the Workflow
- Manufactured and analytic solutions are chosen early, so every new term has a verification test waiting for it.
- Benchmark cases and, where possible, validation data are identified before development, defining what success means.
- The order-of-accuracy target is set in advance, so a new scheme is judged against its formal promise immediately.
- Every fixed bug becomes a new benchmark, so the same defect cannot return.
Why It Helps
Retrofitting verification onto a mature code is painful: the code may not be structured to make components testable, and building a test suite after the fact competes with delivering results. Designing the benchmarks first keeps the code testable, catches errors when they are cheap to fix, and ensures that at every stage the credibility evidence keeps pace with the capability. It also prevents the common failure where verification is promised but perpetually deferred.
Interaction with CI
Benchmark-driven development pairs naturally with continuous integration. The benchmarks defined up front become the automated suite that runs on every change, so the code is continuously confirmed to still pass the tests that defined its capability. The result is a code whose verified state is maintained as a standing property, not rediscovered in a scramble before publication.
The discipline does not guarantee correctness on its own, but it changes the default: instead of a code that works until proven otherwise, it produces a code that is continuously demonstrated to meet defined standards, with the evidence accumulating as the code grows.