Closing Remarks: Climbing the Ladder of Rigor

Closing Remarks: Climbing the Ladder of Rigor#

In this tutorial, we stepped through a ladder of rigor in scientific software development, exploring how to bring greater clarity, reliability, and confidence to our code. Along the way, we emphasized not just tools and techniques, but a mindset: a way of thinking about scientific software that moves beyond the traditional “code-and-fix” mentality.

We progressed through three major levels of rigor:

  • Unit testing: almost always necessary and a foundational practice. Unit tests provide fast feedback and guard against regressions, but they can only check the specific cases you think to test.

  • Property-based testing: often useful and highly effective. By automatically generating many diverse inputs, property-based testing greatly expands coverage and uncovers edge cases you might never have imagined.

  • Theorem proving: applicable in more specialized contexts. By reasoning about code with mathematical precision, symbolic tools can exhaustively verify properties or expose deep corner cases that tests might miss.

Each step brought new capabilities and new tradeoffs. None of these techniques is a silver bullet, nor are we suggesting that every project should adopt all of them. Instead, think of them as tools in your scientific software toolbox. You can choose which to apply depending on the level of confidence you need versus the practicality and cost of adoption.

Beyond Tools: A Scientific Mindset for Software#

While the techniques themselves are valuable, perhaps the most important takeaway is the mindset behind them. The introduction of these methods was built upon a perspective that is both deeper and more universal than any single tool or language:

  • Start by thinking about the properties your code should hold: physical principles, mathematical invariants, or logical guarantees.

  • Hypothesize about these properties through reasoning and specification, much like forming a scientific hypothesis.

  • Attempt to refute your hypothesis through experiments: unit tests, property-based tests, or even formal proofs.

  • Refine your code and your understanding based on the results.

This cycle mirrors the scientific method that underpins the very research your software enables. In this sense, rigorous software development is not separate from science: it is a form of scientific inquiry applied to code.

Moving Forward#

We hope that as you return to your own projects, you will view the techniques we’ve covered not just as optional add-ons, but as expressions of a broader philosophy: software as a vehicle for reliable, reproducible scientific discovery.

Even if you adopt only unit testing for now, or only experiment with property-based testing in one small module, the underlying principles will remain:

  • Clear specifications,

  • Thoughtful design,

  • Systematic reasoning,

  • And a commitment to incremental refinement.

As with any scientific endeavor, the real progress comes not from a single breakthrough, but from a sustained practice of curiosity, rigor, and iteration. We hope the lessons of this tutorial help you build better software and more trustworthy scientific applications.

Resources#

Baugh, John. “CE-537 Course Materials - NCSU Civil Engineering”, 2024.

Liskov, Barbara, and John Guttag. “Program development in JAVA: abstraction, specification, and object-oriented design.” Pearson Education, 2000.

Bradley, Aaron R., and Zohar Manna. “The calculus of computation: decision procedures with applications to verification.” Berlin, Heidelberg: Springer Berlin Heidelberg, 2007.

Jackson, Daniel. “Software Abstractions: logic, language, and analysis.” MIT press, 2012.

Jackson, Daniel. The essence of software: Why concepts matter for great design. Princeton University Press, 2021.

Lamport, Leslie. “The TLA+ Video Course”, https://lamport.azurewebsites.net/video/videos.html, accessed September 2025.

Ousterhout, John. “A Philosophy of Software Design, 2nd Edition”, Yaknyam Press, 2021.

R3Sw tutorial by Alper Altuntas (NSF NCAR). Sponsored by the BSSw Fellowship Program. © 2025.

Cite as: Alper Altuntas, Deepak Cherian, Adrianna Foster, Manish Venumuddula, and Helen Kershaw. (2025). “Rigor and Reasoning in Research Software (R3Sw) Tutorial.” Retrieved from https://www.alperaltuntas.com/R3Sw