Gabriel WlazłowskiIntroduction
The priorities for W-SLDA Toolkit developers are as follow (from highest to lowest):
- Correctness of computation.
- Performace.
- User-friendly interface.
The highest priority is computation correctness, which is the fundamental issue in computational physics.
W-SLDA Toolkit quality
Within W-SLDA Toolkit we apply the following strategies to ensure a high level of confidence of the computation correctness:
Level 1: Inspection of the results
It is the most standard method (and in many groups the only one). Namely,
results are correct if they look to be correct.
While it may look to be rather a weak method of testing of computation correctness, in practice, it works very well. Typically, we know what to expect as the results (more or less), at least at the level of qualitative behavior. Physics is equipped with various methods, experiments, etc, and making extensive comparisons with them automatically leads to the inspection of sources of discrepancies (if present). From point of view of the implementation, the W-SLDA Toolkit delivers extensive support for results analysis.
Actions taken within W-SLDA Toolkit to support the level 1 quality checks:
- W-data format: conceptually easy data format that allows a variety of tools/languages to be used for data analysis.
- Integration with VisIt: advanced platform for data visualisation & analysis.
- Auxiliary tools & Extensions: we provide various examples of tested codes for data analysis.
Level 2: Internal tests
While Level 1 actions are sufficient to detect most of errors that lead to results that are incorrect at the qualitative level, they are not sufficient to assure the correctness of the code computation at the quantitative level. The next level of testing is
checking correctness of the computation for cases where solutions are know.
Developers of W-SLDA Toolkit constantly seek for cases which can be used as test cases.
Actions taken within W-SLDA Toolkit to support the level 2 quality checks:
- Testsuite: an automated system of executing test cases that checks correctness of computation at the quantitative level. The automatic tests are executed routinely after each new commit to the main engine of the code.
Level 3: Open-source & Transparency
The review process by external researchers is a crucial aspect in science. It is standard practice in the case of scientific publication. In the context of scientific codes, it less popular since it requires to make your code available to other. However, here we also observe that open-source/access movement get stronger with time in computational physics. It is next level method that supports the implementation of high-quality code. It steams from Linus's law:
given enough eyeballs, all bugs are shallow.
Actions taken within W-SLDA Toolkit to support the level 3 quality checks:
- The W-SLDA Toolkit is open-source code.
- Results reproducibility: Together with publications, we deliver reproducibility packs. Using them independent research may inspect the computation process from beginning to the end. As example see arXiv:2201.07626.