|
|
# Introduction
|
|
|
Results reproducibility is a very important issue in science. It has been already noted that in many cases reproducing your own results even after a few months (typical time scale of referee process) may be challenging. It is because in most cases it is not sufficient to have the same version of the code, but you also need precise knowledge about input parameters that were used.
|
|
|
Since the standard methodology in science is based on _try and fail_ methodology, typically at the end we end up with many datasets, and only a few of them is released to publication finally, while others serve as _experimental runs_.
|
|
|
Since the standard methodology in science is based on _try and fail_ methodology, typically at the end we end up with many datasets, and only a few of them is released to publication finally, while others serve as _experimental runs_. W-SLDA implements a methodology that allows for results reproductions. Namely, the generated **results** are always accompanied by the **reproducibility pack**:
|
|
|
![reproducibility](uploads/c7145a8680529be0dae0212b3d81764b/reproducibility.png)
|
|
|
|
|
|
# W-SLDA mechanism of results reproducibility
|
|
|
Developers of W-SLDA Toolkit recognize the need for intrinsically implemented support that will simplify the process of reproducing of the results. To comply with this requirement following mechanism has been implemented:
|
|
|
1. Each file generated by W-SLDA Toolkit in the header provides basic info about the code version that has been used, for example header of `wlog` file may look like:
|
|
|
Developers of W-SLDA Toolkit recognize the need for intrinsically implemented support that will simplify the process of reproducing of the results. To comply with this requirement following mechanism has been implemented (called as reproducibility pack):
|
|
|
1. Each file generated by W-SLDA Toolkit in the header provides basic info about the code version that has been used, for example, header of `wlog` file may look like:
|
|
|
```
|
|
|
# CREATION TIME OF THE LOG: Sun Feb 7 15:29:44 2021
|
|
|
# EXECUTION COMMAND : ./st-wslda-2d input.txt
|
... | ... | @@ -19,6 +20,8 @@ test_predefines.h # predefines selected at compilation stage |
|
|
test_problem-definition.h # user's definition of the problem
|
|
|
test_logger.h # user's logger
|
|
|
test.stdout # standard output generated by the code
|
|
|
test_checkpoint.dat.init # checkpoint point that was used as input (st codes only)
|
|
|
test_reprowf.tar # reproducibility pack for restoring wave-functions that were used as input (td codes only)
|
|
|
```
|
|
|
This provides full information (apart from any external data added by user) required to reproduce your results.
|
|
|
|
... | ... | @@ -26,7 +29,7 @@ This provides full information (apart from any external data added by user) requ |
|
|
1. For each project use a separate folder, do not mix results from various project in the same files. Use a meaningful name for folders.
|
|
|
2. Use meaningful `outprefix` names.
|
|
|
3. Do not modify output files, except `wtxt` file. This one is designed to store various metadata information, including your comments. `wtxt` file is easy to reproduce in case if you destroy it accidentally, which is not the case of other files. Add your comments/remarks/etc in form of comments starting with `#`.
|
|
|
4. When copying results to a new location/machine, copy all files assisted with the run. The simplest way is to execute the command (for more info see [here](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/wikis/Output%20files#copying-results-to-a-new-location)):
|
|
|
4. When copying results to a new location/machine, copy all files assisted with the run. The simplest way is to execute the command (for more info see [here](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/wikis/Output%20files#copying-results-to-a-new-location)):
|
|
|
```bash
|
|
|
scp outprefix* new_location
|
|
|
```
|
... | ... | |