Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • wslda wslda
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • wtools
  • wsldawslda
  • Wiki
  • Results reproducibility

Results reproducibility · Changes

Page history
Update Results reproducibility authored Mar 26, 2025 by Gabriel Wlazłowski's avatar Gabriel Wlazłowski
Hide whitespace changes
Inline Side-by-side
Results-reproducibility.md
View page @ fe7c2801
# Introduction
Results reproducibility is a critical issue in science. It has already been noted that in many cases, reproducing your results even after a few months (the typical time scale of the referee process) may be challenging. In most cases, having the same code version is insufficient, but you also need precise knowledge about the input parameters used, and the same input data must be provided. Since the standard methodology in science is based on try-and-fail methodology, typically, the researcher ends up with many datasets. Only a few are released for publication, while others serve as experimental runs. Under such conditions, tracking changes introduced to codes in the research process becomes problematic. W-SLDA implements a methodology that does it automatically and allows for reproducing the results (up to machine precision). Namely, the generated **results** are always accompanied by the **reproducibility pack**, where complete information needed to reproduce them is included.
Results reproducibility is a critical issue in science. It has already been noted that in many cases, reproducing your results even after a few months (the typical time scale of the referee process) may be challenging. In most cases, having the same code version is insufficient, but you also need precise knowledge about the input parameters used, and the same input data must be provided. Since the standard methodology in science is based on try-and-fail methods, typically, the researcher ends up with many datasets. Only a few are released for publication, while others serve as experimental runs. Under such conditions, tracking changes introduced to codes in the research process becomes problematic. The W-SLDA Toolkit implements a methodology that does it automatically and allows for reproducing the results (up to machine precision). Namely, the generated **results** are always accompanied by the **reproducibility pack**, where complete information needed to reproduce them is included.
![reproducibility](uploads/8ee07ac131aa636f0b6e041cc0948cac/reproducibility.png)
For the meaning of each file, see [here](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/wikis/Output%20files).
# W-SLDA mechanism of results reproducibility
Developers of W-SLDA Toolkit recognize the need for intrinsically implemented support that will simplify the process of reproducing the results. To comply with this requirement, the following mechanism has been implemented (called a reproducibility pack):
1. Each file generated by W-SLDA Toolkit in the header provides basic info about the code version that has been used; for example, the header of `wlog` file may look like:
Developers of the W-SLDA Toolkit recognize the need for intrinsically implemented support that will simplify the process of reproducing the results. To comply with this requirement, the following mechanism has been implemented (called a reproducibility pack):
1. Each file generated by the W-SLDA Toolkit in the header provides basic info about the code version that has been used; for example, the header of `wlog` file may look like:
```
# CREATION TIME OF THE LOG: Sun Feb 7 15:29:44 2021
# EXECUTION COMMAND : ./st-wslda-2d input.txt
......@@ -15,19 +15,19 @@ Developers of W-SLDA Toolkit recognize the need for intrinsically implemented su
# COMPILATION DATE & TIME : Feb 7 2021, 15:19:57
```
2. When executing the code, all user-definable files are recreated and attached to the output-set. For example, if the user set `outprefix` as `test`, then among output files there will be:
2. When executing the code, all user-definable files are recreated and attached to the output-set. For example, if the user sets `outprefix` as `test`, then among output files, there will be:
```bash
test_input.txt # input file used for calculations
test_predefines.h # predefines selected at compilation stage
test_problem-definition.h # user's definition of the problem
test_logger.h # user's logger
test_machine.h # machine configuration that was used in calculations
test_machine.h # machine configuration used in the calculations
test.stdout # standard output generated by the code
test_checkpoint.dat.init # checkpoint file that was used as input (st codes only)
test_extra_data.dat # Binary file with the extra_data array (if provided)
test_reprowf.tar # reproducibility pack for restoring wave-functions that were used as input (td codes only)
test_reprowf.tar # reproducibility pack for restoring wave functions that were used as input (td codes only)
```
This provides the full information required to reproduce your results (up to machine precision).
This provides the complete information required to reproduce your results (up to machine precision).
# Good practices
1. For each project, use a separate folder; do not mix results from various projects in the same folder. Use a meaningful name for the folder.
......@@ -47,5 +47,5 @@ void wfprintf(FILE *stream, const char * format, ... );
```
These are analogs of [printf](http://www.cplusplus.com/reference/cstdio/printf/) and [fprintf](http://www.cplusplus.com/reference/cstdio/fprintf/) with the difference that the message will also be added to `outprefix.stdout`.
To learn more about good practices related to results reproducibility issues see:
To learn more about good practices related to results reproducibility issues, see:
* [Creating Reproducible Data Science Projects](https://towardsdatascience.com/creating-reproducible-data-science-projects-1fa446369386)
\ No newline at end of file
Clone repository
  • API version
  • Automatic interpolations
  • Auxiliary tools
  • Browsing the code
  • Broyden algorithm
  • C and CUDA
  • Campaign of calculations
  • Checking correctness of settings
  • Chemical potentials control
  • Code & Results quality
  • Common failures of static codes
  • Common failures of time dependent codes
  • Computation domain
  • Configuring GPU machine
  • Constraining densities and potentials
View All Pages