Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • wslda wslda
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • CI/CD
    • Repository
    • Value stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • wtools
  • wsldawslda
  • Wiki
  • Results reproducibility

Results reproducibility · Changes

Page history
Update Results reproducibility authored May 20, 2021 by Gabriel Wlazłowski's avatar Gabriel Wlazłowski
Hide whitespace changes
Inline Side-by-side
Results-reproducibility.md
View page @ 830cd09a
# Introduction
Results reproducibility is a very important issue in science. It has been already noted that in many cases reproducing your own results even after a few months (typical time scale of referee process) may be challenging. It is because in most cases it is not sufficient to have the same version of the code, but you also need precise knowledge about input parameters that were used.
Since the standard methodology in science is based on _try and fail_ methodology, typically at the end we end up with many datasets, and only a few of them is released to publication finally, while others serve as _experimental runs_.
Since the standard methodology in science is based on _try and fail_ methodology, typically at the end we end up with many datasets, and only a few of them is released to publication finally, while others serve as _experimental runs_. W-SLDA implements a methodology that allows for results reproductions. Namely, the generated **results** are always accompanied by the **reproducibility pack**:
![reproducibility](uploads/c7145a8680529be0dae0212b3d81764b/reproducibility.png)
# W-SLDA mechanism of results reproducibility
Developers of W-SLDA Toolkit recognize the need for intrinsically implemented support that will simplify the process of reproducing of the results. To comply with this requirement following mechanism has been implemented:
1. Each file generated by W-SLDA Toolkit in the header provides basic info about the code version that has been used, for example header of `wlog` file may look like:
Developers of W-SLDA Toolkit recognize the need for intrinsically implemented support that will simplify the process of reproducing of the results. To comply with this requirement following mechanism has been implemented (called as reproducibility pack):
1. Each file generated by W-SLDA Toolkit in the header provides basic info about the code version that has been used, for example, header of `wlog` file may look like:
```
# CREATION TIME OF THE LOG: Sun Feb 7 15:29:44 2021
# EXECUTION COMMAND : ./st-wslda-2d input.txt
......@@ -19,6 +20,8 @@ test_predefines.h # predefines selected at compilation stage
test_problem-definition.h # user's definition of the problem
test_logger.h # user's logger
test.stdout # standard output generated by the code
test_checkpoint.dat.init # checkpoint point that was used as input (st codes only)
test_reprowf.tar # reproducibility pack for restoring wave-functions that were used as input (td codes only)
```
This provides full information (apart from any external data added by user) required to reproduce your results.
......@@ -26,7 +29,7 @@ This provides full information (apart from any external data added by user) requ
1. For each project use a separate folder, do not mix results from various project in the same files. Use a meaningful name for folders.
2. Use meaningful `outprefix` names.
3. Do not modify output files, except `wtxt` file. This one is designed to store various metadata information, including your comments. `wtxt` file is easy to reproduce in case if you destroy it accidentally, which is not the case of other files. Add your comments/remarks/etc in form of comments starting with `#`.
4. When copying results to a new location/machine, copy all files assisted with the run. The simplest way is to execute the command (for more info see [here](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/wikis/Output%20files#copying-results-to-a-new-location)):
4. When copying results to a new location/machine, copy all files assisted with the run. The simplest way is to execute the command (for more info see [here](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/wikis/Output%20files#copying-results-to-a-new-location)):
```bash
scp outprefix* new_location
```
......
Clone repository
  • API version
  • Automatic interpolations
  • Auxiliary tools
  • Browsing the code
  • Broyden algorithm
  • C and CUDA
  • Campaign of calculations
  • Checking correctness of settings
  • Chemical potentials control
  • Code & Results quality
  • Common failures of static codes
  • Common failures of time dependent codes
  • Computation domain
  • Configuring GPU machine
  • Constraining densities and potentials
View All Pages