Introduction

Results reproducibility is a very important issue in science. It has been already noted that in many cases reproducing your own results even after a few months (typical time scale of referee process) may be challenging. It is because in most cases it is not sufficient to have the same version of the code, but you also need precise knowledge about input parameters that were used. Since the standard methodology in science is based on try and fail methodology, typically at the end we end up with many datasets, and only a few of them is released to publication finally, while others serve as experimental runs.

W-SLDA mechanism of results reproducibility

Developers of W-SLDA Toolkit recognize the need for support that will simplify the process of reproducing of results. To comply with this requirement following mechanism has been implemented:

1. Each file generated by W-SLDA Toolkit in the header provides basic info about the code version that has been used, for example header of wlog file may look like:

# CREATION TIME OF THE LOG: Sun Feb  7 15:29:44 2021
# EXECUTION COMMAND       : ./st-wslda-2d input.txt
# CODE NAME               : "W-SLDA-TOOLKIT"
# VERSION OF THE CODE     : 2021.01.27
# COMPILATION DATE & TIME : Feb  7 2021, 15:19:57

2. When executing the code, all user-definable files are recreated and attached to the data set. For example, if the user set outprefix as test, then among output files there will be:

test_input.txt             # input file used for calculations
test_predefines.h          # predefines selected at compilation stage
test_problem-definition.h  # user's definition of the problem 
test_logger.h              # user's logger

This provides full information (apart from any external data added by user) required to reproduce your results.

Good practices

1. For each project use a separate folder, do not mix results from various project in the same files. Use a meaningful name for folders.
2. Use meaningful outprefix names.
3. Do not modify output files, except wtxt file. This one is designed to store various metadata information, including your comments. wtxt file is easy to reproduce in case if you destroy it accidentally, which is not the case of other files. Add your comments/remarks/ect in form of comments starting with #.

To learn more about good practices related to results reproducibility issue see:

• Creating Reproducible Data Science Projects