Results-reproducibility.md

+[[_TOC_]]
 # Introduction
 Results reproducibility is a critical issue in science. It has already been noted that in many cases, reproducing your results even after a few months (the typical time scale of the referee process) may be challenging. In most cases, having the same code version is insufficient, but you also need precise knowledge about the input parameters used, and the same input data must be provided. Since the standard methodology in science is based on try-and-fail methods, typically, the researcher ends up with many datasets. Only a few are released for publication, while others serve as experimental runs. Under such conditions, tracking changes introduced to codes in the research process becomes problematic. The W-SLDA Toolkit implements a methodology that does it automatically and allows for reproducing the results (up to machine precision). Namely, the generated **results** are always accompanied by the **reproducibility pack**, where complete information needed to reproduce them is included.   
 ![reproducibility](uploads/8ee07ac131aa636f0b6e041cc0948cac/reproducibility.png)
@@ -49,3 +50,37 @@ These are analogs of [printf](http://www.cplusplus.com/reference/cstdio/printf/)
 
 To learn more about good practices related to results reproducibility issues, see:
 * [Build a Reproducible and Maintainable Data Science Project](https://khuyentran1401.github.io/reproducible-data-science/README.html)
+
+# How to reproduce the results
+1. _Get the code_: 
+Check one of the provided `*.h` files to determine the version of code that should be used to reproduce the results. In general, W-SLDA assures backward compatibility and it most cases you can use the latest version o the code. See [here](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/releases) for a list of releases.
+
+2. _Create a working folder_:
+Use template folders (`st-project-template` or `td-project-template`) to create an empty project.
+
+3. _Overwrite user-defined files_:
+Overwrite the user-defined files `*.h` and the input file by those provided in the reproducibility pack. 
+
+# List of reproducibility packs
+Since 2022, reproducibility packs have been provided for all our papers where new results generated by the W-SLDA are presented. For the list of our publications and links to the reproducibility packs, see [here](https://wslda.fizyka.pw.edu.pl/index.php/Gallery).
+
+# Known issues with existing reproducibility packs
+## New J. Phys. 25, 033013 (2023)
+**Problem** with a compilation of type:
+```
+/net/software/v1/software/binutils/2.38-GCCcore-11.3.0/bin/ld: obj//wslda_functionals.o: in function `compute_potentials_bdg':
+wslda_functionals.c:(.text+0x1ecc): undefined reference to `scattering_length'
+/net/software/v1/software/binutils/2.38-GCCcore-11.3.0/bin/ld: obj//wslda_functionals.o: in function `compute_potentials_sldae':
+wslda_functionals.c:(.text+0x2c27): undefined reference to `scattering_length'
+/net/software/v1/software/binutils/2.38-GCCcore-11.3.0/bin/ld: obj//wslda_functionals.o: in function `compute_energy_sldae':
+wslda_functionals.c:(.text+0x41c9): undefined reference to `scattering_length'
+collect2: error: ld returned 1 exit status
+make: *** [/net/pr2/projects/plgrid/plgg_ntg_wut/share/cold-atoms//hpc-engine/mk.st:95: d2d] Error 1
+```
+**Solution**: Add to the `problem-definition.h` function:
+```c
+double scattering_length(int ix, int iy, int iz, int it, double *params, size_t extra_data_size, void *extra_data)
+{
+    return input->sclgth; // by default return value from input file.
+}
+```