|
|
|
**Version>=2021.05.25**
|
|
|
|
|
|
|
|
W-SLDA Toolkit has implemented an automated tests system. Each version is a subject extensive testing process before release. The latest version (2022.01.05) was subject of more than 250 tests:
|
|
|
|
The W-SLDA Toolkit has implemented an automated test system. Each version undergoes extensive testing before release. The latest version (2022.01.05) was the subject of more than 250 tests:
|
|
|
|
```
|
|
|
|
--- Summary ---
|
|
|
|
TESTS: 258
|
|
|
|
OK: 258
|
|
|
|
FAIL: 0
|
|
|
|
```
|
|
|
|
Documentation below describes how to define and execute the tests - for developers.
|
|
|
|
The documentation below describes how to define and execute the tests for developers.
|
|
|
|
|
|
|
|
### WSLDA test system instruction
|
|
|
|
|
|
|
|
The documentation contains instructions, on how to prepare the tests used for the W-SLDA project purposes. The testing system performs all given tests according to the order set by the user, and then generates a report based on the results of the simulations.
|
|
|
|
The documentation includes instructions for preparing the tests used for the W-SLDA project. The testing system performs all provided tests in the order specified by the user, then generates a report based on the simulation results.
|
|
|
|
|
|
|
|
Tests can be prepared on the local machine, yet due to requirements of working memory, to be run actual calculations, access to the computing cluster is highly recommended.
|
|
|
|
Tests can be prepared on the local machine, but for running actual calculations, access to the computing cluster is highly recommended.
|
|
|
|
|
|
|
|
The W-SLDA project relies on the `wslda` repository, which includes the `testsuite` directory. Testsuite's directory is a working folder and all tests should be placed and executed there.
|
|
|
|
The W-SLDA project relies on the `wslda` repository, which includes the `testsuite` directory. The testsuite's directory is a working folder, and all tests should be placed and executed there.
|
|
|
|
### Preparation of the test list:
|
|
|
|
|
|
|
|
1. In the path `testsuite` prepare a text file named: `tlist.txt`.
|
|
|
|
2. The `tlist.txt` file contains information about the **directories with tests** that User intends to perform.
|
|
|
|
3. Comments can be placed in the `tlist.txt` file but each comment is preceded by a hash sign (\#). Hash sign is MANDATORY.
|
|
|
|
1. In the path `testsuite`, prepare a text file named: `tlist.txt`.
|
|
|
|
2. The `tlist.txt` file contains information about the **directories with tests** that the user intends to perform.
|
|
|
|
3. Comments can be placed in the `tlist.txt` file, but each comment is preceded by a hash sign (\#). Hash sign is MANDATORY.
|
|
|
|
4. Name of the **directory with tests** is elective, yet must be a string. Do not use any special character other than dot ".", dash "-" or underscore "_".
|
|
|
|
5. Each subsequent name of the directory must start from a new line in the `tlist.txt`. One line for one directory is MANDATORY.
|
|
|
|
5. Each subsequent name of the directory must start on a new line in the `tlist.txt`. One line for one directory is MANDATORY.
|
|
|
|
|
|
|
|
|
|
|
|
**Sample "tlist.txt" file format:**
|
| ... | ... | @@ -38,7 +38,6 @@ second_dir |
|
|
|
# Directory below will be processed as well:
|
|
|
|
third_dir # despite the comment left after the name of the test
|
|
|
|
|
|
|
|
|
|
|
|
fourth_dirrr # this directory will also run despite a comment or two blank lines left above
|
|
|
|
# dir_fifth # these tests will NOT run due to the comment hash sign (#)
|
|
|
|
# dir_six will fail either
|
| ... | ... | @@ -50,18 +49,18 @@ end # this will be considered as a group of tests to run |
|
|
|
|
|
|
|
### Preparation of the directory with tests:
|
|
|
|
|
|
|
|
1. When `tlist.txt` file is ready, User needs to prepare directories with tests in the `testsuite` directory.
|
|
|
|
2. Each subsequent directory may contain unlimited amount of tests.
|
|
|
|
1. When the `tlist.txt` file is ready, the user needs to prepare directories with tests in the `testsuite` directory.
|
|
|
|
2. Each subsequent directory may contain an unlimited number of tests.
|
|
|
|
3. Each directory name must match the name of the directory given in `tlist.txt`.
|
|
|
|
4. Each directory needs to contain **reference file** with expected values.
|
|
|
|
4. Each dir must contain a text file `test.desc`. It contains a list of tests and Unix commands to be run upon them.
|
|
|
|
The format of the text file `test.desc` is shown below.
|
|
|
|
5. `Test.desc` file contains assigned flags depending on the purpose. All flags are MANDATORY:
|
|
|
|
- `tag` - refers to the name of test (also an information for an User)
|
|
|
|
- `tag` - refers to the name of the test (also information for a user)
|
|
|
|
- `exec` - command to be executed
|
|
|
|
- `test` - name of the **output file** for each test
|
|
|
|
- `diff` - test is over and ready to compare **output file** values with the reference values.
|
|
|
|
6. `Test.desc` file needs no ending. Placing comments or blank lines are optional.
|
|
|
|
6. `Test.desc` file needs no ending. Placing comments or blank lines is optional.
|
|
|
|
|
|
|
|
**Example of test.desc file:**
|
|
|
|
|
| ... | ... | @@ -74,11 +73,11 @@ diff: test1.cmp ref.test1 |
|
|
|
|
|
|
|
### Start testing:
|
|
|
|
|
|
|
|
1. Switch directory to: `testsuite`
|
|
|
|
1. Switch directory to: `testsuite`.
|
|
|
|
2. Make sure that all directories are included, according to the `tlist.txt` file.
|
|
|
|
3. Make sure that all directories contain the necessary files, including description file `test.desc` and **reference file**.
|
|
|
|
4. If everything has been prepared correctly, run the test script with the command: `./testsuiteRun.sh`.
|
|
|
|
5. After the script is done, a text report will be received in the format: `user_data_hour`, so that every user could recognize his test.
|
|
|
|
5. After the script is done, a text report will be received in the format: `user_data_hour`, so that every user can recognize their test.
|
|
|
|
|
|
|
|
|
|
|
|
### Obtaining report:
|
| ... | ... | @@ -88,14 +87,14 @@ diff: test1.cmp ref.test1 |
|
|
|
3. The report is in the format of a 5-column text file: | Folder | Tag | Make | Run | Check |:
|
|
|
|
- `Folder` - name of dir
|
|
|
|
- `Tag` - name of test
|
|
|
|
- `Make` - string value: `OK` or `FAIL`. `OK` if W-SLDA toolkit worked successfully.
|
|
|
|
- `Run` - string value: `OK` or `FAIL`. 'OK' if **output file** from `test.desc` exists.
|
|
|
|
- `Make` - string value: `OK` or `FAIL`. `OK` if the W-SLDA toolkit worked successfully.
|
|
|
|
- `Run` - string value: `OK` or `FAIL`. 'OK' if the **output file** from `test.desc` exists.
|
|
|
|
- `Check` - string value: `OK` or `FAIL`. 'OK' when ALL **output values** compared with **reference values**
|
|
|
|
are below the tolerance specified in the reference file.
|
|
|
|
|
|
|
|
`FAIL` value will be obtained if otherwise (for each flag respectively).
|
|
|
|
`FAIL` value will be obtained if otherwise (for each flag, respectively).
|
|
|
|
|
|
|
|
After receiving report, it is recommended to copy it to your local device and remove it from the `testsuite` directory.
|
|
|
|
After receiving the report, it is recommended to copy it to your local device and remove it from the `testsuite` directory.
|
|
|
|
|
|
|
|
# Contribution
|
|
|
|
The Testsuite was implemented by:
|
| ... | ... | |