|
|
|
**Version>2021.05.xx**
|
|
|
|
|
|
|
|
W-SLDA Toolkit has implemented automated tests systems. Each version is subject extensive testing process before release. The lates version was subject of more that 200 tests:
|
|
|
|
```
|
|
|
|
--- Summary ---
|
|
|
|
TESTS: 210
|
|
|
|
OK: 210
|
|
|
|
FAIL: 0
|
|
|
|
Done.
|
|
|
|
--- 20285.29 seconds ---
|
|
|
|
```
|
|
|
|
Documentation below describes how to define and execute the tests - for developers.
|
|
|
|
|
|
|
|
### WSLDA test system instruction
|
|
|
|
|
|
|
|
The documentation contains instructions, on how to prepare the tests used for the W-SLDA project purposes. The testing system performs all given tests according to the order set by the user, and then generates a report based on the results of the simulations.
|
|
|
|
|
|
|
|
Tests can be prepared on the local machine, yet due to requirements of working memory, to be run actual calculations, access to the computing cluster is highly recommended.
|
|
|
|
|
|
|
|
The W-SLDA project relies on the `wslda` repository, which includes the `testsuite` directory. Testsuite's directory is a working folder and all tests should be placed and executed there.
|
|
|
|
### Preparation of the test list:
|
|
|
|
|
|
|
|
1. In the path `testsuite` prepare a text file named: `tlist.txt`.
|
|
|
|
2. The `tlist.txt` file contains information about the **directories with tests** that User intends to perform.
|
|
|
|
3. Comments can be placed in the `tlist.txt` file but each comment is preceded by a hash sign (\#). Hash sign is MANDATORY.
|
|
|
|
4. Name of the **directory with tests** is elective, yet must be a string. Do not use any special character other than dot ".", dash "-" or underscore "_".
|
|
|
|
5. Each subsequent name of the directory must start from a new line in the `tlist.txt`. One line for one directory is MANDATORY.
|
|
|
|
|
|
|
|
|
|
|
|
**Sample "tlist.txt" file format:**
|
|
|
|
|
|
|
|
```bash
|
|
|
|
# User: John
|
|
|
|
# Date: 01/01/1970
|
|
|
|
|
|
|
|
# A directory with tests below is the first directory to process.
|
|
|
|
first_dir
|
|
|
|
# Here is the next group of tests to be run.
|
|
|
|
second_dir
|
|
|
|
# Directory below will be processed as well:
|
|
|
|
third_dir # despite the comment left after the name of the test
|
|
|
|
|
|
|
|
|
|
|
|
fourth_dirrr # this directory will also run despite a comment or two blank lines left above
|
|
|
|
# dir_fifth # these tests will NOT run due to the comment hash sign (#)
|
|
|
|
# dir_six will fail either
|
|
|
|
|
|
|
|
end # this will be considered as a group of tests to run
|
|
|
|
#end # this will not be considered as a group of tests to run
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
### Preparation of the directory with tests:
|
|
|
|
|
|
|
|
1. When `tlist.txt` file is ready, User needs to prepare directories with tests in the `testsuite` directory.
|
|
|
|
2. Each subsequent directory may contain unlimited amount of tests.
|
|
|
|
3. Each directory name must match the name of the directory given in `tlist.txt`.
|
|
|
|
4. Each directory needs to contain **reference file** with expected values.
|
|
|
|
4. Each dir must contain a text file `test.desc`. It contains a list of tests and Unix commands to be run upon them.
|
|
|
|
The format of the text file `test.desc` is shown below.
|
|
|
|
5. `Test.desc` file contains assigned flags depending on the purpose. All flags are MANDATORY:
|
|
|
|
- `tag` - refers to the name of test (also an information for an User)
|
|
|
|
- `exec` - command to be executed
|
|
|
|
- `test` - name of the **output file** for each test
|
|
|
|
- `diff` - test is over and ready to compare **output file** values with the reference values.
|
|
|
|
6. `Test.desc` file needs no ending. Placing comments or blank lines are optional.
|
|
|
|
|
|
|
|
**Example of test.desc file:**
|
|
|
|
|
|
|
|
```bash
|
|
|
|
tag: test1
|
|
|
|
exec: make 1d
|
|
|
|
test: test1.cmp
|
|
|
|
diff: test1.cmp ref.test1
|
|
|
|
```
|
|
|
|
|
|
|
|
### Start testing:
|
|
|
|
|
|
|
|
1. Switch directory to: `testsuite`
|
|
|
|
2. Make sure that all directories are included, according to the `tlist.txt` file.
|
|
|
|
3. Make sure that all directories contain the necessary files, including description file `test.desc` and **reference file**.
|
|
|
|
4. If everything has been prepared correctly, run the test script with the command: `./testsuiteRun.sh`.
|
|
|
|
5. After the script is done, a text report will be received in the format: `user_data_hour`, so that every user could recognize his test.
|
|
|
|
|
|
|
|
|
|
|
|
### Obtaining report:
|
|
|
|
|
|
|
|
1. The report will be performed only at the very end of the script operation. Interrupting the program will provide no results.
|
|
|
|
2. The report will be generated regardless of the correctness of the test execution.
|
|
|
|
3. The report is in the format of a 5-column text file: | Folder | Tag | Make | Run | Check |:
|
|
|
|
- `Folder` - name of dir
|
|
|
|
- `Tag` - name of test
|
|
|
|
- `Make` - string value: `OK` or `FAIL`. `OK` if W-SLDA toolkit worked successfully.
|
|
|
|
- `Run` - string value: `OK` or `FAIL`. 'OK' if **output file** from `test.desc` exists.
|
|
|
|
- `Check` - string value: `OK` or `FAIL`. 'OK' when ALL **output values** compared with **reference values**
|
|
|
|
are below the tolerance specified in the reference file.
|
|
|
|
|
|
|
|
`FAIL` value will be obtained if otherwise (for each flag respectively).
|
|
|
|
|
|
|
|
After receiving report, it is recommended to copy it to your local device and remove it from the `testsuite` directory.
|
|
|
|
|
