... | @@ -2,6 +2,7 @@ |
... | @@ -2,6 +2,7 @@ |
|
## Static codes (st-wslda)
|
|
## Static codes (st-wslda)
|
|
* C compiler (eg. `gcc`, intel compiler `icc`, ...)
|
|
* C compiler (eg. `gcc`, intel compiler `icc`, ...)
|
|
* MPI C compiler (eg. `mpicc`, intel compiler `mpiicc`, ...)
|
|
* MPI C compiler (eg. `mpicc`, intel compiler `mpiicc`, ...)
|
|
|
|
* python
|
|
* [FFTW library](http://www.fftw.org/)
|
|
* [FFTW library](http://www.fftw.org/)
|
|
* LAPACK library
|
|
* LAPACK library
|
|
* ScaLAPACK library
|
|
* ScaLAPACK library
|
... | @@ -12,12 +13,13 @@ |
... | @@ -12,12 +13,13 @@ |
|
|
|
|
|
*Note 2*: LAPACK and ScaLAPACK libraries are part of Intel® Math Kernel Library.
|
|
*Note 2*: LAPACK and ScaLAPACK libraries are part of Intel® Math Kernel Library.
|
|
|
|
|
|
*Note 3*: It is recommended to use `st-wslda` with ELPA support if possible. It provides substantial speed-up of the computation process.
|
|
*Note 3*: It is recommended to use `st-wslda` with ELPA support if possible. It provides substantial speed-up of the computation process. See [here](ELPA installation guide) for more instructions related to installing ELPA.
|
|
|
|
|
|
## Time-dependent codes (td-wslda)
|
|
## Time-dependent codes (td-wslda)
|
|
* C compiler (eg. `gcc`, intel compiler `icc`, ...)
|
|
* C compiler (eg. `gcc`, intel compiler `icc`, ...)
|
|
* MPI C compiler (eg. `mpicc`, intel compiler `mpiicc`, ...)
|
|
* MPI C compiler (eg. `mpicc`, intel compiler `mpiicc`, ...)
|
|
* [CUDA compiler](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html)
|
|
* [CUDA compiler](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html)
|
|
|
|
* python
|
|
|
|
|
|
*Note 1*: Time-dependent codes **require** systems accelerated by GPUs. `td-wslda` codes cannot run on standard systems based on CPUs only.
|
|
*Note 1*: Time-dependent codes **require** systems accelerated by GPUs. `td-wslda` codes cannot run on standard systems based on CPUs only.
|
|
|
|
|
... | @@ -29,11 +31,35 @@ git clone https://gitlab.fizyka.pw.edu.pl/wtools/wslda.git |
... | @@ -29,11 +31,35 @@ git clone https://gitlab.fizyka.pw.edu.pl/wtools/wslda.git |
|
Optionally you can download it manually from [main webpage](https://gitlab.fizyka.pw.edu.pl/wtools/wslda).
|
|
Optionally you can download it manually from [main webpage](https://gitlab.fizyka.pw.edu.pl/wtools/wslda).
|
|
The toolkit is contained in `wslda` folder, see [here](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/wikis/Folders-structure) for more info about content.
|
|
The toolkit is contained in `wslda` folder, see [here](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/wikis/Folders-structure) for more info about content.
|
|
|
|
|
|
|
|
# Compaling standard libraries and tools
|
|
|
|
The toolkit is equipped with various [libs](https://gitlab.fizyka.pw.edu.pl/wtools) and tools that support the computation process. See [here](Auxiliary tools) for a list of presently available tools. While it is not mandatory we recommend compiling them at the stage of the installation process. You can do it by executing the provided script in the main folder:
|
|
|
|
```
|
|
|
|
./install-libs.sh
|
|
|
|
```
|
|
|
|
Once the compilation is finished you will get instruction how to set system variable in order to be able use W-SLDA Toolkit, example
|
|
|
|
```
|
|
|
|
gabrielw@e1135:~/grant_518/project_data/wslda$ ./install-libs.sh
|
|
|
|
Installing libs...
|
|
|
|
# Making lib/wderiv..
|
|
|
|
# ...
|
|
|
|
make: Leaving directory `/mnt/storage_2/project_data/grant_518/cold-atoms/tools'
|
|
|
|
# ==========================================================================
|
|
|
|
# =========================== SET ENVIRONMENT ==============================
|
|
|
|
# ==========================================================================
|
|
|
|
# Add to your .bashrc
|
|
|
|
|
|
|
|
export WSLDA=/mnt/storage_2/project_data/grant_518/wslda
|
|
|
|
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/mnt/storage_2/project_data/grant_518/wslda/lib/wdata:/mnt/storage_2/project_data/grant_518/wslda/lib/wderiv:/mnt/storage_2/project_data/grant_518/wslda/lib/winterp
|
|
|
|
export PATH=$PATH:/mnt/storage_2/project_data/grant_518/wslda/lib/wdata/bin:/mnt/storage_2/project_data/grant_518/wslda/tools/bin
|
|
|
|
```
|
|
|
|
|
|
|
|
Share `export` statements with other users of the toolkit, so they can also use it. You need to make `wslda` folder readable for other members of the group.
|
|
|
|
|
|
# Creating Makefiles
|
|
# Creating Makefiles
|
|
W-SLDA exploits a working model based on templates.
|
|
W-SLDA exploits a working model based on templates.
|
|
User starts with the template of a project where items like external potential, external pairing potential, external velocity must be provided in C language. Templates are localized in folders:
|
|
User starts with the template of a project where items like external potential, external pairing potential, external velocity must be provided in C language. Templates are localized in folders:
|
|
* [st-project-template](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/st-project-template) for static problems (`st-wslda` codes)
|
|
* [st-project-template](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/st-project-template) for static problems (`st-wslda` codes)
|
|
* [td-project-template](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/td-project-template) for time dependent problems (`td-wslda` codes)
|
|
* [td-project-template](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/td-project-template) for time-dependent problems (`td-wslda` codes)
|
|
|
|
|
|
Within this folders `Makefile` compatible with your computing system must be provided.
|
|
Within this folders `Makefile` compatible with your computing system must be provided.
|
|
## Makefile for static codes
|
|
## Makefile for static codes
|
... | @@ -75,13 +101,18 @@ In the example, we also assume that ELPA lib will be utilized, and paths to head |
... | @@ -75,13 +101,18 @@ In the example, we also assume that ELPA lib will be utilized, and paths to head |
|
```bash
|
|
```bash
|
|
make
|
|
make
|
|
```
|
|
```
|
|
|
|
|
|
|
|
To clear your compilation use:
|
|
|
|
```bash
|
|
|
|
make clean
|
|
|
|
```
|
|
|
|
|
|
*Note 1*: The ELPA Library must be in addition activated in [predefines.h](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/blob/public/st-project-template/predefines.h) file. To learn more about ELPA see [here](Setting up diagonalization engine).
|
|
*Note 1*: The ELPA Library must be in addition activated in [predefines.h](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/blob/public/st-project-template/predefines.h) file. To learn more about ELPA see [here](Setting up diagonalization engine).
|
|
|
|
|
|
*Note 2*: `st-wslda` codes utilizes LAPACK and ScaLAPACK libraries. The standard method of calling fortran functions from C requires supplementing routine's names by `_` at the end, i.e fortran routine `DGEMM` is called from C as `dgemm_`. However in cases of some systems/compilers (very rare) it is not needed. If this is the case of your system then you need to add to `CFLAGS` flag
|
|
*Note 2*: `st-wslda` codes utilizes LAPACK and ScaLAPACK libraries. The standard method of calling Fortran functions from C requires supplementing routine's names by `_` at the end, i.e fortran routine `DGEMM` is called from C as `dgemm_`. However in cases of some systems/compilers (very rare) it is not needed. If this is the case of your system then you need to add to `CFLAGS` flag
|
|
`-DFOTRAN_NO_UNDERSCORE`.
|
|
`-DFOTRAN_NO_UNDERSCORE`.
|
|
|
|
|
|
*Recommendation*: Typically (super)computers utilize `module load` system for configuring the programming environment. We recommend placing a list of in a separate file, for example `env.sh`:
|
|
*Recommendation*: Typically (super)computers utilize `module load` system for configuring the programming environment. We recommend placing a list of modules in a separate file, for example `env.sh`:
|
|
```bash
|
|
```bash
|
|
# Content of env.sh
|
|
# Content of env.sh
|
|
module load daint-gpu
|
|
module load daint-gpu
|
... | @@ -90,6 +121,7 @@ module load cudatoolkit |
... | @@ -90,6 +121,7 @@ module load cudatoolkit |
|
module load craype-accel-nvidia60
|
|
module load craype-accel-nvidia60
|
|
module load cray-fftw
|
|
module load cray-fftw
|
|
export LD_LIBRARY_PATH=/project/pr91/share/lib:$LD_LIBRARY_PATH
|
|
export LD_LIBRARY_PATH=/project/pr91/share/lib:$LD_LIBRARY_PATH
|
|
|
|
module list
|
|
```
|
|
```
|
|
The compilation process looks like this:
|
|
The compilation process looks like this:
|
|
```bash
|
|
```bash
|
... | @@ -154,17 +186,10 @@ source ./env.sh |
... | @@ -154,17 +186,10 @@ source ./env.sh |
|
make
|
|
make
|
|
```
|
|
```
|
|
|
|
|
|
# Compiling W-DATA library and visit plugin (optional)
|
|
# Compiling visit plugin (optional)
|
|
It is recommended to generate a library for w-data managing format:
|
|
|
|
```bash
|
|
|
|
cd lib/wdata/
|
|
|
|
make libs
|
|
|
|
```
|
|
|
|
Visit plugin is located in [lib-wdata/visit-plugin](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/lib-wdata/visit-plugin).
|
|
|
|
Follow instruction from [README.txt](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/lib-wdata/visit-plugin/README.txt) file to install it.
|
|
|
|
|
|
|
|
# Compiling auxiliary tools (optional)
|
|
Visit plugin is located in [lib/wdata/visit-plugin](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/lib/wdata/visit-plugin).
|
|
In folder [tools](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/tools) we provide various useful tools. See [here](Auxiliary tools) for list of presently available tools. In order to compile them, edit `Makefile` and execute `make`. After compilation binary files will be stored in `wslda/tools/bin`.
|
|
Follow instruction from [README.txt](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/lib/wdata/visit-plugin/README.txt) file to install it.
|
|
|
|
|
|
# Creating templates of submission scripts (optional)
|
|
# Creating templates of submission scripts (optional)
|
|
It is recommended to place in template folders submission scripts suitable for your computer. Examples of submission scripts can be found in the folder [job-scripts-templates](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/job-scripts-templates).
|
|
It is recommended to place in template folders submission scripts suitable for your computer. Examples of submission scripts can be found in the folder [job-scripts-templates](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/job-scripts-templates).
|
... | @@ -178,5 +203,6 @@ chmod -R a+r wslda |
... | @@ -178,5 +203,6 @@ chmod -R a+r wslda |
|
Provide to users path to W-SLDA Toolkit. Recommend for users to place it in `.bashrc` file, example:
|
|
Provide to users path to W-SLDA Toolkit. Recommend for users to place it in `.bashrc` file, example:
|
|
```bash
|
|
```bash
|
|
export WSLDA=/project/pr91/share/wslda/
|
|
export WSLDA=/project/pr91/share/wslda/
|
|
export PATH=/project/pr91/share/wslda/tools/bin:$PATH
|
|
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/project/pr91/share/wslda/lib/wdata:/project/pr91/share/wslda/lib/wderiv:/project/pr91/share/wslda//lib/winterp
|
|
``` |
|
export PATH=$PATH:/project/pr91/share/wslda/lib/wdata/bin:/project/pr91/share/wslda/tools/bin
|
|
|
|
``` |
|
|
|
\ No newline at end of file |