| ... | ... | @@ -10,7 +10,9 @@ |
|
|
|
|
|
|
|
*Note 1*: compilers and libraries (FFTW, LAPACK, ScaLAPACK) are installed by default on most of computing clusters. If computing system is equipped with GPUs most likely CUDA Toolkit is also installed. On many systems ELPA is also installed by default. If not you can compile it yourself.
|
|
|
|
|
|
|
|
*Note 2*: It is recommend to use `st-wslda` with ELPA support if possible. It provides substantial speed-up of computation process.
|
|
|
|
*Note 2*: LAPACK and ScaLAPACK libraries are part of Intel® Math Kernel Library.
|
|
|
|
|
|
|
|
*Note 3*: It is recommend to use `st-wslda` with ELPA support if possible. It provides substantial speed-up of computation process.
|
|
|
|
|
|
|
|
## Time-dependent codes (td-wslda)
|
|
|
|
* C compiler (eg. `gcc`, intel compiler `icc`, ...)
|
| ... | ... | @@ -27,5 +29,142 @@ git clone https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda.git |
|
|
|
Optionally you can download it manually from [main webpage](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda).
|
|
|
|
The toolkit is contained in `wslda` folder, see [here](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/wikis/Folders-structure) for more info about content.
|
|
|
|
|
|
|
|
# Configuring template folders
|
|
|
|
In folders todo |
|
|
\ No newline at end of file |
|
|
|
# Creating Makefiles
|
|
|
|
W-SLDA exploits working model based on templates.
|
|
|
|
User starts with template of a project where items like external potential, external pairing potential, external velocity must be provided in C language. Templates are localized in folders:
|
|
|
|
* [st-project-template](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/st-project-template) for static problems (`st-wslda` codes)
|
|
|
|
* [td-project-template](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/td-project-template) for time dependent problems (`td-wslda` codes)
|
|
|
|
|
|
|
|
Within this folders `Makefile` compatible with your computing system must be provided.
|
|
|
|
## Makefile for static codes
|
|
|
|
Operation below apply to folders
|
|
|
|
* [st-project-template](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/st-project-template)
|
|
|
|
* [st-testcase-uniform](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/st-testcase-uniform)
|
|
|
|
|
|
|
|
Edit `Makefile` accordingly to you system. Below we provide example of `Makefile` targeted to [Piz Daint](https://www.cscs.ch/computers/piz-daint/) supercomputer.
|
|
|
|
```make
|
|
|
|
# LOAD BEFORE COMPILATION
|
|
|
|
#
|
|
|
|
# module load daint-gpu
|
|
|
|
# module swap PrgEnv-cray PrgEnv-gnu
|
|
|
|
# module load cudatoolkit
|
|
|
|
# module load craype-accel-nvidia60
|
|
|
|
# module load cray-fftw
|
|
|
|
# export LD_LIBRARY_PATH=/project/pr91/share/lib:$LD_LIBRARY_PATH
|
|
|
|
|
|
|
|
# COMPILER
|
|
|
|
CXX=cc
|
|
|
|
|
|
|
|
# DIRECTORY SETTINGS (must end with /)
|
|
|
|
WSLDADIR=/project/pr91/share/wslda/
|
|
|
|
OBJDIR=./obj/
|
|
|
|
BINDIR=./
|
|
|
|
|
|
|
|
# COMPILER FLAGS
|
|
|
|
CFLAGS= -O3 -I/project/pr91/share/include/elpa-2020.05.001
|
|
|
|
LIBS=-lfftw3 -L/project/pr91/share/lib -lelpa
|
|
|
|
|
|
|
|
include mk.st
|
|
|
|
```
|
|
|
|
It is recommended to provide absolute path to `wslda` folder, in this example:
|
|
|
|
```make
|
|
|
|
WSLDADIR=/project/pr91/share/wslda/
|
|
|
|
```
|
|
|
|
In the example we also assume that ELPA lib will be utilized, and paths to header files and lib files are provided in `CFLAGS` and `LIBS`. Various examples of makefiles are located in [Makefiles-templates](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/Makefiles-templates)In order to check correctness of `Makefile` try to compile codes by typing:
|
|
|
|
```bash
|
|
|
|
make
|
|
|
|
```
|
|
|
|
|
|
|
|
*Note 1*: The ELPA Library must be in addition activated in [predefines.h](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/blob/public/st-project-template/predefines.h) file. To learn more about ELPA see [here](Activating ELPA).
|
|
|
|
|
|
|
|
*Note 2*: `st-wslda` codes utilizes LAPACK and ScaLAPACK libraries. Standard method of calling fortran functions from C requires supplementing routine's names by `_` at the end, i.e fortran routine `DGEMM` is called from C as `dgemm_`. However in cases of some systems/compilers (very rare) it is not needed. If this is case of your system then you need add to `CFLAGS` flag
|
|
|
|
`-DFOTRAN_NO_UNDERSCORE`.
|
|
|
|
|
|
|
|
*Recommendation*: Typically (super)computers utilizes `module load` system for configuring programming environment. We recommend to place list of in separate file, for example `env.sh`:
|
|
|
|
```bash
|
|
|
|
# Content of env.sh
|
|
|
|
module load daint-gpu
|
|
|
|
module swap PrgEnv-cray PrgEnv-gnu
|
|
|
|
module load cudatoolkit
|
|
|
|
module load craype-accel-nvidia60
|
|
|
|
module load cray-fftw
|
|
|
|
export LD_LIBRARY_PATH=/project/pr91/share/lib:$LD_LIBRARY_PATH
|
|
|
|
```
|
|
|
|
The compilation process looks like:
|
|
|
|
```bash
|
|
|
|
source env.sh
|
|
|
|
make
|
|
|
|
```
|
|
|
|
|
|
|
|
## Makefile for time dependent codes
|
|
|
|
Operation below apply to folders
|
|
|
|
* [td-project-template](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/td-project-template)
|
|
|
|
* [td-testcase-uniform](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/td-testcase-uniform)
|
|
|
|
|
|
|
|
Edit `Makefile` accordingly to you system. Below we provide example of `Makefile` targeted to [Piz Daint](https://www.cscs.ch/computers/piz-daint/) supercomputer.
|
|
|
|
```make
|
|
|
|
# LOAD BEFORE COMPILATION
|
|
|
|
#
|
|
|
|
# module load daint-gpu
|
|
|
|
# module load cudatoolkit
|
|
|
|
# module load craype-accel-nvidia60
|
|
|
|
|
|
|
|
# COMPILERS
|
|
|
|
CXX=cc
|
|
|
|
NVCC = nvcc
|
|
|
|
|
|
|
|
# DIRECTORY SETTINGS (must end with /)
|
|
|
|
WSLDADIR=/project/pr91/share/wslda/
|
|
|
|
OBJDIR=./obj/
|
|
|
|
BINDIR=./
|
|
|
|
|
|
|
|
# COMPILER FLAGS
|
|
|
|
CFLAGS= -std=gnu99 -O3
|
|
|
|
NVCCFLAGS = -arch sm_60 -O3
|
|
|
|
|
|
|
|
# LIBRARIES
|
|
|
|
LIBS=-lcudart -lcufft -lm
|
|
|
|
|
|
|
|
|
|
|
|
include mk.td
|
|
|
|
```
|
|
|
|
It is recommended to provide absolute path to `wslda` folder, in this example:
|
|
|
|
```make
|
|
|
|
WSLDADIR=/project/pr91/share/wslda/
|
|
|
|
```
|
|
|
|
Various examples of makefiles are located in [Makefiles-templates](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/Makefiles-templates)In order to check correctness of `Makefile` try to compile codes by typing:
|
|
|
|
```bash
|
|
|
|
make
|
|
|
|
```
|
|
|
|
|
|
|
|
*Note*: In some cases additional information about architecture of computing system must be provided in [predefines.h](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/blob/public/td-project-template/predefines.h) file. For more information see [here](Configuring target machine).
|
|
|
|
|
|
|
|
*Recommendation*: Typically (super)computers utilizes `module load` system for configuring programming environment. We recommend to place list of in separate file, for example `env.sh`:
|
|
|
|
```bash
|
|
|
|
# Content of env.sh
|
|
|
|
module load daint-gpu
|
|
|
|
module load cudatoolkit
|
|
|
|
module load craype-accel-nvidia60
|
|
|
|
```
|
|
|
|
The compilation process looks like:
|
|
|
|
```bash
|
|
|
|
source env.sh
|
|
|
|
make
|
|
|
|
```
|
|
|
|
# Compiling auxiliary tools (optional)
|
|
|
|
In folder [tools](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/tools) we provide various useful tools. See [here](Auxiliary tools) for list of presently available tools. In order to compile them, edit `Makefile` and execute `make`. After compilation binary files will be stored in `wslda/tools/bin`.
|
|
|
|
|
|
|
|
# Creating templates of submission scripts (optional)
|
|
|
|
It is recommended to place in template folders submission scripts suitable for your computer. Examples of submission scripts can be find in folder [job-scripts-templates](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/job-scripts-templates).
|
|
|
|
|
|
|
|
# Making W-SLDA Toolkit accessible to other users (optional)
|
|
|
|
To make W-SLDA Toolkit accessible to other users assure that `wslda` is placed in location that is readable for all users. Make sure that permission are set correctly, for example:
|
|
|
|
```bash
|
|
|
|
chmod -R a+r wslda
|
|
|
|
```
|
|
|
|
|
|
|
|
Provide to users path to W-SLDA Toolkit. Recommend for users to place it in `.bashrc` file, example:
|
|
|
|
```bash
|
|
|
|
export WSLDA=/project/pr91/share/wslda/
|
|
|
|
export PATH=/project/pr91/share/wslda/tools/bin:$PATH
|
|
|
|
``` |
