Skip to content

GitLab

Installing the toolkit · Changes

Page history
Update Installing the toolkit authored by Gabriel Wlazłowski's avatar Gabriel Wlazłowski
Hide whitespace changes
Inline Side-by-side
Installing-the-toolkit.md
View page @ 2dc5262b
...@@ -8,40 +8,40 @@ ...@@ -8,40 +8,40 @@
* [ELPA library](https://elpa.mpcdf.mpg.de/) (optionally) * [ELPA library](https://elpa.mpcdf.mpg.de/) (optionally)
* [CUDA compiler](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html) (optionally, for ELPA) * [CUDA compiler](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html) (optionally, for ELPA)
*Note 1*: compilers and libraries (FFTW, LAPACK, ScaLAPACK) are installed by default on most of computing clusters. If computing system is equipped with GPUs most likely CUDA Toolkit is also installed. On many systems ELPA is also installed by default. If not you can compile it yourself. *Note 1*: compilers and libraries (FFTW, LAPACK, ScaLAPACK) are installed by default on most of the computing clusters. If the computing system is equipped with GPUs most likely CUDA Toolkit is also installed. On many systems, ELPA is also installed by default. If not you can compile it yourself.
*Note 2*: LAPACK and ScaLAPACK libraries are part of Intel® Math Kernel Library. *Note 2*: LAPACK and ScaLAPACK libraries are part of Intel® Math Kernel Library.
*Note 3*: It is recommend to use `st-wslda` with ELPA support if possible. It provides substantial speed-up of computation process. *Note 3*: It is recommended to use `st-wslda` with ELPA support if possible. It provides substantial speed-up of the computation process.
## Time-dependent codes (td-wslda) ## Time-dependent codes (td-wslda)
* C compiler (eg. `gcc`, intel compiler `icc`, ...) * C compiler (eg. `gcc`, intel compiler `icc`, ...)
* MPI C compiler (eg. `mpicc`, intel compiler `mpiicc`, ...) * MPI C compiler (eg. `mpicc`, intel compiler `mpiicc`, ...)
* [CUDA compiler](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html) * [CUDA compiler](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html)
*Note 1*: Time dependent codes **require** systems accelerated by GPUs. `td-wslda` codes cannot run on standard systems based on CPUs only. *Note 1*: Time-dependent codes **require** systems accelerated by GPUs. `td-wslda` codes cannot run on standard systems based on CPUs only.
# Downloading the toolkit # Downloading the toolkit
It is recommended to download the toolkit directly from repository using git It is recommended to download the toolkit directly from the repository using git
```bash ```bash
git clone https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda.git git clone https://gitlab.fizyka.pw.edu.pl/wtools/wslda.git
``` ```
Optionally you can download it manually from [main webpage](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda). Optionally you can download it manually from [main webpage](https://gitlab.fizyka.pw.edu.pl/wtools/wslda).
The toolkit is contained in `wslda` folder, see [here](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/wikis/Folders-structure) for more info about content. The toolkit is contained in `wslda` folder, see [here](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/wikis/Folders-structure) for more info about content.
# Creating Makefiles # Creating Makefiles
W-SLDA exploits working model based on templates. W-SLDA exploits a working model based on templates.
User starts with template of a project where items like external potential, external pairing potential, external velocity must be provided in C language. Templates are localized in folders: User starts with the template of a project where items like external potential, external pairing potential, external velocity must be provided in C language. Templates are localized in folders:
* [st-project-template](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/st-project-template) for static problems (`st-wslda` codes) * [st-project-template](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/st-project-template) for static problems (`st-wslda` codes)
* [td-project-template](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/td-project-template) for time dependent problems (`td-wslda` codes) * [td-project-template](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/td-project-template) for time dependent problems (`td-wslda` codes)
Within this folders `Makefile` compatible with your computing system must be provided. Within this folders `Makefile` compatible with your computing system must be provided.
## Makefile for static codes ## Makefile for static codes
Operation below apply to folders Operation below apply to folders
* [st-project-template](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/st-project-template) * [st-project-template](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/st-project-template)
* [st-testcase-uniform](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/st-testcase-uniform) * [st-testcase-uniform](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/st-testcase-uniform)
Edit `Makefile` accordingly to you system. Below we provide example of `Makefile` targeted to [Piz Daint](https://www.cscs.ch/computers/piz-daint/) supercomputer. Edit `Makefile` accordingly to your system. Below we provide an example of `Makefile` targeted to [Piz Daint](https://www.cscs.ch/computers/piz-daint/) supercomputer.
```make ```make
# LOAD BEFORE COMPILATION # LOAD BEFORE COMPILATION
# #
...@@ -64,23 +64,24 @@ BINDIR=./ ...@@ -64,23 +64,24 @@ BINDIR=./
CFLAGS= -O3 -I/project/pr91/share/include/elpa-2020.05.001 CFLAGS= -O3 -I/project/pr91/share/include/elpa-2020.05.001
LIBS=-lfftw3 -L/project/pr91/share/lib -lelpa LIBS=-lfftw3 -L/project/pr91/share/lib -lelpa
include mk.st # ----- DO NOT MODIFY -----
include $(WSLDADIR)hpc-engine/mk.st
``` ```
It is recommended to provide absolute path to `wslda` folder, in this example: It is recommended to provide absolute path to `wslda` folder, in this example:
```make ```make
WSLDADIR=/project/pr91/share/wslda/ WSLDADIR=/project/pr91/share/wslda/
``` ```
In the example we also assume that ELPA lib will be utilized, and paths to header files and lib files are provided in `CFLAGS` and `LIBS`. Various examples of makefiles are located in [Makefiles-templates](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/Makefiles-templates)In order to check correctness of `Makefile` try to compile codes by typing: In the example, we also assume that ELPA lib will be utilized, and paths to header files and lib files are provided in `CFLAGS` and `LIBS`. Various examples of makefiles are located in [Makefiles-templates](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/Makefiles-templates). In order to check the correctness of `Makefile` try to compile codes by typing:
```bash ```bash
make make
``` ```
*Note 1*: The ELPA Library must be in addition activated in [predefines.h](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/blob/public/st-project-template/predefines.h) file. To learn more about ELPA see [here](Setting up diagonalization engine). *Note 1*: The ELPA Library must be in addition activated in [predefines.h](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/blob/public/st-project-template/predefines.h) file. To learn more about ELPA see [here](Setting up diagonalization engine).
*Note 2*: `st-wslda` codes utilizes LAPACK and ScaLAPACK libraries. Standard method of calling fortran functions from C requires supplementing routine's names by `_` at the end, i.e fortran routine `DGEMM` is called from C as `dgemm_`. However in cases of some systems/compilers (very rare) it is not needed. If this is case of your system then you need add to `CFLAGS` flag *Note 2*: `st-wslda` codes utilizes LAPACK and ScaLAPACK libraries. The standard method of calling fortran functions from C requires supplementing routine's names by `_` at the end, i.e fortran routine `DGEMM` is called from C as `dgemm_`. However in cases of some systems/compilers (very rare) it is not needed. If this is the case of your system then you need to add to `CFLAGS` flag
`-DFOTRAN_NO_UNDERSCORE`. `-DFOTRAN_NO_UNDERSCORE`.
*Recommendation*: Typically (super)computers utilizes `module load` system for configuring programming environment. We recommend to place list of in separate file, for example `env.sh`: *Recommendation*: Typically (super)computers utilize `module load` system for configuring the programming environment. We recommend placing a list of in a separate file, for example `env.sh`:
```bash ```bash
# Content of env.sh # Content of env.sh
module load daint-gpu module load daint-gpu
...@@ -90,18 +91,18 @@ module load craype-accel-nvidia60 ...@@ -90,18 +91,18 @@ module load craype-accel-nvidia60
module load cray-fftw module load cray-fftw
export LD_LIBRARY_PATH=/project/pr91/share/lib:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/project/pr91/share/lib:$LD_LIBRARY_PATH
``` ```
The compilation process looks like: The compilation process looks like this:
```bash ```bash
source env.sh source ./env.sh
make make
``` ```
## Makefile for time dependent codes ## Makefile for time dependent codes
Operation below apply to folders Operation below apply to folders
* [td-project-template](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/td-project-template) * [td-project-template](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/td-project-template)
* [td-testcase-uniform](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/td-testcase-uniform) * [td-testcase-uniform](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/td-testcase-uniform)
Edit `Makefile` accordingly to you system. Below we provide example of `Makefile` targeted to [Piz Daint](https://www.cscs.ch/computers/piz-daint/) supercomputer. Edit `Makefile` accordingly to your system. Below we provide an example of `Makefile` targeted to [Piz Daint](https://www.cscs.ch/computers/piz-daint/) supercomputer.
```make ```make
# LOAD BEFORE COMPILATION # LOAD BEFORE COMPILATION
# #
...@@ -126,20 +127,21 @@ NVCCFLAGS = -arch sm_60 -O3 ...@@ -126,20 +127,21 @@ NVCCFLAGS = -arch sm_60 -O3
LIBS=-lcudart -lcufft -lm LIBS=-lcudart -lcufft -lm
include mk.td # ----- DO NOT MODIFY -----
include $(WSLDADIR)hpc-engine/mk.td
``` ```
It is recommended to provide absolute path to `wslda` folder, in this example: It is recommended to provide an absolute path to `wslda` folder, in this example:
```make ```make
WSLDADIR=/project/pr91/share/wslda/ WSLDADIR=/project/pr91/share/wslda/
``` ```
Various examples of makefiles are located in [Makefiles-templates](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/Makefiles-templates). In order to check correctness of `Makefile` try to compile codes by typing: Various examples of makefiles are located in [Makefiles-templates](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/Makefiles-templates). In order to check the correctness of `Makefile` try to compile codes by typing:
```bash ```bash
make make
``` ```
*Note*: In some cases additional information about architecture of computing system must be provided in [predefines.h](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/blob/public/td-project-template/predefines.h) file. For more information see [here](Configuring GPU machine). *Note*: In some cases, additional information about the architecture of the computing system must be provided in [predefines.h](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/blob/public/td-project-template/predefines.h) file. For more information see [here](Configuring GPU machine).
*Recommendation*: Typically (super)computers utilizes `module load` system for configuring programming environment. We recommend to place list of in separate file, for example `env.sh`: *Recommendation*: Typically (super)computers utilize `module load` system for configuring the programming environment. We recommend placing list of in a separate file, for example `env.sh`:
```bash ```bash
# Content of env.sh # Content of env.sh
module load daint-gpu module load daint-gpu
...@@ -148,27 +150,27 @@ module load craype-accel-nvidia60 ...@@ -148,27 +150,27 @@ module load craype-accel-nvidia60
``` ```
The compilation process looks like: The compilation process looks like:
```bash ```bash
source env.sh source ./env.sh
make make
``` ```
# Compiling W-DATA library and visit plugin (optional) # Compiling W-DATA library and visit plugin (optional)
It is recommended to generate a library for w-data managing format: It is recommended to generate a library for w-data managing format:
```bash ```bash
cd lib-wdata/ cd lib/wdata/
make libs make libs
``` ```
Visit plugin is located in [lib-wdata/visit-plugin](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/lib-wdata/visit-plugin). Visit plugin is located in [lib-wdata/visit-plugin](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/lib-wdata/visit-plugin).
Follow instruction from [README.txt](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/lib-wdata/visit-plugin/README.txt) file to install it. Follow instruction from [README.txt](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/lib-wdata/visit-plugin/README.txt) file to install it.
# Compiling auxiliary tools (optional) # Compiling auxiliary tools (optional)
In folder [tools](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/tools) we provide various useful tools. See [here](Auxiliary tools) for list of presently available tools. In order to compile them, edit `Makefile` and execute `make`. After compilation binary files will be stored in `wslda/tools/bin`. In folder [tools](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/tools) we provide various useful tools. See [here](Auxiliary tools) for list of presently available tools. In order to compile them, edit `Makefile` and execute `make`. After compilation binary files will be stored in `wslda/tools/bin`.
# Creating templates of submission scripts (optional) # Creating templates of submission scripts (optional)
It is recommended to place in template folders submission scripts suitable for your computer. Examples of submission scripts can be find in folder [job-scripts-templates](https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda/-/tree/public/job-scripts-templates). It is recommended to place in template folders submission scripts suitable for your computer. Examples of submission scripts can be found in the folder [job-scripts-templates](https://gitlab.fizyka.pw.edu.pl/wtools/wslda/-/tree/public/job-scripts-templates).
# Making W-SLDA Toolkit accessible to other users (optional) # Making W-SLDA Toolkit accessible to other users (optional)
To make W-SLDA Toolkit accessible to other users assure that `wslda` is placed in location that is readable for all users. Make sure that permission are set correctly, for example: To make W-SLDA Toolkit accessible to other users assure that `wslda` is placed in a location that is readable for all users. Make sure that permissions are set correctly, for example:
```bash ```bash
chmod -R a+r wslda chmod -R a+r wslda
``` ```
......