This is an old version of this page. You can view the most recent version or browse the history.

Installing the toolkit

Requirements

Static codes (st-wslda)

C compiler (eg. gcc, intel compiler icc, ...)
MPI C compiler (eg. mpicc, intel compiler mpiicc, ...)
FFTW library
LAPACK library
ScaLAPACK library
ELPA library (optionally)
CUDA compiler (optionally, for ELPA)

Note 1: compilers and libraries (FFTW, LAPACK, ScaLAPACK) are installed by default on most of computing clusters. If computing system is equipped with GPUs most likely CUDA Toolkit is also installed. On many systems ELPA is also installed by default. If not you can compile it yourself.

Note 2: LAPACK and ScaLAPACK libraries are part of Intel® Math Kernel Library.

Note 3: It is recommend to use st-wslda with ELPA support if possible. It provides substantial speed-up of computation process.

Time-dependent codes (td-wslda)

C compiler (eg. gcc, intel compiler icc, ...)
MPI C compiler (eg. mpicc, intel compiler mpiicc, ...)
CUDA compiler

Note 1: Time dependent codes require systems accelerated by GPUs. td-wslda codes cannot run on standard systems based on CPUs only.

Downloading the toolkit

It is recommended to download the toolkit directly from repository using git

git clone https://gitlab.fizyka.pw.edu.pl/gabrielw/wslda.git

Optionally you can download it manually from main webpage.
The toolkit is contained in wslda folder, see here for more info about content.

Creating Makefiles

W-SLDA exploits working model based on templates. User starts with template of a project where items like external potential, external pairing potential, external velocity must be provided in C language. Templates are localized in folders:

st-project-template for static problems (st-wslda codes)
td-project-template for time dependent problems (td-wslda codes)

Within this folders Makefile compatible with your computing system must be provided.

Makefile for static codes

Operation below apply to folders

Edit Makefile accordingly to you system. Below we provide example of Makefile targeted to Piz Daint supercomputer.

# LOAD BEFORE COMPILATION
#
# module load daint-gpu
# module swap PrgEnv-cray PrgEnv-gnu
# module load cudatoolkit
# module load craype-accel-nvidia60
# module load cray-fftw
# export LD_LIBRARY_PATH=/project/pr91/share/lib:$LD_LIBRARY_PATH

# COMPILER
CXX=cc

# DIRECTORY SETTINGS (must end with /)
WSLDADIR=/project/pr91/share/wslda/
OBJDIR=./obj/
BINDIR=./

# COMPILER FLAGS
CFLAGS= -O3  -I/project/pr91/share/include/elpa-2020.05.001
LIBS=-lfftw3  -L/project/pr91/share/lib -lelpa

include mk.st

It is recommended to provide absolute path to wslda folder, in this example:

WSLDADIR=/project/pr91/share/wslda/

In the example we also assume that ELPA lib will be utilized, and paths to header files and lib files are provided in CFLAGS and LIBS. Various examples of makefiles are located in Makefiles-templatesIn order to check correctness of Makefile try to compile codes by typing:

make

Note 1: The ELPA Library must be in addition activated in predefines.h file. To learn more about ELPA see here.

Note 2: st-wslda codes utilizes LAPACK and ScaLAPACK libraries. Standard method of calling fortran functions from C requires supplementing routine's names by _ at the end, i.e fortran routine DGEMM is called from C as dgemm_. However in cases of some systems/compilers (very rare) it is not needed. If this is case of your system then you need add to CFLAGS flag -DFOTRAN_NO_UNDERSCORE.

Recommendation: Typically (super)computers utilizes module load system for configuring programming environment. We recommend to place list of in separate file, for example env.sh:

# Content of env.sh
module load daint-gpu
module swap PrgEnv-cray PrgEnv-gnu
module load cudatoolkit
module load craype-accel-nvidia60
module load cray-fftw
export LD_LIBRARY_PATH=/project/pr91/share/lib:$LD_LIBRARY_PATH

The compilation process looks like:

source env.sh
make

Makefile for time dependent codes

Operation below apply to folders

Edit Makefile accordingly to you system. Below we provide example of Makefile targeted to Piz Daint supercomputer.

# LOAD BEFORE COMPILATION
# 
# module load daint-gpu
# module load cudatoolkit
# module load craype-accel-nvidia60

# COMPILERS
CXX=cc
NVCC = nvcc 

# DIRECTORY SETTINGS (must end with /)
WSLDADIR=/project/pr91/share/wslda/
OBJDIR=./obj/
BINDIR=./

# COMPILER FLAGS
CFLAGS= -std=gnu99 -O3
NVCCFLAGS = -arch sm_60 -O3 

# LIBRARIES
LIBS=-lcudart -lcufft -lm


include mk.td

It is recommended to provide absolute path to wslda folder, in this example:

WSLDADIR=/project/pr91/share/wslda/

Various examples of makefiles are located in Makefiles-templates. In order to check correctness of Makefile try to compile codes by typing:

make

Note: In some cases additional information about architecture of computing system must be provided in predefines.h file. For more information see here.

Recommendation: Typically (super)computers utilizes module load system for configuring programming environment. We recommend to place list of in separate file, for example env.sh:

# Content of env.sh
module load daint-gpu
module load cudatoolkit
module load craype-accel-nvidia60

The compilation process looks like:

source env.sh
make

Compiling W-DATA library and visit plugin (optional)

It is recommended to generate a library for w-data managing format:

cd lib-wdata/
make libs

Visit plugin is located in lib-wdata/visit-plugin. Follow instruction from README.txt file to install it.

Compiling auxiliary tools (optional)

In folder tools we provide various useful tools. See here for list of presently available tools. In order to compile them, edit Makefile and execute make. After compilation binary files will be stored in wslda/tools/bin.

Creating templates of submission scripts (optional)

It is recommended to place in template folders submission scripts suitable for your computer. Examples of submission scripts can be find in folder job-scripts-templates.

Making W-SLDA Toolkit accessible to other users (optional)

To make W-SLDA Toolkit accessible to other users assure that wslda is placed in location that is readable for all users. Make sure that permission are set correctly, for example:

chmod -R a+r wslda

Provide to users path to W-SLDA Toolkit. Recommend for users to place it in .bashrc file, example:

export WSLDA=/project/pr91/share/wslda/
export PATH=/project/pr91/share/wslda/tools/bin:$PATH