Introduction

W-data was designed to satisfy the following requirements:

• Binary data is stored in a conceptually easy format that allows a variety of tools/languages to be used.
• Format provides storage for data with time stepping (frames/measurements/cycles).
• Data format is suitable for parallel processing (preferably with MPI I/O).
• Data is easy for processing via VisIt.
• It provides an extensible framework - new variables can be created and easily added to the existing dataset.
• Data is convenient for copy between computing systems.
• It allows for easy extraction/copy of selected variables

W-data format concept

Data set will consist of a set of files, for example:

test.wtxt           # metadata file, this one should be indicated when opening in VisIt
test_density_a.wdat # binary file with data
test_delta.wdat     # binary file with data
test_current_a.wdat # binary file with data

Content of test.wtxt may look like:

# Comments with additional  info about the data set
# Comments are ignored when reading by the parser  

nx                       24   # lattice
ny                       28   # lattice
nz                       32   # lattice
dx                        1   # spacing
dy                        1   # spacing
dz                        1   # spacing
datadim                   3   # dimension of block size: 1=nx, 2=nx*ny, 3=nx*ny*nz
prefix                 test   # prefix for files belonging to this data set, binary files have names `prefix_variable-name.format`
cycles                   10   # number of cycles (measurements)
t0                        0   # time value for the first cycle
dt                       -1   # time interval between cycles. If `dt` negative then time step is varying and is read from binary file `prefix__t.wdata`

# variables
# tag                  name                    type                    unit                  format
var               density_a                    real                    none                    wdat
var                   delta                 complex                    none                    wdat
var               current_a                  vector                    none                    wdat

# links
# tag                  name                 link-to
link              density_b               density_a
link              current_b               current_a

# consts
# tag                  name                   value                    unit
const                    eF                     0.5                     MeV
const                    kF                       1                    1/fm

According to our experience, three types of variables (real, complex, vector) are sufficient and cover more than 90% of applications. However, we have implemented a single-precision (float in C notation) types denoted as real4, complex8 and vector4, to be consistent with the notation one can use real8, complex16 or vector8 exchangeable for real, complex, vector.

Binary files store data as row arrays, called datablocks:

The size of datablock depends on variable type and data dimensionality.

• real/real8: blocksize=blocklength*8 Bytes
• real4: blocksize=blocklength*4 Bytes
• complex/complex16: blocksize=blocklength*16 Bytes
• complex8: blocksize=blocklength*8 Bytes
• vector/vector8: blocksize=blocklength*3*8 Bytes
• vector4: blocksize=blocklength*3*4 Bytes

where blocklength is

• for datadim=3: blocklength=nx*ny*nz
• for datadim=2: blocklength=nx*ny
• for datadim=1: blocklength=nx

Note that for vector variables we use following storage pattern:

To compute time associated with given cycle we use the wdata_set_time() function. Which takes as parameters wdata_metadata struct, icycle current number of cycle, and current_time which will be calculated. !Note If dt is positive (to be precise, non negative) then we assume evenly distributed times, according to formula: current_time=t0 + icycle*dt. Hovewer if dt is negative then current_time parameter has to be calculated by the user and parsed to the function in which binary file is created and the value is stored. For more information see Example.

Let's get back to the *wtxt file.

W-data format allows for the representation of the following elements:

• variable:
  Each variable is represented by the binary file of name prefix_varname.format. The variable description has the following format:
  var name type unit format
  Following formats are allowed:
  • wdat: default format for WSLDA codes. Binary files contain row data (no header).
  • dpca: (deprecated) previous format of cold atomic codes. Binary file contains header of size 68B where additional info about file content is stored. For this format wdata lib provides only reading functionality.
  • npy: binary files are numpy arrays. Functionality under construction
    In order to switch WSLDA codes to writing in this format add to input file:
    dataformat npy
• link: It is an alternative name for a given variable. In the case of WSLDA codes in many cases, we do the computation for systems that exhibit some symmetries, like spin symmetry. Then densities for spin-a and spin-b particles are exactly the same. In order to save disk space we can save only one of them:
  var density_a real none wdat
  and for another one set link:
  link density_b density_a
• constant:
  Typically besides variables, we have some constants that are useful during the data analysis process. For example, when making plots it is convenient to express variables in dimensionless form, like delta/eF. To provide user info what are values of selected constants we use const field:
  const eF 0.5

See here for complete list of tags.