VAPoR  3.0.0
Public Types | Public Member Functions | Protected Attributes | Friends | List of all members
VAPoR::VDC Class Referenceabstract

Defines API for reading, writing, and appending data to a VAPOR Data Collection (Version 3) More...

#include <VDC.h>

Inheritance diagram for VAPoR::VDC:
VAPoR::DC VetsUtil::MyBase VAPoR::VDCNetCDF

Public Types

enum  AccessMode { R, W, A }
 
- Public Types inherited from VAPoR::DC
enum  XType {
  INVALID = -1, FLOAT, DOUBLE, INT32,
  INT64, TEXT
}
 External storage types for primitive data. More...
 
- Public Types inherited from VetsUtil::MyBase
typedef void(* ErrMsgCB_T) (const char *msg, int err_code)
 
typedef void(* DiagMsgCB_T) (const char *msg)
 

Public Member Functions

 VDC ()
 
virtual ~VDC ()
 
virtual int Initialize (const std::vector< string > &paths, AccessMode mode)
 
virtual int Initialize (const std::vector< string > &paths)
 
int SetCompressionBlock (std::vector< size_t > bs, string wname, std::vector< size_t > cratios)
 
void GetCompressionBlock (std::vector< size_t > &bs, string &wname, std::vector< size_t > &cratios) const
 
void SetPeriodicBoundary (std::vector< bool > periodic)
 
std::vector< bool > GetPeriodicBoundary () const
 
int DefineDimension (string dimname, size_t length, int axis)
 
bool GetDimension (string dimname, DC::Dimension &dimension) const
 
std::vector< string > GetDimensionNames () const
 
int DefineCoordVar (string varname, std::vector< string > dimnames, string units, int axis, XType type, bool compressed)
 
int DefineCoordVarUniform (string varname, std::vector< string > dimname, string units, int axis, XType type, bool compressed)
 
bool GetCoordVarInfo (string varname, std::vector< string > &dimnames, string &units, int &axis, XType &type, bool &compressed, bool &uniform) const
 
bool GetCoordVarInfo (string varname, DC::CoordVar &cvar) const
 
int DefineDataVar (string varname, std::vector< string > dimnames, std::vector< string > coordvars, string units, XType type, bool compressed)
 
int DefineDataVar (string varname, std::vector< string > dimnames, std::vector< string > coordvars, string units, XType type, double missing_value, string maskvar)
 
bool GetDataVarInfo (string varname, std::vector< string > &dimnames, std::vector< string > &coordvars, string &units, XType &type, bool &compressed, string &maskvar) const
 
bool GetDataVarInfo (string varname, DC::DataVar &datavar) const
 
bool GetBaseVarInfo (string varname, DC::BaseVar &var) const
 
virtual std::vector< string > GetDataVarNames () const
 
virtual std::vector< string > GetCoordVarNames () const
 
int GetNumRefLevels (string varname) const
 
int PutAtt (string varname, string attname, XType type, const vector< double > &values)
 
int PutAtt (string varname, string attname, XType type, const vector< long > &values)
 
int PutAtt (string varname, string attname, XType type, const string &values)
 
int GetAtt (string varname, string attname, vector< double > &values) const
 
int GetAtt (string varname, string attname, vector< long > &values) const
 
int GetAtt (string varname, string attname, string &values) const
 
std::vector< string > GetAttNames (string varname) const
 
XType GetAttType (string varname, string attname) const
 
virtual int GetMapProjection (string lonname, string latname, string &projstring) const
 
virtual int SetMapProjection (string lonname, string latname, string projstring)
 
int EndDefine ()
 
virtual int GetPath (string varname, size_t ts, string &path, size_t &file_ts, size_t &max_ts) const =0
 
virtual int GetDimLensAtLevel (string varname, int level, std::vector< size_t > &dims_at_level, std::vector< size_t > &bs_at_level) const =0
 
virtual int OpenVariableRead (size_t ts, string varname, int level=0, int lod=0)=0
 
virtual int OpenVariableWrite (size_t ts, string varname, int lod=-1)=0
 
virtual int CloseVariable ()=0
 
virtual int Write (const float *data)=0
 
virtual int WriteSlice (const float *slice)=0
 
virtual int Read (float *data)=0
 
virtual int ReadSlice (float *slice)=0
 
virtual int ReadRegion (const vector< size_t > &min, const vector< size_t > &max, float *region)=0
 
virtual int ReadRegionBlock (const vector< size_t > &min, const vector< size_t > &max, float *region)=0
 
virtual int PutVar (string varname, int lod, const float *data)=0
 
virtual int PutVar (size_t ts, string varname, int lod, const float *data)=0
 
virtual int GetVar (string varname, int level, int lod, float *data)=0
 
virtual int GetVar (size_t ts, string varname, int level, int lod, float *data)=0
 
virtual bool CompressionInfo (vector< size_t > bs, string wname, size_t &nlevels, size_t &maxcratio) const =0
 
virtual bool VariableExists (size_t ts, string varname, int reflevel=0, int lod=0) const =0
 
- Public Member Functions inherited from VAPoR::DC
 DC ()
 
virtual ~DC ()
 
virtual int Initialize (const vector< string > &paths)=0
 
virtual bool GetDimension (string dimname, size_t &length, int &axis) const
 
virtual std::vector< string > GetDataVarNames (int ndim, bool spatial) const
 
virtual std::vector< string > GetCoordVarNames (int ndim, bool spatial) const
 
virtual bool IsTimeVarying (string varname) const
 
virtual bool IsCompressed (string varname) const
 
virtual int GetNumTimeSteps (string varname) const
 
virtual int GetCRatios (string varname, vector< size_t > &cratios) const
 
virtual bool IsDataVar (string varname) const
 
virtual bool IsCoordVar (string varname) const
 
virtual int GetMapProjection (string varname, string &projstring) const
 
- Public Member Functions inherited from VetsUtil::MyBase
 MyBase ()
 
const string & getClassName () const
 

Protected Attributes

string _master_path
 
AccessMode _mode
 
bool _defineMode
 
std::vector< size_t > _bs
 
string _wname
 
std::vector< size_t > _cratios
 
vector< bool > _periodic
 
VAPoR::UDUnits _udunits
 
std::map< string, Dimension_dimsMap
 
std::map< string, Attribute_atts
 
std::map< string, CoordVar_coordVars
 
std::map< string, DataVar_dataVars
 

Friends

std::ostream & operator<< (std::ostream &o, const VDC &vdc)
 

Additional Inherited Members

- Static Public Member Functions inherited from VAPoR::DC
static bool ParseDimensions (const vector< DC::Dimension > &dimensions, vector< size_t > &sdims, size_t &numts)
 
- Static Public Member Functions inherited from VetsUtil::MyBase
static void SetErrMsg (const char *format,...)
 Record a formatted error message. More...
 
static void SetErrMsg (int errcode, const char *format,...)
 Record a formatted error message and an error code. More...
 
static const char * GetErrMsg ()
 
static void SetErrCode (int err_code)
 Record an error code. More...
 
static int GetErrCode ()
 Retrieve the current error code. More...
 
static void SetErrMsgCB (ErrMsgCB_T cb)
 
static ErrMsgCB_T GetErrMsgCB ()
 
static void SetErrMsgFilePtr (FILE *fp)
 
static const FILE * SetErrMsgFilePtr ()
 
static void SetDiagMsg (const char *format,...)
 Record a formatted diagnostic message. More...
 
static const char * GetDiagMsg ()
 
static void SetDiagMsgCB (DiagMsgCB_T cb)
 
static DiagMsgCB_T GetDiagMsgCB ()
 
static void SetDiagMsgFilePtr (FILE *fp)
 
static bool EnableErrMsg (bool enable)
 
- Static Public Attributes inherited from VetsUtil::MyBase
static char * ErrMsg
 
static int ErrCode
 
static int ErrMsgSize
 
static FILE * ErrMsgFilePtr
 
static ErrMsgCB_T ErrMsgCB
 
static char * DiagMsg
 
static int DiagMsgSize
 
static FILE * DiagMsgFilePtr
 
static DiagMsgCB_T DiagMsgCB
 
static bool Enabled
 
- Protected Member Functions inherited from VetsUtil::MyBase
void SetClassName (const string &name)
 

Detailed Description

Defines API for reading, writing, and appending data to a VAPOR Data Collection (Version 3)

Author
John Clyne
Date
July, 2014

This abstract class efines API for reading, writing, and appending data to a VAPOR Data Collection (Version 3). The VDC class is an abstract virtual class, providing a public API, but performing no actual storage operations. Derived implementations of the VDC base class are required to support the API.

In version 3 of the VDC the metadata (.vdf) file found in VDC version 1 and 2 is replaced with a "master" file that describes the contents of the entire VDC. The master file imposes structure on the organization of the files containing data, determining, for example, which data files contain which variables and time steps.

Unlike the .vdf file, it is intended that the master file will be stored in the same scientific data file format as the data themselves (though this depends on the implementation of the derived class). Another important change in version 3 is that both the master file and the accompanying data files are intended to be accessible using the native file format API. I.e. users may operate on files in the VDC using, for example, the NetCDF API, or they may use the API provided by the VDC class object. The latter is only required when reading or writing compressed variables (not all variables in a VDC version 3 must be compressed). Thus if NetCDF is chosen as the underlying format the NetCDF API may be used directly to read and write NetCDF "attributes" and variables (provided the variables are not compressed).

Variables in a VDC may have 1, 2, or 3 spatial dimensions, and 0 or 1 temporal dimensions.

The VDC is structured in the spirit of the "NetCDF Climate and Forecast (CF) Metadata Conventions", version 1.6, 5, December 2011. It supports only a subset of the CF functionality (e.g. there is no support for "Discrete Sampling Geometries"). Moreover, it is more restrictive than the CF in a number of areas. Particular items of note include:

This class inherits from VetsUtil::MyBase. Unless otherwise documented any method that returns an integer value is returning status. A negative value indicates failure. Error messages are logged via VetsUtil::MyBase::SetErrMsg(). Methods that return a boolean do not, unless otherwise documented, log an error message upon failure (return of false).

Parameters
level

Grid refinement level for multiresolution variables. Compressed variables in the VDC have a multi-resolution representation: the sampling grid for multi-resolution variables is hierarchical, and the dimension lengths of adjacent levels in the hierarchy differ by a factor of two. The level parameter is used to select a particular depth of the hierarchy.

To provide maximum flexibility as well as compatibility with previous versions of the VDC the interpretation of level is somewhat complex. Both positive and negative values may be used to specify the refinement level and have different interpretations.

For positive values of level, a value of 0 indicates the coarsest member of the grid hierarchy. A value of 1 indicates the next grid refinement after the coarsest, and so on. Using postive values the finest level in the hierarchy is given by GetNumRefLevels() - 1. Values of level that are greater than GetNumRefLevels() - 1 are treated as if they were equal to GetNumRefLevels() - 1.

For negative values of level a value of -1 indicates the variable's native grid resolution (the finest resolution available). A value of -2 indicates the next coarsest member in the hierarchy after the finest, and so on. Using negative values the coarsest available level in the hierarchy is given by negating the value returned by GetNumRefLevels(). Values of level that are less than the negation of GetNumRefLevels() are treated as if they were equal to the negation of the GetNumRefLevels() return value.

lodThe level-of-detail parameter, lod, selects the approximation level for a compressed variable. The lod parameter is similar to the level parameter in that it provides control over accuracy of a compressed variable. However, instead of selecting the grid resolution the lod parameter controls the compression factor by indexing into the cratios vector (see below). As with the level parameter, both positive and negative values may be used to index into cratios and different interpretations.

For positive values of lod, a value of 0 indicates the the first element of cratios, a value of 1 indicates the second element, and so on up to the size of the cratios vector (See DC::GetCRatios()).

For negative values of lod a value of -1 indexes the last element of cratios, a value of -2 indexes the second to last element, and so on. Using negative values the first element of cratios - the greatest compression rate - is indexed by negating the size of the cratios vector.

Parameters
cratiosA monotonically decreasing vector of compression ratios. Compressed variables in the VDC are stored with a fixed, finite number of compression factors. The cratios vector is used to specify the available compression factors (ratios). A compression factor of 1 indicates no compression (1:1). A value of 2 indciates two to one compression (2:1), and so on. The minimum valid value of cratios is 1. The maximum value is determined by a number of factors and can be obtained using the CompressionInfo() method.
bsAn ordered list of block dimensions that specifies the block decomposition of the variable. The rank of bs may be less than that of a variable's array dimensions, in which case only the n fastest varying variable dimensions will be blocked, where n is the rank of bs. The ordering of the dimensions in bs is from fastest to slowest. A block is the basic unit of compression in the VDC: variables are decomposed into blocks, and individual blocks are compressed independently.
wnameName of wavelet used for transforming compressed variables between wavelet and physical space. Valid values are "bior1.1", "bior1.3", "bior1.5", "bior2.2", "bior2.4", "bior2.6", "bior2.8", "bior3.1", "bior3.3", "bior3.5", "bior3.7", "bior3.9", "bior4.4"

Definition at line 177 of file VDC.h.

Member Enumeration Documentation

Read, Write, Append access mode

Enumerator

Definition at line 182 of file VDC.h.

Constructor & Destructor Documentation

VAPoR::VDC::VDC ( )

Class constuctor

virtual VAPoR::VDC::~VDC ( )
inlinevirtual

Definition at line 188 of file VDC.h.

Member Function Documentation

virtual int VAPoR::VDC::CloseVariable ( )
pure virtual

Close the currently opened variable

Close the handle for variable opened with OpenVariableWrite(), or OpenVariableRead()

See also
OpenVariableWrite(), OpenVariableRead()

Implements VAPoR::DC.

Implemented in VAPoR::VDCNetCDF.

virtual bool VAPoR::VDC::CompressionInfo ( vector< size_t >  bs,
string  wname,
size_t &  nlevels,
size_t &  maxcratio 
) const
pure virtual

This method computes and returns the depth (number of levels) in a a multi-resolution hierarch for a given wavelet, wname, and decomposition block, bs. It also computes the maximum compression ratio, cratio, possible for the the specified combination of block size, bs, and wavelet, wname. The maximum compression ratio is cratio:1.

Parameters
[in]bsDimensions of native decomposition block. The rank of bs may be less than or equal to the rank of dims.
[in]wnamewavelet name. Empty string if no compression is to be performed.
[out]nlevelsNumber of levels in hierarchy
[out]maxcratioMaximum compression ratio
Return values
boolIf bs, wname, or the combination there of is invalid false is returned and the values of nlevels and maxcratio are undefined. Upon success true is returned.
int VAPoR::VDC::DefineCoordVar ( string  varname,
std::vector< string >  dimnames,
string  units,
int  axis,
XType  type,
bool  compressed 
)

Define a coordinate variable

This method provides the definition for a coordinate variable: a variable providing spatial or temporal coordinates for a subsequently defined data variable.

If the variable's name, varname, matches a dimension defined with DefineDimension(), only the units and external data type may differ from the 1D coordinate variable impliclity defined by DefineDimension()

Parameters
[in]varnameThe name of the coordinate variable.
[in]dimnamesAn ordered vector specifying the variables dimension names. The dimension names must have previously been defined with the DefineDimension() method.
[in]unitsThis parameter specifies a string describing the units of measure for the variable. The string is compatible with the Unidata udunits2 conversion package. If the quantity is unitless an empty string may be specified.
[in]axisAn integer indicating the spatial or temporal coordinate axis. Acceptable values are 0 (for X or longitude), 1 (for Y or latitude), 2 (for Z or vertical), and 3 (for time).
[in]typeThe primitive data type storage format. Currently supported values are FLOAT. This is the type that will be used to store the variable on disk
[in]compressedA boolean indicating whether the coordinate variable is to be wavelet transformed.

It is an error to call this method if the VDC master is not currently in define mode.

Note
Temporal coordinate variables (axis=3) must have exactly one dimension.
When in append (A) mode it is an error to redefine an existing variable.
Return values
statusA negative int is returned on error
See also
DefineDimension(), DefineCoordVar(), SetCompressionBlock()
int VAPoR::VDC::DefineCoordVarUniform ( string  varname,
std::vector< string >  dimname,
string  units,
int  axis,
XType  type,
bool  compressed 
)

Define a coordinate variable with uniform sampling

This method provides the definition for a uniform coordinate variable. A uniformly sampled coordinate variable is a variable for which the coordinates along the fastest varying axis may be given by i * dx , where i is an index starting from zero, and dx is a real number representing the spacing between points.

One-dimensional coordinate variables that have uniform sampling should be declared as such using this method rather than the more general DefineCoordVar().

Parameters
[in]varnameThe name of the coordinate variable.
[in]dimnamesAn ordered vector specifying the variables dimension names. The dimension names must have previously been defined with the DefineDimension() method.
[in]unitsThis parameter specifies a string describing the units of measure for the variable. The string is compatible with the udunits2 conversion package. If the quantity is unitless an empty string may be specified.
[in]axisAn integer indicating the spatial or temporal coordinate axis. Acceptable values are 0 (for X or longitude), 1 (for Y or latitude), 2 (for Z or vertical), and 3 (for time).
[in]typeThe primitive data type storage format. Currently supported values are FLOAT
[in]compressedA boolean indicating whether the coordinate variable is to be wavelet transformed.

It is an error to call this method if the VDC master is not currently in define mode.

Note
When in append (A) mode it is an error to redefine an existing variable.
Return values
statusA negative int is returned on error
See also
DefineDimension(), DefineCoordVar(), SetCompressionBlock()
int VAPoR::VDC::DefineDataVar ( string  varname,
std::vector< string >  dimnames,
std::vector< string >  coordvars,
string  units,
XType  type,
bool  compressed 
)

Define a data variable

This method defines a data variable in the VDC master file

Parameters
[in]varnameThe name of the data variable.
[in]dimnamesAn ordered vector specifying the variables dimension names. The dimension names must have previously be defined with the DefineDimension() method.
[in]coordvarsAn ordered vector specifying the coordinate variable names providing the coordinates for this variable. The coordinate variables must have previously been defined with the DefineCoordVar() method. Moreover, the dimension names of each coordinate variable must be a subset of those in dimnames.
[in]unitsThis parameter specifies a string describing the units of measure for the variable. The string is compatible with the udunits2 conversion package. If the quantity is unitless an empty string may be specified.
[in]typeThe primitive data type storage format. Currently supported values are FLOAT
[in]compressedA boolean indicating whether the coordinate variable is to be wavelet transformed.

It is an error to call this method if the VDC master is not currently in define mode.

Note
When in append (A) mode it is an error to redefine an existing variable.
See also
DefineDimension(), DefineCoordVar(), SetCompressionBlock()
int VAPoR::VDC::DefineDataVar ( string  varname,
std::vector< string >  dimnames,
std::vector< string >  coordvars,
string  units,
XType  type,
double  missing_value,
string  maskvar 
)

Define a compressed data variable with missing data

Define a data variable

This method defines a data variable in the VDC master file

Parameters
[in]varnameThe name of the data variable.
[in]dimnamesAn ordered vector specifying the variables dimension names. The dimension names must have previously be defined with the DefineDimension() method.
[in]coordvarsAn ordered vector specifying the coordinate variable names providing the coordinates for this variable. The coordinate variables must have previously been defined with the DefineCoordVar() method. Moreover, the dimension names of each coordinate variable must be a subset of those in dimnames.
[in]unitsThis parameter specifies a string describing the units of measure for the variable. The string is compatible with the udunits2 conversion package. If the quantity is unitless an empty string may be specified.
[in]typeThe primitive data type storage format. Currently supported values are FLOAT
[in]compressedA boolean indicating whether the coordinate variable is to be wavelet transformed.

It is an error to call this method if the VDC master is not currently in define mode.

Note
When in append (A) mode it is an error to redefine an existing variable.
See also
DefineDimension(), DefineCoordVar(), SetCompressionBlock()
Parameters
[in]missing_valueSpecifies a value that should be used for masked grid locations after a variable is reconstructed.
[in]maskvarSpecifies the name of a variable whose contents indicate the presense or absense of invalid entries in the data variable. The contents of the mask array are treated as booleans, true values indicating valid data. The rank of of the variable may be less than or equal to that of varname. The dimensions of maskvar must match the fastest varying dimensions of varname. The maskvar variable must have been previously defined with DefineDataVar().
See also
DefineDimension(), DefineCoordVar(), SetCompressionBlock()
int VAPoR::VDC::DefineDimension ( string  dimname,
size_t  length,
int  axis 
)

Define a dimension in the VDC

This method specifies the name, length, and axis of a dimension. A variable in the VDC may have one to four dimensions (one to three spatial, and zero or one temporal). Dimensions may be of any length greater than or equal to one.

This method also defines a 1D VDC coordinate variable with the same name as the dimension, dimname. The implicitly defined coordinate variable will be defined to be unitless, have uniform sampling, an external data type of FLOAT, and not compressed. The units, external data type, and uniformity of this coordinate variable can subsequently be redefined with DefineCoordVar() or DefineCoordVarUniform(). Thus for each dimension a 1D coordinate variable of the same name must exist.

This method must be called prior to defining any variables requring the defined dimensions.

There are no default dimensions defined.

It is an error to call this method if the VDC master is not currently in define mode.

Parameters
[in]dimnameA string specifying the name of the dimension.
[in]lengthThe dimension length, which must be greater than zero.
[in]axisThe axis associated with the dimension. Acceptable values are 0 (for X or longitude), 1 (for Y or latitude), 2 (for Z or vertical), and 3 (for time).
Note
When the VDC master file is initialized in append (mode = A) mode it is an error to redefine an existing dimension. New dimensions may, however, be defined.
Return values
statusA negative int is returned on error
See also
DefineCoordVar(), DefineDataVar(), GetDimension()
int VAPoR::VDC::EndDefine ( )

When the open mode mode is A or W this method signals the class object that metadata defintions have been completed and it commits them to the master VDC file. This method also prepares the VDC for the reading or writing of variable or coordinate data.

Ignored if mode is R.

Return values
statusA negative it is returned if the master file is not successfully written for any reason
Note
The master file should not be accessed with the native file format API (e.g. NetCDF) if the VDC is in define mode (e.g. until after EndDefine() is called).
See also
VDC()
int VAPoR::VDC::GetAtt ( string  varname,
string  attname,
vector< double > &  values 
) const
virtual

Read an attribute

This method reads an attribute from the VDC. The attribute can either be "global", if varname is the empty string, or bound to a variable if varname indentifies a variable in the VDC.

Parameters
[in]varnameThe name of the variable the attribute is bound to, or the empty string if the attribute is global
[in]attnameThe attributes name
[out]typeThe primitive data type storage format. This is the type that will be used to store the attribute on disk
[out]valuesA vector to contain the returned floating point attribute values
Return values
statusA negative int is returned on failure
See also
PutAtt()

Implements VAPoR::DC.

int VAPoR::VDC::GetAtt ( string  varname,
string  attname,
vector< long > &  values 
) const
virtual

Implements VAPoR::DC.

int VAPoR::VDC::GetAtt ( string  varname,
string  attname,
string &  values 
) const
virtual

Implements VAPoR::DC.

std::vector<string> VAPoR::VDC::GetAttNames ( string  varname) const
virtual

Return a list of available attribute's names

Returns a vector of all attribute names for the variable, varname. If varname is the empty string the names of all of the global attributes are returned. If varname is not defined an empty vector is returned.

Parameters
[in]varnameThe name of the variable to query, or the empty string if the names of global attributes are desired.
Return values
attnamesA vector of returned attribute names
See also
PutAtt(), GetAtt()

Implements VAPoR::DC.

XType VAPoR::VDC::GetAttType ( string  varname,
string  attname 
) const
virtual

Return the external data type for an attribute

Returns the external storage type of the named variable attribute.

Parameters
[in]varnameThe name of the variable to query, or the empty string if the names of global attributes are desired.
[in]nameName of the attribute.
Return values
Ifan attribute named by name does not exist, a negative value is returned.

Implements VAPoR::DC.

bool VAPoR::VDC::GetBaseVarInfo ( string  varname,
DC::BaseVar var 
) const
virtual

Return metadata about a data or coordinate variable

If the variable varname is defined as either a data or coordinate variable its metadata will be returned in var.

Return values
boolIf the named variable cannot be found false is returned and the values of var are undefined.
See also
GetDataVarInfo(), GetCoordVarInfo()

Implements VAPoR::DC.

void VAPoR::VDC::GetCompressionBlock ( std::vector< size_t > &  bs,
string &  wname,
std::vector< size_t > &  cratios 
) const

Retrieve current compression block settings.

Parameters
[out]bsAn ordered vector containing the current compression block dimensions.
[out]wnameThe wavelet family name.
[out]cratiosA vector of compression of integer compression factors.
See also
SetCompressionBlock()
bool VAPoR::VDC::GetCoordVarInfo ( string  varname,
std::vector< string > &  dimnames,
string &  units,
int &  axis,
XType type,
bool &  compressed,
bool &  uniform 
) const

Return a coordinate variable's definition

This method returns the definition for the coordinate variable named by varname. If varname is not defined as a coordinate variable dimnames will be set to a zero-length vector, and values of all other output parameters will be undefined.

Parameters
[in]varnameA string specifying the name of the variable.
[out]dimnamesThe ordered list of dimension names for this variable
[out]unitsThe variable's units string
[out]axisThe axis associated with the dimension.
[out]typeThe external data storage type
[out]compressedA boolean indicating if the variable is compressed
[out]uniformA boolean indicating if the variable has uniform sampling
Return values
boolIf the named coordinate variable cannot be found false is returned and the values of the output parameters will be undefined.
See also
DefineCoordVar(), DefineCoordVarUniform()
bool VAPoR::VDC::GetCoordVarInfo ( string  varname,
DC::CoordVar cvar 
) const
virtual

Return a coordinate variable's definition

Return a reference to a DC::CoordVar object describing the coordinate variable named by varname

Parameters
[in]varnameA string specifying the name of the coordinate variable.
[out]coordvarA CoordVar object containing the definition of the named variable.
Return values
boolFalse is returned if the named coordinate variable does not exist, and the contents of cvar will be undefined.
See also
DefineCoordVar(), DefineCoordVarUniform(), SetCompressionBlock(), SetPeriodicBoundary()

Implements VAPoR::DC.

virtual std::vector<string> VAPoR::VDC::GetCoordVarNames ( ) const
virtual

Return a list of names for all of the defined coordinate variables.

Returns a list of names for all coordinate variables defined

See also
DefineDataVar()

Implements VAPoR::DC.

bool VAPoR::VDC::GetDataVarInfo ( string  varname,
std::vector< string > &  dimnames,
std::vector< string > &  coordvars,
string &  units,
XType type,
bool &  compressed,
string &  maskvar 
) const

Return a data variable's definition

This method returns the definition for the data variable named by varname.

Parameters
[in]varnameA string specifying the name of the dimension.
[out]dimnamesAn ordered list of dimension names for this variable
[out]coordvarsAn ordered list of coordinate names for this variable
[out]unitsThe variable's units string
[out]typeThe external data storage type
[out]compressedA boolean indicating if the variable is compressed
[out]maskvarA string, possibly empty, containing the name of the mask variable if one exists.
Return values
boolIf the named data variable cannot be found false is returned and the values of the output parameters will be undefined.
See also
DefineCoordVar(), DefineCoordVarUniform()
bool VAPoR::VDC::GetDataVarInfo ( string  varname,
DC::DataVar datavar 
) const
virtual

Return a data variable's definition

Return a reference to a DC::DataVar object describing the data variable named by varname

Parameters
[in]varnameA string specifying the name of the variable.
[out]datavarA DataVar object containing the definition of the named Data variable.
Return values
boolIf the named data variable cannot be found false is returned and the values of datavar are undefined.
See also
DefineCoordVar(), DefineCoordVarUniform(), SetCompressionBlock(), SetPeriodicBoundary()

Implements VAPoR::DC.

virtual std::vector<string> VAPoR::VDC::GetDataVarNames ( ) const
virtual

Return a list of names for all of the defined data variables.

Returns a list of names for all data variables defined

See also
DefineDataVar()

Implements VAPoR::DC.

bool VAPoR::VDC::GetDimension ( string  dimname,
DC::Dimension dimension 
) const
virtual

Return a dimensions's definition

This method returns the definition of the dimension named by dimname as a reference to a DC::Dimension object. If dimname is not defined as a dimension then the name of dimension will be the empty string()

Parameters
[in]dimnameA string specifying the name of the dimension.
[out]dimensionThe returned Dimension object reference
Return values
boolIf the named dimension can not be found false is returned.

Implements VAPoR::DC.

std::vector<string> VAPoR::VDC::GetDimensionNames ( ) const
virtual

Return names of all defined dimensions

This method returns the list of names of all of the dimensions defined in the VDC.

See also
DefineDimension()

Implements VAPoR::DC.

virtual int VAPoR::VDC::GetDimLensAtLevel ( string  varname,
int  level,
std::vector< size_t > &  dims_at_level,
std::vector< size_t > &  bs_at_level 
) const
pure virtual

Return a variable's dimension lengths at a specified refinement level

Compressed variables have a multi-resolution grid representation. This method returns the variable's ordered spatial and temporal dimension lengths, and block dimensions at the multiresolution refinement level specified by level.

If the variable named by varname is not compressed the variable's native dimensions are returned.

Parameters
[in]varnameData or coordinate variable name.
[in]levelSpecifies a member of a multi-resolution variable's grid hierarchy as described above.
[out]dims_at_levelAn ordered vector containing the variable's dimensions at the specified refinement level
[out]bs_at_levelAn ordered vector containing the variable's block dimensions at the specified refinement level
Return values
statusZero is returned upon success, otherwise -1.
See also
VAPoR::VDC, DC::BaseVar::GetBS(), DC::BaseVar::GetDimensions()

Implements VAPoR::DC.

virtual int VAPoR::VDC::GetMapProjection ( string  lonname,
string  latname,
string &  projstring 
) const
virtual

Return a Proj4 map projection string.

For georeference data sets that have map projections this method returns a properly formatted Proj4 projection string for mapping from geographic to cartographic coordinates. If no such projection exists an empty string is returned.

Parameters
[in]lonnameName of longitude coordinate variable
[in]latnameName of latitude coordinate variable
[out]projstringAn empty string if a Proj4 map projection is not available for the named coordinate pair, otherwise a properly formatted Proj4 projection string is returned.

Implements VAPoR::DC.

int VAPoR::VDC::GetNumRefLevels ( string  varname) const
virtual

Return the number of refinement levels for the indicated variable

Compressed variables have a multi-resolution grid representation. This method returns the number of levels in the hiearchy. A value of one indicates that only the native resolution is available. A value of two indicates that two levels, the native plus the next coarsest are available, and so on.

Parameters
[in]varnameData or coordinate variable name.
Return values
numIf varname is unknown zero is returned. if varname is not compressed (has no multi-resolution representation) one is returned. Otherwise the total number of levels in the multi-resolution hierarchy are returned.

Implements VAPoR::DC.

virtual int VAPoR::VDC::GetPath ( string  varname,
size_t  ts,
string &  path,
size_t &  file_ts,
size_t &  max_ts 
) const
pure virtual

Return the path name and temporal offset for a variable

Data and coordinate variables in a VDC are in general distributed into multiple files. For example, for large variables only a single time step may be stored per file. This method returns the file path name, path, of the file containing varname at time step ts. Also returned is the integer time offset of the variable within path.

Parameters
[in]varnameData or coordinate variable name.
[in]tsInteger offset relative to a variable's temporal dimension
[out]pathPath to file containing variable varname at time step ts.
[out]file_tsTemporal offset of variable varname in file path.
[out]max_tsMaximum number of time steps stored in path
Return values
statusA negative int is returned if varname or ts are invalid, or if the class object is in define mode.

Implemented in VAPoR::VDCNetCDF.

std::vector<bool> VAPoR::VDC::GetPeriodicBoundary ( ) const
inline

Retrieve current boundary periodic settings

See also
SetPeriodicBoundary()

Definition at line 331 of file VDC.h.

virtual int VAPoR::VDC::GetVar ( string  varname,
int  level,
int  lod,
float *  data 
)
pure virtual

Read an entire variable in one call

This method reads and entire variable (all time steps, all grid points) from a VDC. This is the simplest interface for reading data from a VDC. If the variable is split across multiple files GetVar() ensures that the data are correctly gathered and assembled into memory Any variables currently opened with OpenVariableRead() are first closed. Thus variables need not be opened with OpenVariableRead() prior to calling GetVar();

It is an error to call this method in define mode

Parameters
[in]varnameName of the variable to write
[in]levelRefinement level of the variable. Ignored if the variable is not compressed.
[in]lodApproximation level of the variable. A value of -1 indicates the maximum approximation level defined for the VDC. Ignored if the variable is not compressed.
[out]dataPointer to where data will be copied. It is the caller's responsbility to ensure data points to sufficient memory.
Return values
statusA negative int is returned on failure
See also
PutVar()

Implements VAPoR::DC.

Implemented in VAPoR::VDCNetCDF.

virtual int VAPoR::VDC::GetVar ( size_t  ts,
string  varname,
int  level,
int  lod,
float *  data 
)
pure virtual

Read an entire variable at a given time step in one call

This method reads and entire variable (all grid points) at time step ts from a VDC. This is the simplest interface for reading data from a VDC. Any variables currently opened with OpenVariableRead() are first closed. Thus variables need not be opened with OpenVariableRead() prior to calling GetVar();

It is an error to call this method in define mode

Parameters
[in]tsTime step of the variable to write. This is the integer offset into the variable's temporal dimension. If the variable does not have a temporal dimension ts is ignored.
[in]varnameName of the variable to write
[in]levelRefinement level of the variable. Ignored if the variable is not compressed.
[in]lodApproximation level of the variable. A value of -1 indicates the maximum approximation level defined for the VDC. Ignored if the variable is not compressed.
[out]dataPointer to where data will be copied. It is the caller's responsbility to ensure data points to sufficient memory.
Return values
statusA negative int is returned on failure
See also
PutVar()

Implements VAPoR::DC.

Implemented in VAPoR::VDCNetCDF.

virtual int VAPoR::VDC::Initialize ( const std::vector< string > &  paths,
AccessMode  mode 
)
virtual

Initialize the VDC class

Prepare a VDC for reading or writing/appending. This method prepares the master VDC file indicated by path for reading or writing. The method should be called immediately after the constructor, before any other class methods. This method exists only because C++ constructors can not return error codes.

Parameters
[in]pathA single element vector that specifies the name of file that contains, or will contain, the VDC master file for this data collection
[in]modeOne of R, W, or A, indicating whether path will be opened for reading, writing, or appending, respectively. When mode is A underlying NetCDF files will be opened opened with nc_open(path, NC_WRITE)). When mode is W NetCDF files will be created (opened with nc_create(path)). When mode is A additional time steps may be added to an existing file.
Note
The parameter mode controls the access to the master file indicated by path and the variable data files in a somewhat unintuitive manner. If mode is R or A the master file path must already exist. If mode is A or W the contents of the VDC master may be changed (written) and the VDC is put into define mode until EndDefine() is called. While in define mode metadata that will be contained in the VDC master file may be changed, but coordinate and data variables may not be accessed (read or written). Similarly, when not in define mode coordinate and data variables may be accessed (read or written), but metadata in the VDC master may not be changed. See OpenVariableRead() and OpenVariableWrite() for discussion on how mode effects reading and writing of coordinate and data variables.
Return values
statusA negative int is returned on failure
See also
EndDefine();
virtual int VAPoR::VDC::Initialize ( const std::vector< string > &  paths)
inlinevirtual

Definition at line 230 of file VDC.h.

virtual int VAPoR::VDC::OpenVariableRead ( size_t  ts,
string  varname,
int  level = 0,
int  lod = 0 
)
pure virtual

Open the named variable for reading

This method prepares a data or coordinate variable, indicated by a variable name and time step pair, for subsequent read operations by methods of this class. The value of the refinement levels parameter, level, indicates the resolution of the volume in the multiresolution hierarchy as described by GetDimLensAtLevel().

The level-of-detail parameter, lod, selects the approximation level. Valid values for lod are integers in the range 0..n-1, where n is returned by DC::BaseVar::GetCRatios().size(), or the value -1 may be used to select the best approximation available.

An error occurs, indicated by a negative return value, if the volume identified by the {varname, timestep, level, lod} tupple is not available. Note the availability of a volume can be tested with the VariableExists() method.

Parameters
[in]tsTime step of the variable to read. This is the integer offset into the variable's temporal dimension. If the variable does not have a temporal dimension ts is ignored.
[in]varnameName of the variable to read
[in]levelRefinement level of the variable. Ignored if the variable is not compressed.
[in]lodApproximation level of the variable. A value of -1 indicates the maximum approximation level defined for the VDC. Ignored if the variable is not compressed.
Return values
statusReturns a non-negative value on success
See also
GetNumRefLevels(), DC::BaseVar::GetCRatios(), OpenVariableRead()

Implements VAPoR::DC.

Implemented in VAPoR::VDCNetCDF.

virtual int VAPoR::VDC::OpenVariableWrite ( size_t  ts,
string  varname,
int  lod = -1 
)
pure virtual

Open the named variable for writing

This method prepares a data or coordinate variable, indicated by a variable name and time step pair, for subsequent write operations by methods of this class.

The behavior of this method is impacted somewhat by the setting of the Initialize() mode parameter. Coordinate or data variable files may be written regardless of the mode setting. However, if mode is W the first time a coordinate or data file is written it will be created (opened with nc_create(path), for example) regareless of whether the file previously existed. If mode is A or R existing coordinate or data files will be opened for appending (e.g. opened with nc_open(path, NC_WRITE)). New files will be created (opened with nc_create(path)).

An error occurs, indicated by a negative return value, if the varible identified by the {varname, timestep, lod} tupple is not defined.

Parameters
[in]tsTime step of the variable to read. This is the integer offset into the variable's temporal dimension. If the variable does not have a temporal dimension ts is ignored.
[in]varnameName of the variable to read
[in]lodApproximation level of the variable. A value of -1 indicates the maximum approximation level defined for the VDC. Ignored if the variable is not compressed.
Return values
statusReturns a non-negative value on success
See also
GetNumRefLevels(), DC::BaseVar::GetCRatios(), OpenVariableRead()

Implemented in VAPoR::VDCNetCDF.

int VAPoR::VDC::PutAtt ( string  varname,
string  attname,
XType  type,
const vector< double > &  values 
)

Write an attribute

This method write an attribute to the VDC. The attribute can either be "global", if varname is the empty string, or bound to a variable if varname indentifies a variable in the VDC.

Parameters
[in]varnameThe name of a variable already defined in the VDC, or the empty string if the attribute is to be global
[in]attnameThe attributes name
[in]typeThe primitive data type storage format. This is the type that will be used to store the attribute on disk
[in]valuesA vector of floating point attribute values
Return values
statusA negative int is returned on failure
See also
GetAtt()
int VAPoR::VDC::PutAtt ( string  varname,
string  attname,
XType  type,
const vector< long > &  values 
)
int VAPoR::VDC::PutAtt ( string  varname,
string  attname,
XType  type,
const string &  values 
)
virtual int VAPoR::VDC::PutVar ( string  varname,
int  lod,
const float *  data 
)
pure virtual

Write an entire variable in one call

This method writes and entire variable (all time steps, all grid points) into a VDC. This is the simplest interface for writing data into a VDC. If the variable is split across multiple files PutVar() ensures that the data are correctly distributed. Any variables currently opened with OpenVariableWrite() are first closed. Thus variables need not be opened with OpenVariableWrite() prior to calling PutVar();

It is an error to call this method in define mode

Parameters
[in]varnameName of the variable to write
[in]lodApproximation level of the variable. A value of -1 indicates the maximum approximation level defined for the VDC. Ignored if the variable is not compressed.
[in]dataPointer from where the data will be copied
Return values
statusA negative int is returned on failure
See also
GetVar()

Implemented in VAPoR::VDCNetCDF.

virtual int VAPoR::VDC::PutVar ( size_t  ts,
string  varname,
int  lod,
const float *  data 
)
pure virtual

Write a variable at single time step

This method writes a variable hyperslab consisting of the variable's entire spatial dimensions at the time step indicated by ts. Any variables currently opened with OpenVariableWrite() are first closed. Thus variables need not be opened with OpenVariableWrite() prior to calling PutVar();

It is an error to call this method in define mode

Parameters
[in]tsTime step of the variable to write. This is the integer offset into the variable's temporal dimension. If the variable does not have a temporal dimension ts is ignored.
[in]varnameName of the variable to write
[in]lodApproximation level of the variable. A value of -1 indicates the maximum approximation level defined for the VDC. Ignored if the variable is not compressed.
[in]dataPointer from where the data will be copied
Return values
statusA negative int is returned on failure
See also
GetVar()

Implemented in VAPoR::VDCNetCDF.

virtual int VAPoR::VDC::Read ( float *  data)
pure virtual

Read all spatial values of the currently opened variable

This method reads, and decompresses as necessary, the contents of the currently opened variable into the array data. The number of values read into data is given by the product of the spatial dimensions of the open variable at the refinement level specified.

It is the caller's responsibility to ensure data points to adequate space.

Parameters
[out]dataAn array of data to be written
Return values
statusReturns a non-negative value on success
See also
OpenVariableRead()

Implements VAPoR::DC.

Implemented in VAPoR::VDCNetCDF.

virtual int VAPoR::VDC::ReadRegion ( const vector< size_t > &  min,
const vector< size_t > &  max,
float *  region 
)
pure virtual

Read in and return a subregion from the currently opened variable

This method reads and returns a subset of variable data. The min and max vectors, whose dimensions must match the spatial rank of the currently opened variable, identify the minimum and maximum extents, in grid coordinates, of the subregion of interest. The minimum and maximum valid values of an element of min or max are 0 and n-1, respectively, where n is the length of the associated dimension at the opened refinement level.

The region returned is stored in the memory region pointed to by region. It is the caller's responsbility to ensure adequate space is available.

Parameters
[in]minMinimum region extents in grid coordinates
[in]maxMaximum region extents in grid coordinates
[out]regionThe requested volume subregion
Return values
statusReturns a non-negative value on success
See also
OpenVariableRead(), GetDimension(), GetDimensionNames()

Implements VAPoR::DC.

virtual int VAPoR::VDC::ReadRegionBlock ( const vector< size_t > &  min,
const vector< size_t > &  max,
float *  region 
)
pure virtual

Read in and return a blocked subregion from the currently opened variable.

This method is identical to ReadRegion() with the exceptions that for compressed variables:

  • The vectors start and count must be aligned with the underlying storage block of the variable. See DC::SetCompressionBlock()
  • The hyperslab copied to region will preserve its underlying storage blocking (the data will not be contiguous)

Implements VAPoR::DC.

Implemented in VAPoR::VDCNetCDF.

virtual int VAPoR::VDC::ReadSlice ( float *  slice)
pure virtual

Read a single slice of data from the currently opened variable

Decompress, as necessary, and read a single slice (2D array) of data from the variable indicated by the most recent call to OpenVariableRead(). The dimensions of a slices are NX by NY, where NX is the dimension of the array along the fastest varying spatial dimension, specified in grid points, and NY is the length of the second fastest varying dimension at the currently opened grid refinement level. See OpenVariableRead().

This method should be called exactly NZ times for each opened variable, where NZ is the dimension of third, and slowest varying dimension. In the case of a 2D variable, NZ is 1.

It is the caller's responsibility to ensure slice points to adequate space.

Parameters
[out]sliceA 2D slice of data
Return values
statusReturns a non-negative value on success
See also
OpenVariableRead()

Implements VAPoR::DC.

Implemented in VAPoR::VDCNetCDF.

int VAPoR::VDC::SetCompressionBlock ( std::vector< size_t >  bs,
string  wname,
std::vector< size_t >  cratios 
)

Sets various parameters for storage blocks for subsequent variable definitions

This method sets the storage parameters for subsequent variable definitions for compressed variables. bs is a three-element array, with the first element specifying the length of the fastest varying dimension (e.g. X) of the storage block, the second element specifies the length of the next fastest varying dimension, etc. If a variable definition defines a variable with n spatial dimensions, where n is less than three, only the first n elements of bs will be used. For example, if the rank of bs is greater than two a 2D variable will be stored in blocks having dimensions bs[0] x bs[1].

Variables whose spatial dimension lengths are less than the coresponding dimension of bs will be padded to block boundaries.

wname set the wavelet family name and boundary handling mode for subsequent compressed variable definitions. Wider wavelets (those requiring more filter coefficients) will typically yield higher compression rates, but are more computationally expensive and will limit the depth of of the grid resolution refinement hierarchy.

Recommended values for wname are bior1.1, bior1.3, bior1.5 bior3.3, bior3.5, bior3.7, bior3.9, bior2.2, bior2.6, bior2.6, and bior2.8. For odd length filters (e.g. bior1.3)

Finally, cratios specifies a vector of compression factors for subsequent compressed variable definitions.

Note
For compressed variables compression is applied to individual blocks. Larger blocks permit deeper grid refinement hierarchies, but may result in poor cache performance and slowed disk storage access
The wavelet and compression ratio parameters are ignored by variable definitions for variables that are not compressed.
Parameters
[in]bsA one to three-element array specifying the storage block size. All elements of must be great than or equal to one. The default value of bs is (64, 64, 64).
[in]wnameA wavelet family name. The default value is "bior4.4".
[in]cratiosA vector of compression of integer compression factors. The default compression ratio vector is: (1, 10, 100, 500)
Return values
statusA negative int is returned if an invalid parameter or parameter combination is specified.
See also
DefineDataVar(), DefineCoordVar(), VDC()
virtual int VAPoR::VDC::SetMapProjection ( string  lonname,
string  latname,
string  projstring 
)
virtual

Set a map projection string for a lat-lon coordinate pair

This method sets a Proj4 map projection string for the longitude-latitude coordinate pair indicated by lonname and latname

See also
GetMapProjection()
void VAPoR::VDC::SetPeriodicBoundary ( std::vector< bool >  periodic)
inline

Set the boundary periodic for subsequent variable definitions

This method specifies an ordered, three-element boolean vector indicating the boundary periodicty for a variable's spatial dimensions. The ordering is from fastest to slowest varying dimension.

Parameters
[in]periodicA three-element array of booleans. The default value of periodic is (false, false, false).
Return values
statusA negative int is returned on error

Definition at line 322 of file VDC.h.

virtual bool VAPoR::VDC::VariableExists ( size_t  ts,
string  varname,
int  reflevel = 0,
int  lod = 0 
) const
pure virtual

Returns true if indicated data volume is available

Returns true if the variable identified by the timestep, variable name, refinement level, and level-of-detail is present in the data set. Returns false if the variable is not available.

Parameters
[in]tsA valid time step between 0 and GetNumTimesteps()-1
[in]varnameA valid variable name
[in]reflevelRefinement level requested.
[in]lodCompression level of detail requested. refinement level contained in the VDC.

Implements VAPoR::DC.

Implemented in VAPoR::VDCNetCDF.

virtual int VAPoR::VDC::Write ( const float *  data)
pure virtual

Write all spatial values to the currently opened variable

This method writes, and compresses as necessary, the contents of the array contained in data to the currently opened variable. The number of values written from data is given by the product of the spatial dimensions of the open variable.

Parameters
[in]dataAn array of data to be written
Return values
statusReturns a non-negative value on success
See also
OpenVariableWrite()

Implemented in VAPoR::VDCNetCDF.

virtual int VAPoR::VDC::WriteSlice ( const float *  slice)
pure virtual

Write a single slice of data to the currently opened variable

Compress, and necessary, and write a single slice (2D array) of data to the variable indicated by the most recent call to OpenVariableWrite(). The dimensions of a slices are NX by NY, where NX is the dimension of the array along the fastest varying spatial dimension, specified in grid points, and NY is the length of the second fastest varying dimension at the currently opened refinement level. See OpenVariableWrite().

This method should be called exactly NZ times for each opened variable, where NZ is the dimension of third, and slowest varying dimension. In the case of a 2D variable, NZ is 1.

Parameters
[in]sliceA 2D slice of data
Return values
statusReturns a non-negative value on success
See also
OpenVariableWrite()

Implemented in VAPoR::VDCNetCDF.

Friends And Related Function Documentation

std::ostream& operator<< ( std::ostream &  o,
const VDC vdc 
)
friend

Member Data Documentation

std::map<string, Attribute> VAPoR::VDC::_atts
protected

Definition at line 1257 of file VDC.h.

std::vector<size_t> VAPoR::VDC::_bs
protected

Definition at line 1250 of file VDC.h.

std::map<string, CoordVar> VAPoR::VDC::_coordVars
protected

Definition at line 1258 of file VDC.h.

std::vector<size_t> VAPoR::VDC::_cratios
protected

Definition at line 1252 of file VDC.h.

std::map<string, DataVar> VAPoR::VDC::_dataVars
protected

Definition at line 1259 of file VDC.h.

bool VAPoR::VDC::_defineMode
protected

Definition at line 1249 of file VDC.h.

std::map<string, Dimension> VAPoR::VDC::_dimsMap
protected

Definition at line 1256 of file VDC.h.

string VAPoR::VDC::_master_path
protected

Definition at line 1247 of file VDC.h.

AccessMode VAPoR::VDC::_mode
protected

Definition at line 1248 of file VDC.h.

vector<bool> VAPoR::VDC::_periodic
protected

Definition at line 1253 of file VDC.h.

VAPoR::UDUnits VAPoR::VDC::_udunits
protected

Definition at line 1254 of file VDC.h.

string VAPoR::VDC::_wname
protected

Definition at line 1251 of file VDC.h.


The documentation for this class was generated from the following file: