VAPoR  3.0.0
Classes | Public Member Functions | List of all members
VAPoR::DataMgrV3_0 Class Reference

A cache based data reader. More...

#include <DataMgrV3_0.h>

Inheritance diagram for VAPoR::DataMgrV3_0:
VetsUtil::MyBase

Classes

class  BlkExts
 

Public Member Functions

 DataMgrV3_0 (string format, size_t mem_size, int nthreads=0)
 
virtual ~DataMgrV3_0 ()
 
virtual int Initialize (const std::vector< string > &files)
 
virtual std::vector< string > GetDataVarNames () const
 
virtual std::vector< string > GetDataVarNames (int ndim, bool spatial) const
 
virtual std::vector< string > GetCoordVarNames () const
 
virtual std::vector< string > GetCoordVarNames (int ndim, bool spatial) const
 
void GetTimeCoordinates (std::vector< double > &timecoords) const
 
bool GetDataVarInfo (string varname, VAPoR::DC::DataVar &datavar) const
 
bool GetBaseVarInfo (string varname, VAPoR::DC::BaseVar &var) const
 
bool GetCoordVarInfo (string varname, VAPoR::DC::CoordVar &cvar) const
 
bool IsTimeVarying (string varname) const
 
bool IsCompressed (string varname) const
 
int GetNumTimeSteps (string varname) const
 
int GetNumRefLevels (string varname) const
 
int GetCRatios (string varname, std::vector< size_t > &cratios) const
 
VAPoR::RegularGridGetVariable (size_t ts, string varname, int level, int lod, bool lock=false)
 
VAPoR::RegularGridGetVariable (size_t ts, string varname, int level, int lod, std::vector< double > min, std::vector< double > max, bool lock=false)
 
VAPoR::RegularGridGetVariable (size_t ts, string varname, int level, int lod, std::vector< size_t > min, std::vector< size_t > max, bool lock=false)
 
int GetVariableExtents (size_t ts, string varname, int level, std::vector< double > &min, std::vector< double > &max)
 
virtual int GetDimLensAtLevel (string varname, int level, bool spatial, std::vector< size_t > &dims_at_level, std::vector< size_t > &bs_at_level) const
 
void UnlockGrid (const VAPoR::RegularGrid *rg)
 
void Clear ()
 
virtual bool VariableExists (size_t ts, string varname, int level=0, int lod=0)
 
bool IsVariableDerived (string varname) const
 
bool IsVariableNative (string varname) const
 
void PurgeVariable (string varname)
 
virtual string GetMapProjection () const
 
- Public Member Functions inherited from VetsUtil::MyBase
 MyBase ()
 
const string & getClassName () const
 

Additional Inherited Members

- Public Types inherited from VetsUtil::MyBase
typedef void(* ErrMsgCB_T) (const char *msg, int err_code)
 
typedef void(* DiagMsgCB_T) (const char *msg)
 
- Static Public Member Functions inherited from VetsUtil::MyBase
static void SetErrMsg (const char *format,...)
 Record a formatted error message. More...
 
static void SetErrMsg (int errcode, const char *format,...)
 Record a formatted error message and an error code. More...
 
static const char * GetErrMsg ()
 
static void SetErrCode (int err_code)
 Record an error code. More...
 
static int GetErrCode ()
 Retrieve the current error code. More...
 
static void SetErrMsgCB (ErrMsgCB_T cb)
 
static ErrMsgCB_T GetErrMsgCB ()
 
static void SetErrMsgFilePtr (FILE *fp)
 
static const FILE * SetErrMsgFilePtr ()
 
static void SetDiagMsg (const char *format,...)
 Record a formatted diagnostic message. More...
 
static const char * GetDiagMsg ()
 
static void SetDiagMsgCB (DiagMsgCB_T cb)
 
static DiagMsgCB_T GetDiagMsgCB ()
 
static void SetDiagMsgFilePtr (FILE *fp)
 
static bool EnableErrMsg (bool enable)
 
- Static Public Attributes inherited from VetsUtil::MyBase
static char * ErrMsg
 
static int ErrCode
 
static int ErrMsgSize
 
static FILE * ErrMsgFilePtr
 
static ErrMsgCB_T ErrMsgCB
 
static char * DiagMsg
 
static int DiagMsgSize
 
static FILE * DiagMsgFilePtr
 
static DiagMsgCB_T DiagMsgCB
 
static bool Enabled
 
- Protected Member Functions inherited from VetsUtil::MyBase
void SetClassName (const string &name)
 

Detailed Description

A cache based data reader.

Author
John Clyne

The DataMgr class is an abstract class that defines public methods for accessing (reading) 1D, 2D and 3D field variables. The class implements a memory cache to speed data access – once a variable is read it is stored in cache for subsequent access. The DataMgr class is abstract: it declares a number of public and protected pure virtual methods that must be implemented by specializations of this class to allow access to particular file formats.

This class inherits from VetsUtil::MyBase. Unless otherwise documented any method that returns an integer value is returning status. A negative value indicates failure. Error messages are logged via VetsUtil::MyBase::SetErrMsg().

Methods that return a boolean do not, unless otherwise documented, log an error message upon failure (return of false).

Parameters
level

Grid refinement level for multiresolution variables. Compressed variables in the VDC have a multi-resolution representation: the sampling grid for multi-resolution variables is hierarchical, and the dimension lengths of adjacent levels in the hierarchy differ by a factor of two. The level parameter is used to select a particular depth of the hierarchy.

To provide maximum flexibility as well as compatibility with previous versions of the VDC the interpretation of level is somewhat complex. Both positive and negative values may be used to specify the refinement level and have different interpretations.

For positive values of level, a value of 0 indicates the coarsest member of the grid hierarchy. A value of 1 indicates the next grid refinement after the coarsest, and so on. Using postive values the finest level in the hierarchy is given by GetNumRefLevels() - 1. Values of level that are greater than GetNumRefLevels() - 1 are treated as if they were equal to GetNumRefLevels() - 1.

For negative values of level a value of -1 indicates the variable's native grid resolution (the finest resolution available). A value of -2 indicates the next coarsest member in the hierarchy after the finest, and so on. Using negative values the coarsest available level in the hierarchy is given by negating the value returned by GetNumRefLevels(). Values of level that are less than the negation of GetNumRefLevels() are treated as if they were equal to the negation of the GetNumRefLevels() return value.

Parameters
lod

The level-of-detail parameter, lod, selects the approximation level for a compressed variable. The lod parameter is similar to the level parameter in that it provides control over accuracy of a compressed variable. However, instead of selecting the grid resolution the lod parameter controls the compression factor by indexing into the cratios vector (see below). As with the level parameter, both positive and negative values may be used to index into cratios and different interpretations.

For positive values of lod, a value of 0 indicates the the first element of cratios, a value of 1 indicates the second element, and so on up to the size of the cratios vector (See DC::GetCRatios()).

For negative values of lod a value of -1 indexes the last element of cratios, a value of -2 indexes the second to last element, and so on. Using negative values the first element of cratios - the greatest compression rate - is indexed by negating the size of the cratios vector.

Definition at line 100 of file DataMgrV3_0.h.

Constructor & Destructor Documentation

VAPoR::DataMgrV3_0::DataMgrV3_0 ( string  format,
size_t  mem_size,
int  nthreads = 0 
)

Constructor for the DataMgrV3_0 class.

The DataMgrV3_0 will attempt to cache previously read data and coordinate variables in memory. The mem_size specifies the requested cache size in MEGABYTES!!!

Parameters
[in]formatA string indicating the format of data collection.
[in]mem_sizeSize of memory cache to be created, specified in MEGABYTES!!
[in]numthreadsNumber of parallel execution threads to be run during encoding and decoding of compressed data. A value of 0, the default, indicates that the thread count should be determined by the environment in a platform-specific manner, for example using sysconf(_SC_NPROCESSORS_ONLN) under *nix OSes.
virtual VAPoR::DataMgrV3_0::~DataMgrV3_0 ( )
virtual

Member Function Documentation

void VAPoR::DataMgrV3_0::Clear ( )

Clear the memory cache

This method clears the internal memory cache of all entries

bool VAPoR::DataMgrV3_0::GetBaseVarInfo ( string  varname,
VAPoR::DC::BaseVar var 
) const

Return metadata about a data or coordinate variable

If the variable varname is defined as either a data or coordinate variable its metadata will be returned in var.

Return values
boolIf true the method was successful. If false the named variable is invalid (unknown)
See also
GetDataVarInfo(), GetCoordVarInfo()
bool VAPoR::DataMgrV3_0::GetCoordVarInfo ( string  varname,
VAPoR::DC::CoordVar cvar 
) const

Return a coordinate variable's definition

Return a reference to a DC::CoordVar object describing the coordinate variable named by varname

Parameters
[in]varnameA string specifying the name of the coordinate variable.
[out]coordvarA CoordVar object containing the definition of the named variable.
Return values
boolIf true the method was successful. If false the named variable is invalid (unknown)
See also
GetDataVarInfo()
virtual std::vector<string> VAPoR::DataMgrV3_0::GetCoordVarNames ( ) const
virtual

Return a list of names for all of the defined coordinate variables.

This method returns a list of all coordinate variables defined in the data set.

Return values
listA vector containing a list of all the coordinate variable names
See also
GetDataVarNames()
virtual std::vector<string> VAPoR::DataMgrV3_0::GetCoordVarNames ( int  ndim,
bool  spatial 
) const
virtual

Return a list of coordinate variables with a given dimension rank

This method returns a list of all coordinate variables defined in the coordinate set with the specified dimensionality

Parameters
[in]ndimVariable rank
[in]spatialA boolean indicating whether only the variables spatial rank should be compared against ndim
Return values
listA vector containing a list of all the coordinate variable names with the specified number of dimensions.
int VAPoR::DataMgrV3_0::GetCRatios ( string  varname,
std::vector< size_t > &  cratios 
) const

Return the compression ratio vector for the indicated variable

Return the compression ratio vector for the indicated variable. The vector returned contains an ordered list of available compression ratios for the variable named by variable. If the variable is not compressed, the cratios parameter will contain a single element, one.

Parameters
[in]varnameData or coordinate variable name.
[out]cratiosOrdered vector of compression ratios
Return values
statusA negative int is returned on failure
See also
DC::BaseVar::GetCRatios();
bool VAPoR::DataMgrV3_0::GetDataVarInfo ( string  varname,
VAPoR::DC::DataVar datavar 
) const

Return a data variable's definition

Return a reference to a DC::DataVar object describing the data variable named by varname

Parameters
[in]varnameA string specifying the name of the variable.
[out]datavarA DataVar object containing the definition of the named Data variable.
Return values
boolIf true the method was successful. If false the named variable is invalid (unknown)
See also
GetCoordVarInfo()
virtual std::vector<string> VAPoR::DataMgrV3_0::GetDataVarNames ( ) const
virtual

Return a list of names for all of the defined data variables.

This method returns a list of all data variables defined in the data set.

Return values
listA vector containing a list of all the data variable names
See also
GetCoordVarNames()
Test:
New in 3.0
virtual std::vector<string> VAPoR::DataMgrV3_0::GetDataVarNames ( int  ndim,
bool  spatial 
) const
virtual

Return a list of data variables with a given dimension rank

This method returns a list of all data variables defined in the data set with the specified dimension rank (number of dimensions). Data variables may have 0 to 3 spatial dimensions, and 0 of 1 time dimension.

Parameters
[in]ndimVariable rank (number of dimensions)
[in]spatialA boolean, if true, indicates that only the variable's spatial rank should be compared against ndim. Otherwise the sum of the number of spatial dimensions plus the time dimension (if time varying) is compared agains ndim.
Return values
listA vector containing a list of all the data variable names with the specified number of dimensions (rank).
Test:
New in 3.0
virtual int VAPoR::DataMgrV3_0::GetDimLensAtLevel ( string  varname,
int  level,
bool  spatial,
std::vector< size_t > &  dims_at_level,
std::vector< size_t > &  bs_at_level 
) const
virtual

Return a variable's dimension lengths at a specified refinement level

Compressed variables have a multi-resolution grid representation. This method returns the variable's ordered spatial and temporal dimension lengths, and block dimensions at the multiresolution refinement level specified by level.

If the variable named by varname is not compressed the variable's native dimensions are returned.

Parameters
[in]varnameData or coordinate variable name.
[in]levelSpecifies a member of a multi-resolution variable's grid hierarchy as described above.
[out]dims_at_levelAn ordered vector containing the variable's dimensions at the specified refinement level
[out]bs_at_levelAn ordered vector containing the variable's block dimensions at the specified refinement level
Return values
statusZero is returned upon success, otherwise -1.
See also
VAPoR::DC, DC::BaseVar::GetBS(), DC::BaseVar::GetDimensions()
Parameters
[in]spatialA boolean, if true, indicates that only the variable's spatial dimensions and block size should be returned. A time varying dimension, if one exists, is not included
virtual string VAPoR::DataMgrV3_0::GetMapProjection ( ) const
inlinevirtual
Todo:
Need to define

Definition at line 528 of file DataMgrV3_0.h.

int VAPoR::DataMgrV3_0::GetNumRefLevels ( string  varname) const

Return the number of refinement levels for the indicated variable

Compressed variables have a multi-resolution grid representation. This method returns the number of levels in the hiearchy. A value of one indicates that only the native resolution is available. A value of two indicates that two levels, the native plus the next coarsest are available, and so on.

Parameters
[in]varnameData or coordinate variable name.
Return values
numIf varname is unknown zero is returned. if varname is not compressed (has no multi-resolution representation) one is returned. Otherwise the total number of levels in the multi-resolution hierarchy are returned.
int VAPoR::DataMgrV3_0::GetNumTimeSteps ( string  varname) const

Return the time dimension length for a variable

Returns the number of time steps (length of the time dimension) for which a variable is defined. If varname does not have a time coordinate 1 is returned. If varname is not defined as a variable a negative int is returned.

Parameters
[in]varnameA string specifying the name of the variable.
Return values
countThe length of the time dimension, or a negative int if varname is undefined.
See also
IsTimeVarying()
void VAPoR::DataMgrV3_0::GetTimeCoordinates ( std::vector< double > &  timecoords) const
inline

Get time coordinates

Get a sorted list of all time coordinates defined for the data set. Multiple time coordinate variables may be defined. This method collects the time coordinates from all time coordinate variables and sorts them into a single, global time coordinate variable.

Note
Need to deal with different units (e.g. seconds and days).
Should methods that take a time step argument ts expect "local" or "global" time steps

Definition at line 208 of file DataMgrV3_0.h.

VAPoR::RegularGrid* VAPoR::DataMgrV3_0::GetVariable ( size_t  ts,
string  varname,
int  level,
int  lod,
bool  lock = false 
)

Read and return variable data

Reads all data for the data or coordinate variable named by varname for the time step, refinement level, and leve-of-detail indicated by ts, level, and lod, respectively.

Parameters
[in]tsAn integer offset into the time coordinate variable returned by GetTimeCoordinates() indicting the time step of the variable to access.
[in]varnameThe name of the data or coordinate variable to access
[in]levelGrid refinement level. See DataMgr
[in]lodThe level-of-detail parameter, lod, selects the approximation level. See DataMgr.
VAPoR::RegularGrid* VAPoR::DataMgrV3_0::GetVariable ( size_t  ts,
string  varname,
int  level,
int  lod,
std::vector< double >  min,
std::vector< double >  max,
bool  lock = false 
)

Read and return a variable hyperslab

This method is identical to the GetVariable() method, however, a subregion is specified in the user coordinate system. min and max specify the minimum and maximum extents of an axis-aligned bounding box containing the region of interest. The VAPoR::RegularGrid object returned contains the intersection between the specified hyperslab and the variable's spatial domain (which is not necessarily rectangular or axis-aligned). If the requested hyperslab lies entirely outside of the domain of the requested variable NULL is returned

Parameters
[in]minA one, two, or three element array specifying the minimum extents, in user coordinates, of an axis-aligned box defining the region-of-interest. The spatial dimensionality of the variable determines the number of elements in min.
[in]maxA one, two, or three element array specifying the maximum extents, in user coordinates, of an axis-aligned box defining the region-of-interest. The spatial dimensionality of the variable determines the number of elements in max.
Note
The RegularGrid structure returned is allocated from the heap. it is the caller's responsiblity to delete the returned object when it is no longer in use.
VAPoR::RegularGrid* VAPoR::DataMgrV3_0::GetVariable ( size_t  ts,
string  varname,
int  level,
int  lod,
std::vector< size_t >  min,
std::vector< size_t >  max,
bool  lock = false 
)
int VAPoR::DataMgrV3_0::GetVariableExtents ( size_t  ts,
string  varname,
int  level,
std::vector< double > &  min,
std::vector< double > &  max 
)

Compute the coordinate extents of a variable

This method finds the spatial domain extents of a variable

This method returns the min and max extents specified in user coordinates, of the smallest axis-aligned bounding box that is guaranteed to contain the variable indicated by varname, and the given refinement level, level

virtual int VAPoR::DataMgrV3_0::Initialize ( const std::vector< string > &  files)
virtual

Initialize the class

This method must be called to initialize the DataMgrV3_0 class with a list of input data files.

Parameters
[in]filesA list of file paths
Return values
statusA negative int is returned on failure and an error message will be logged with MyBase::SetErrMsg()
bool VAPoR::DataMgrV3_0::IsCompressed ( string  varname) const

Return a boolean indicating whether a variable is compressed

This method returns true if the variable named by varname is defined and it has a compressed representation. If either of these conditions is not true the method returns false.

Parameters
[in]varnameA string specifying the name of the variable.
Return values
boolReturns true if variable varname exists and is compressed
See also
DefineCoordVar(), DefineDataVar(), DC::BaseVar::GetCompressed()
bool VAPoR::DataMgrV3_0::IsTimeVarying ( string  varname) const

Return a boolean indicating whether a variable is time varying

This method returns true if the variable named by varname is defined and it has a time axis dimension. If either of these conditions is not true the method returns false.

Parameters
[in]varnameA string specifying the name of the variable.
Return values
boolReturns true if variable varname exists and is time varying.
bool VAPoR::DataMgrV3_0::IsVariableDerived ( string  varname) const

Return true if the named variable is the output of a pipeline

This method returns true if varname matches a variable name in the output list (PipeLine::GetOutputs()) of any pipeline added with NewPipeline()

See also
NewPipeline()
bool VAPoR::DataMgrV3_0::IsVariableNative ( string  varname) const

Return true if the named variable is availble from the derived classes data access methods.

A return value of true does not imply that the variable can be read (

See also
VariableExists()), only that it is part of the data set known to the derived class
NewPipeline()
void VAPoR::DataMgrV3_0::PurgeVariable ( string  varname)

Purge the cache of a variable

Parameters
[in]varnameis the variable name
void VAPoR::DataMgrV3_0::UnlockGrid ( const VAPoR::RegularGrid rg)

Unlock a floating-point region of memory

Decrement the lock counter associatd with a region of memory, and if zero, unlock region of memory previously locked with GetVariable(). When the lock counter reaches zero the region is simply marked available for internal garbage collection during subsequent GetVariable() calls

Parameters
[in]rgA pointer to a RegularGrid previosly returned by GetVariable()
Return values
statusReturns a non-negative value on success
See also
GetVariable()
virtual bool VAPoR::DataMgrV3_0::VariableExists ( size_t  ts,
string  varname,
int  level = 0,
int  lod = 0 
)
virtual

Returns true if indicated data volume is available

Returns true if the variable identified by the timestep, variable name, refinement level, and level-of-detail is present in the data set. Returns false if the variable is not available.

Parameters
[in]tsA valid time step between 0 and GetNumTimesteps()-1
[in]varnameA valid variable name
[in]levelRefinement level requested.
[in]lodCompression level of detail requested. refinement level contained in the DC.

The documentation for this class was generated from the following file: