5. Geo Services Guide¶

This guide covers the details of geo data management and retrieval in rasdaman. The rasdaman Array DBMS is domain agnostic; the specific semantics of space and time is provided through a layer on top of rasdaman, historically known as petascope. It offers spatio-temporal access and analytics through APIs based on the OGC data standard Coverage Implementation Schema (CIS) and the OGC service standards Web Map Service (WMS), Web Coverage Service (WCS), and Web Coverage Processing Service (WCPS).

Note

While the name petascope addresses a specific component we frequently use the name rasdaman to refer to the complete system, including petascope.

5.1. OGC Coverage Standards Overview¶

For operating rasdaman geo services as well as for accessing such geo services through these APIs it is important to understand the mechanics of the relevant standards. In particular, the concept of OGC / ISO coverages is important.

In standardization, coverages are used to represent space/time varying phenomena, concretely: regular and irregular grids, point clouds, and general meshes. The coverage standards offer data and service models for dealing with those. In rasdaman, focus is on multi-dimensional gridded (“raster”) coverages.

In rasdaman, the OGC standards WMS, WCS, and WCPS are supported, being reference implementation for WCS. These APIs serve different purposes:

WMS delivers a 2D map as a visual image, suitable for consunmption by humans
WCS delivers n-D data, suitable for further processing and analysis
WCPS performs flexible server-side processing, filtering, analytics, and fusion on coverages.

These coverage data and service concepts are summarized briefly below. Ample material is also available on the Web for familiarization with coverages (best consult in this sequence):

hands-on demos for multi-dimensional coverage services provided by Jacobs University;
a series of webinars and tutorial slides provided by EarthServer;
a range of background information on these standards provided by OGC;
the official standards documents maintained by OGC:

WCS 2.0.1

WCS-T 2.0

WCPS 1.0

WMS 1.3.0

5.1.1. Coverage Data¶

OGC CIS specifies an interoperable, conformance-testable coverage structure independent from any particular format encoding. Encodings are defined in OGC in GML, JSON, RDF, as well as a series of binary formats including GeoTIFF, netCDF, JPEG2000, and GRIB2).

By separating the data definition (CIS) from the service definition (WCS) it is possible for coverages to be served throuigh a variety of APIs, such as WMS, WPS, and SOS. However, WCS and WCPS have coverage-specific functionality making them particularly suitable for flexible coverage acess, analytics, and fusion.

5.1.2. Coverage Services¶

OGC WMS delivers 2D maps generated from styled layers stacked up. As such, WMS is a visualization service sitting at the end of processing pipelines, geared towards human consumption.

OGC WCS, on the other hand, provides data suitable for further processing (including visualization); as such, it is suitable also for machine-to-machine communication as it appears in the middle of longer processing pipelines. WCS is a modular suite of service functionality on coverages. WCS Core defines download of coverages and parts thereof, through subsetting directives, as well as delivery in some output format requested by the client. A set of WCS Extensions adds further functionality facets.

One of those is WCS Processing; it defines the ProcessCoverages request which allows sending a coverage analytics request through the WCPS spatio-temporal analytics language. WCPS supports extraction, analytics, and fusion of multi-dimensional coverage expressed in a high-level, declarative, and safe language.

5.2. OGC Web Services Endpoint¶

Once the petascope geo service is deployed (see rasdaman installation guide) coverages can be accessed through the HTTP service endpoint /rasdaman/ows.

For example, assuming that the service IP address is 123.456.789.1 and the service port is 8080, the following request URLs would deliver the Capabilities documents for OGC WMS and WCS, respectively:

http://123.456.789.1:8080/rasdaman/ows?SERVICE=WMS&REQUEST=GetCapabilities&VERSION=1.3.0
http://123.456.789.1:8080/rasdaman/ows?SERVICE=WCS&REQUEST=GetCapabilities&VERSION=2.0.1

Text responses, e.g. for GetCapabilities request, will be compressed for faster download if the client sends the request with HTTP header Accept-Encoding: gzip. This is normally automatically done by the browser, but may need to be explicitly handled when developing clients or from using command-line tools.

5.3. OGC Coverage Implementation Schema (CIS)¶

A coverage consists mainly of:

domain set: provides information about where data sit in space and time. All coordinates expressed there are relative to the coverage’s Coordinate Reference System or Native CRS. Both CRS and its axes, units of measure, etc. are indiciated in the domain set. Petascope currently supports grid topologies whose axes are aligned with the axes of the CRS. Along these axes, grid lines can be spaced regularly or irregularly.

range set: the “pixel payload”, ie: the values (which can be atomic, like in a DEM, or records of values, like in hyperspectral imagery).

range type: the semantics of the range values, given by type information, null values, accuracy, etc.

metadata: a black bag which can contain any data: the coverage will not understand these, but duly transport them along so that the connection between data and metadata is not lost.

Further components include Envelope which gives a rough, simplified overview on the coverage’s location in space and time and CoverageFunction which is unused by any implementation known to us.

5.3.1. Coverage CRS¶

Every coverage, as per OGC CIS, must have exactly one native Coordinate Reference System (CRS), which is given by a URL. Resolving this URL should deliver the CRS definition. The OGC CRS resolver is an example of a public service for resolving CRS URLs; the same service is also bundled in every rasdaman installation, so that it is avialable locally. More details on this topic can be found in the CRS Management chapter.

Sometimes definitions for CRSs are readily available, such as the 2-D WGS84 with code EPSG:4326 in the EPSG registry. In particular spatio-temporal CRSs, however, are not always readily available, at least not in all combinations of spatial and temporal axes. To this end, composition of CRS is supported so that the single Native CRS can be built from “ingredient” CRSs by concatenating them into a composite one. For instance, a time-series of WGS84 images would have the following Native CRS:

http://localhost:8080/def/crs-compound?
      1=http://localhost:8080/def/crs/OGC/0/AnsiDate
     &2=http://localhost:8080/def/crs/EPSG/0/4326

Coordinate tuples in this CRS represent an ordered composition of a temporal coordinate expressed in ISO 8601 syntax, such as 2012-01-01T00:01:20Z, followed by latitude and longitude coordinates, as per EPSG:4326.

The native CRS of a coverage domain set can be determined in severay ways:

in a WCS GetCapabilities response, the wcs:CoverageSummary/ows:BoundingBox@crs attribute;
in a WCS DescribeCoverage response, the srsName attribute in the gml:domainSet element; Furthermore, the axisLabels attribute contains the CRS axis names according to the CRS sequency, and the uomLabels attribute contains the units of measure for each corresponding axis.
in WCPS, the function crsSet(e) returns the CRS of a coverage expression e;

The following graphics illustrates, on the example of an image timeseries, how dimension, CRS, and axis labels affect the domain set in a CIS 1.0 RectifiedGridCoverage.

Note

This handling of coordinates in CIS 1.0 bears some legacy burden from GML; in the GeneralGridCoverage introduced with CIS 1.1 coordinate handling is much simplified.

5.3.2. Range Type¶

Range values can be atomic or (possibly nested) records over atomic values, described by the range type. In rasdaman the following atomic data types are supported; all of these can be combined freely in records of values, such as in hyperspectral images or climate variables.

Table 5.1 Mapping of rasdaman base types to SWE Quantity types¶
rasdaman type	size	Quantity types
`boolean`	8 bit	unsignedByte
`octet`	8 bit	signedByte
`char`	8 bit	unsignedByte
`short`	16 bit	signedShort
`unsigned short` = `ushort`	16 bit	unsignedShort
`long`	32 bit	signedInt
`unsigned long` = `ulong`	32 bit	unsignedInt
`float`	32 bit	float32
`double`	64 bit	float64
`complex`	64 bit	cfloat32
`complexd`	128 bit	cfloat64

5.3.3. Nil Values¶

Nil (null) values, as per SWE, are supported by rasdaman in an extended way:

null values can be defined over any data type

nulls can be single values

nulls can be intervals

a null definnition in a coverage can be a list of all of the above alternatives.

Full details can be found in the null values section.

Note

It is highly recommended to NOT define single null values over floating-point data as this causes numerical problems well known in mathematics. This is not related to rasdaman, but intrinsic to the nature and handling of floating-point numbers in computers. A floating-point interval around the desired float null value should be preferred (this corresponds to interval arithmetics in numerical mathematics).

5.3.4. Errors¶

Errors from OGC requests to rasdaman are returned to the client formatted as ows:ExceptionReport (OGC Common Specification). An ExceptionReport can contain multiple Exception elements. For example, when running a WCS GetCoverage or a WCPS query which execute rasql queries in rasdaman, in case of an error the ExceptionReport will contain two Exception elements:

One with the error message returned from rasdaman.
Another with the rasql query that failed.

For example:

<ows:ExceptionReport>
    <ows:Exception exceptionCode="RasdamanRequestFailed">
        <ows:ExceptionText>The Encode function is applicable to array arguments only.</ows:ExceptionText>
    </ows:Exception>
    <ows:Exception exceptionCode="RasdamanRequestFailed">
        <ows:ExceptionText>Failed internal rasql query: SELECT encode(1, "png" ) FROM mean_summer_airtemp AS c</ows:ExceptionText>
    </ows:Exception>
</ows:ExceptionReport>

5.4. OGC Web Coverage Service¶

WCS Core offers the following request types:

GetCapabilities for obtaining a list of coverages offered together with an overall service description;
DescribeCoverage for obtaining information about a coverage without downloading it;
GetCoverage for downloading, extracting, and reformatting of coverages; this is the central workhorse of WCS.

WCS Extensions in part enhance GetCoverage with additional functionality controlled by further parameters, and in part establish new request types, such as:

WCS-T defining InsertCoverage, DeleteCoverage, and UpdateCoverage requests;
WCS Processing defining ProcessCoverages for submitting WCPS analytics code.

You can use http://localhost:8080/rasdaman/ows as service endpoints to which to send WCS requests, for example:

http://localhost:8080/rasdaman/ows?service=WCS&version=2.0.1&request=GetCapabilities

See example queries in the WCS systemtest which send KVP (key value pairs) GET request and XML POST request to Petascope.

5.4.1. Subsetting behavior¶

This section describes how rasdaman translates geo-referenced coordinates (e.g. in spatio-temporal subsets) to internal grid coordinates, which are required in the internal rasql queries generated for each geo request.

This is particular to regular geo axes of a coverage. The behavior for irregular axes is explained here.

5.4.1.1. Terminology¶

General

geo axis - a geo-referenceable axis of a coverage, e.g. Lat, Lon, time, etc.
grid axis - an axis referenceable with grid (integer) coordinates; internally geo axis coordinates have to be translated to grid integer coordinates
subset - a list of trims and/or slices
trim - a pair of lower and upper bounds indicating a range on a geo axis
slice - a single coordinate on a geo axis

Rasdaman-provided information

axis_res - geo resolution (positive or negative) on a regular axis
axis_min - minimum geo axis bound
axis_max - maximum geo axis bound
axis_grid_min - minimum grid axis bound

User-provided information

subset_min - minimum trim geo bound, or a slice coordinate
subset_max - maximum trim geo bound (none if it is a slicing operation)

5.4.1.2. Preconditions¶

Valid trims on a geo axis must fulfill the following precondition:

axis_min <= subset_min <= subset_max <= axis_max

Valid slices on a geo axis must fulfill the following preconditions:

if axis_res > 0 then axis_min <= subset_min < axis_max
if axis_res < 0 then axis_min < subset_min <= axis_max

Furthermore, for irregular axes:

Slicing: exactly one of the axis coefficients must be equal to the slice coordinate subset_min, or a coefficient’s area of validity (if defined) must intersect it.
Trimming: at least one of the axis coefficients must be within the [subset_min, subset_max] interval, or at least one coefficient’s area of validity (if defined) must intersect it.

5.4.1.3. Geo-to-grid coordinate translation¶

Translating geo coordinates to grid coordinates on a regular axis is done with the formula below. It is assumed that the preconditions in the previous section are already satisfied.

# 1. A small value is extracted from subset_max to emulate right-open interval.
#    It cannot be fixed as it eventually becomes ineffective at certain magnitudes;
#    for that reason it is scaled to the resolution.
epsilon = abs(axis_res) * 1e-11

# 2. Handle Axis Y (e.g. Lat) with negative axis_res = opposite behavior:
#    swap coordinates and limits; this allows to use the same formula in 3a/b
#    below for both positive and negative resolution.
if axis_res < 0:
    epsilon *= -1
    # the origin for negative resolution is the upper subset/axis bound
    swap subset_min and subset_max
    swap axis_min and axis_max

# 3a. Calculate grid low bound
grid_min_tmp = (subset_min - axis_min) / axis_res
if grid_min_tmp != floor(grid_min_tmp) and grid_min_tmp < 0:
   # for axis with negative resolution and grid min is a float number, it needs to be adjusted
   # e.g. -5.00000000000001 should be shifted to -5 instead of -6
   grid_min_tmp = grid_min_tmp + abs(epsilon)

# *floor* is used to make sure that the result is an integer grid coordinate;
# some rounding method must be used, and floor works well with left-closed interval.
# NOTE: axis_grid_min is used because the axis grid lower bound is not always 0.
grid_min =  axis_grid_min + floor( grid_min_tmp )

# 3b. Calculate grid high bound
if subset_min != subset_max:
    # The formula is analogous to the one above for grid_min, but
    # we extract a small epsilon from subset_max to emulate right-open interval.
    grid_max = axis_grid_min + floor( ((subset_max - epsilon) - axis_min) / axis_res )

# 4. Output of the formula
if subset_min != subset_max:
  return (grid_min, grid_max) # trim
else:
  return grid_min             # slice

Example. The following subset performs slicing on the Lat and trimming on the Lon axis of coverage $c in WCPS: $c[Lat(-89.0), Lon(0.0, 180.0)]. The formula values provided by rasdaman are as follows:

Axis Lon: axis_res=1, axis_min=-180, axis_max=180, axis_grid_min=0
Axis Lat: axis_res=-1, axis_min=-90, axis_max=90, axis_grid_min=0

The formula would return the following grid indices:

Lat(-89.0) -> 179
Lon(0.0, 180.0) -> (170, 359)

5.4.2. CIS 1.0 to 1.1 Transformation¶

Under WCS 2.1 - ie: with SERVICE=2.1.0 - both DescribeCoverage and GetCoverage requests understand the proprietary parameter OUTPUTTYPE=GeneralGridCoverage which formats the result as CIS 1.1 GeneralGridCoverage even if it has been imported into the server as a CIS 1.0 coverage, for example:

http://localhost:8080/rasdaman/ows?SERVICE=WCS&VERSION=2.1.0
    &REQUEST=DescribeCoverage
    &COVERAGEID=test_mean_summer_airtemp
    &OUTPUTTYPE=GeneralGridCoverage

http://localhost:8080/rasdaman/ows?SERVICE=WCS&VERSION=2.1.0
    &REQUEST=GetCoverage
    &COVERAGEID=test_mean_summer_airtemp
    &FORMAT=application/gml+xml
    &OUTPUTTYPE=GeneralGridCoverage

5.4.3. Polygon/Raster Clipping¶

WCS and WCPS support clipping of polygons expressed in the WKT format format. Polygons can be MultiPolygon (2D), Polygon (2D) and LineString (1D+). The result is always a 2D coverage in case of MultiPolygon and Polygon, and is a 1D coverage in case of LineString.

Further clipping patterns include curtain and corridor on 3D+ coverages from Polygon (2D) and Linestring (1D). The result of curtain clipping has the same dimensionality as the input coverage whereas the result of corridor clipping is always a 3D coverage, with the first axis being the trackline of the corridor by convention.

In WCS, clipping is expressed by adding a &CLIP= parameter to the request. If the SUBSETTINGCRS parameter is specified then this CRS also applies to the clipping WKT, otherwise it is assumed that the WKT is in the Native coverage CRS. In WCPS, clipping is done with a clip function, much like in rasql.

Further information can be found in the rasql clipping section. Below we list examples illustrating the functionality in WCS and WCPS.

5.4.3.1. Clipping Examples¶

Polygon clipping on coverage with Native CRS EPSG:4326, for example:

WCS:

Table 5.2 WCS-T Standard Parameters¶
Request Parameter	Value	Description	Required
SERVICE	WCS	service standard	Yes
VERSION	2.0.1 or later	WCS version used	Yes
REQUEST	InsertCoverage	Request type to be performed	Yes
INPUTCOVERAGEREF	{url}	URl pointing to the coverage to be inserted	One of inputCoverageRef or inputCoverage is required
INPUTCOVERAGE	{coverage}	A coverage to be inserted	One of inputCoverageRef or inputCoverage is required
USEID	new \| existing	Indicates wheter to use the coverage’s id (“existing”) or to generate a new unique one (“new”)	No (default: existing)

Table 5.3 WCS-T Proprietary Enhancements¶
Request Parameter	Value	Description	Required
PIXELDATATYPE	GDAL supported base data type (eg: “Float32”) or comma-separated concatenated data types, (eg: “Float32,Int32,Float32”)	In cases where range values are given in the GML body the datatype can be indicated through this parameter. Default: Byte.	No
TILING	rasdaman tiling clause, see Storage Layout Language	Indicates the array tiling to be applied during insertion	No

Table 5.4 WCS-T Standard Parameters¶
Request Parameter	Value	Description	Required
SERVICE	WCS	service standard	Yes
VERSION	2.0.1 or later	WCS version used	Yes
REQUEST	UpdateCoverage	Request type to be performed	Yes
COVERAGEID	{string}	Identifier of the coverage to be updated	Yes
INPUTCOVERAGEREF	{url}	URl pointing to the coverage to be inserted	One of inputCoverageRef or inputCoverage is required
INPUTCOVERAGE	{coverage}	A coverage to be updated	One of inputCoverageRef or inputCoverage is required
SUBSET	AxisLabel(geoLowerBound, geoUpperBound)	Trim or slice expression, one per updated coverage dimension	No

Table 5.5 Non-standard domain operations¶
Expression	Result
`imageCrsdomain($c, Long)`	`(0:3599)`
`imageCrsdomain($c, Long).lo`	`0`
`imageCrsdomain($c, Long).hi`	`3599`
`domain($c, Long)`	`(-180:180)`
`domain($c, Long).lo`	`-180`
`domain($c, Long).hi`	`180`

5. Geo Services Guide¶

5.1. OGC Coverage Standards Overview¶

5.1.1. Coverage Data¶

5.1.2. Coverage Services¶

5.2. OGC Web Services Endpoint¶

5.3. OGC Coverage Implementation Schema (CIS)¶

5.3.1. Coverage CRS¶

5.3.2. Range Type¶

5.3.3. Nil Values¶

5.3.4. Errors¶

5.4. OGC Web Coverage Service¶

5.4.1. Subsetting behavior¶

5.4.1.1. Terminology¶

5.4.1.2. Preconditions¶

5.4.1.3. Geo-to-grid coordinate translation¶

5.4.2. CIS 1.0 to 1.1 Transformation¶

5.4.3. Polygon/Raster Clipping¶

5.4.3.1. Clipping Examples¶

5.4.4. Areas of validity on irregular axes¶

5.4.5. WCS-T¶

5.4.5.1. Inserting coverages¶

5.4.5.2. Updating Coverages¶

5.4.5.3. Deleting Coverages¶

5.4.6. Renaming a coverage¶

5.4.7. Update coverage metadata¶

5.4.8. Update coverage’s null values¶

5.4.9. Update coverage range type¶

5.4.10. Convert irregular axis to regular axis¶

5.4.11. Update regular axis’ origin¶

5.4.12. INSPIRE Coverages¶

5.4.12.1. Create an INSPIRE coverage¶

5.4.13. Coverage thumbnail¶

5.4.14. Check if a coverage exists¶

5.4.15. GetCapabilities response extensions¶

5.4.16. DescribeCoverage rasdaman metadata¶

5.4.17. GetCoverage request¶

5.4.17.1. Interpolation¶

5.4.18. CRS notation¶

5.5. OGC Web Coverage Processing Service (WCPS)¶

5.5.1. General¶

5.5.2. Polygon/Raster Clipping¶

5.5.2.1. Clipping Examples¶

5.5.3. Auto-ratio for spatial scaling¶

5.5.4. Non-scaled axes are optional¶

5.5.5. Extensions on domain functions¶

5.5.6. LET clause¶

5.5.7. min and max functions¶

5.5.8. Positional parameters¶

5.5.8.1. Example¶

5.5.9. Decode Operator in WCPS¶

5.5.9.1. Syntax¶

5.5.9.2. Example¶

5.5.10. Case Distinction¶

5.5.10.1. Syntax¶

5.5.10.2. Constraints¶

5.5.10.3. Evaluation Rules¶

5.5.10.4. Examples¶

5.5.11. CIS 1.0 to CIS 1.1 encoding¶

5.5.12. Query Parameter¶

5.5.13. Describe Operator in WCPS¶

5.5.13.1. Syntax¶

5.5.13.2. Semantics¶

5.5.13.3. Examples¶

5.5.13.4. Specific Exceptions¶

5.5.14. Flip Operator in WCPS¶

5.5.15. Sort Operator in WCPS¶

5.5.16. Calendar capabilities¶

5.5.16.1. Temporal coordinates¶

5.5.16.2. Shifting temporal coordinates¶

5.5.16.3. Concatenating time components¶

5.5.16.4. Temporal subsets¶

5.5.16.5. Time axis iterator¶

5.5.16.6. Time truncator functions¶

5.5.16.7. Time extractor functions¶

5.5.16.8. Incompatibilites¶

5.5.17. Polygonize function¶

5.6. OGC Web Map Service (WMS)¶

5.6.1. GetMap extensions¶

5.6.1.1. Transparency and background color¶

5.6.1.2. Interpolation¶