5. Geo Services Guide¶

This guide covers the details of geo data management and retrieval in rasdaman. The rasdaman Array DBMS is domain agnostic; the specific semantics of space and time is provided through a layer on top of rasdaman, historically known as petascope. It offers spatio-temporal access and analytics through APIs based on the OGC data standard Coverage Implementation Schema (CIS) and the OGC service standards Web Map Service (WMS), Web Coverage Service (WCS), and Web Coverage Processing Service (WCPS).

Note

While the name petascope addresses a specific component we frequently use the name rasdaman to refer to the complete system, including petascope.

5.1. OGC Coverage Standards Overview¶

For operating rasdaman geo services as well as for accessing such geo services through these APIs it is important to understand the mechanics of the relevant standards. In particular, the concept of OGC / ISO coverages is important.

In standardization, coverages are used to represent space/time varying phenomena, concretely: regular and irregular grids, point clouds, and general meshes. The coverage standards offer data and service models for dealing with those. In rasdaman, focus is on multi-dimensional gridded (“raster”) coverages.

In rasdaman, the OGC standards WMS, WCS, and WCPS are supported, being reference implementation for WCS. These APIs serve different purposes:

WMS delivers a 2D map as a visual image, suitable for consunmption by humans
WCS delivers n-D data, suitable for further processing and analysis
WCPS performs flexible server-side processing, filtering, analytics, and fusion on coverages.

These coverage data and service concepts are summarized briefly below. Ample material is also available on the Web for familiarization with coverages (best consult in this sequence):

hands-on demos for multi-dimensional coverage services provided by Jacobs University;
a series of webinars and tutorial slides provided by EarthServer;
a range of background information on these standards provided by OGC;
the official standards documents maintained by OGC:

WCS 2.0.1

WCS-T 2.0

WCPS 1.0

WMS 1.3.0

5.1.1. Coverage Data¶

OGC CIS specifies an interoperable, conformance-testable coverage structure independent from any particular format encoding. Encodings are defined in OGC in GML, JSON, RDF, as well as a series of binary formats including GeoTIFF, netCDF, JPEG2000, and GRIB2).

By separating the data definition (CIS) from the service definition (WCS) it is possible for coverages to be served throuigh a variety of APIs, such as WMS, WPS, and SOS. However, WCS and WCPS have coverage-specific functionality making them particularly suitable for flexible coverage acess, analytics, and fusion.

5.1.2. Coverage Services¶

OGC WMS delivers 2D maps generated from styled layers stacked up. As such, WMS is a visualization service sitting at the end of processing pipelines, geared towards human consumption.

OGC WCS, on the other hand, provides data suitable for further processing (including visualization); as such, it is suitable also for machine-to-machine communication as it appears in the middle of longer processing pipelines. WCS is a modular suite of service functionality on coverages. WCS Core defines download of coverages and parts thereof, through subsetting directives, as well as delivery in some output format requested by the client. A set of WCS Extensions adds further functionality facets.

One of those is WCS Processing; it defines the ProcessCoverages request which allows sending a coverage analytics request through the WCPS spatio-temporal analytics language. WCPS supports extraction, analytics, and fusion of multi-dimensional coverage expressed in a high-level, declarative, and safe language.

5.2. OGC Web Services Endpoint¶

Once the petascope geo service is deployed (see rasdaman installation guide) coverages can be accessed through the HTTP service endpoint /rasdaman/ows.

For example, assuming that the service IP address is 123.456.789.1 and the service port is 8080, the following request URLs would deliver the Capabilities documents for OGC WMS and WCS, respectively:

http://123.456.789.1:8080/rasdaman/ows?SERVICE=WMS&REQUEST=GetCapabilities&VERSION=1.3.0
http://123.456.789.1:8080/rasdaman/ows?SERVICE=WCS&REQUEST=GetCapabilities&VERSION=2.0.1

5.3. OGC Coverage Implementation Schema (CIS)¶

A coverage consists mainly of:

domain set: provides information about where data sit in space and time. All coordinates expressed there are relative to the coverage’s Coordinate Reference System or Native CRS. Both CRS and its axes, units of measure, etc. are indiciated in the domain set. Petascope currently supports grid topologies whose axes are aligned with the axes of the CRS. Along these axes, grid lines can be spaced regularly or irregularly.

range set: the “pixel payload”, ie: the values (which can be atomic, like in a DEM, or records of values, like in hyperspectral imagery).

range type: the semantics of the range values, given by type information, null values, accuracy, etc.

metadata: a black bag which can contain any data: the coverage will not understand these, but duly transport them along so that the connection between data and metadata is not lost.

Further components include Envelope which gives a rough, simplified overview on the coverage’s location in space and time and CoverageFunction which is unused by any implementation known to us.

5.3.1. Coverage CRS¶

Every coverage, as per OGC CIS, must have exactly one native Coordinate Reference System (CRS), which is given by a URL. Resolving this URL should deliver the CRS definition. The OGC CRS resolver is an example of a public service for resolving CRS URLs; the same service is also bundled in every rasdaman installation, so that it is avialable locally. More details on this topic can be found in the CRS Management chapter.

Sometimes definitions for CRSs are readily available, such as the 2-D WGS84 with code EPSG:4326 in the EPSG registry. In particular spatio-temporal CRSs, however, are not always readily available, at least not in all combinations of spatial and temporal axes. To this end, composition of CRS is supported so that the single Native CRS can be built from “ingredient” CRSs by concatenating them into a composite one. For instance, a time-series of WGS84 images would have the following Native CRS:

http://localhost:8080/def/crs-compound?
      1=http://localhost:8080/def/crs/OGC/0/AnsiDate
     &2=http://localhost:8080/def/crs/EPSG/0/4326

Coordinate tuples in this CRS represent an ordered composition of a temporal coordinate expressed in ISO 8601 syntax, such as 2012-01-01T00:01:20Z, followed by latitude and longitude coordinates, as per EPSG:4326.

The native CRS of a coverage domain set can be determined in severay ways:

in a WCS GetCapabilities response, the wcs:CoverageSummary/ows:BoundingBox@crs attribute;
in a WCS DescribeCoverage response, the srsName attribute in the gml:domainSet element; Furthermore, the axisLabels attribute contains the CRS axis names according to the CRS sequency, and the uomLabels attribute contains the units of measure for each corresponding axis.
in WCPS, the function crsSet(e) returns the CRS of a coverage expression e;

The following graphics illustrates, on the example of an image timeseries, how dimension, CRS, and axis labels affect the domain set in a CIS 1.0 RectifiedGridCoverage.

Note

This handling of coordinates in CIS 1.0 bears some legacy burden from GML; in the GeneralGridCoverage introduced with CIS 1.1 coordinate handling is much simplified.

5.3.2. Range Type¶

Range values can be atomic or (possibly nested) records over atomic values, described by the range type. In rasdaman the following atomic data types are supported; all of these can be combined freely in records of values, such as in hyperspectral images or climate variables.

Table 5.1 Mapping of rasdaman base types to SWE Quantity types¶
rasdaman type	size	Quantity types
`boolean`	8 bit	unsignedByte
`octet`	8 bit	signedByte
`char`	8 bit	unsignedByte
`short`	16 bit	signedShort
`unsigned short` = `ushort`	16 bit	unsignedShort
`long`	32 bit	signedInt
`unsigned long` = `ulong`	32 bit	unsignedInt
`float`	32 bit	float32
`double`	64 bit	float64
`complex`	64 bit	cfloat32
`complexd`	128 bit	cfloat64

5.3.3. Nil Values¶

Nil (null) values, as per SWE, are supported by rasdaman in an extended way:

null values can be defined over any data type

nulls can be single values

nulls can be intervals

a null definnition in a coverage can be a list of all of the above alternatives.

Full details can be found in the null values section.

Note

It is highly recommended to NOT define single null values over floating-point data as this causes numerical problems well known in mathematics. This is not related to rasdaman, but intrinsic to the nature and handling of floating-point numbers in computers. A floating-point interval around the desired float null value should be preferred (this corresponds to interval arithmetics in numerical mathematics).

5.3.4. Errors¶

Errors from OGC requests to rasdaman are returned to the client formatted as ows:ExceptionReport (OGC Common Specification). An ExceptionReport can contain multiple Exception elements. For example, when running a WCS GetCoverage or a WCPS query which execute rasql queries in rasdaman, in case of an error the ExceptionReport will contain two Exception elements:

One with the error message returned from rasdaman.
Another with the rasql query that failed.

For example:

<ows:ExceptionReport>
    <ows:Exception exceptionCode="RasdamanRequestFailed">
        <ows:ExceptionText>The Encode function is applicable to array arguments only.</ows:ExceptionText>
    </ows:Exception>
    <ows:Exception exceptionCode="RasdamanRequestFailed">
        <ows:ExceptionText>Failed internal rasql query: SELECT encode(1, "png" ) FROM mean_summer_airtemp AS c</ows:ExceptionText>
    </ows:Exception>
</ows:ExceptionReport>

5.4. OGC Web Coverage Service¶

WCS Core offers the following request types:

GetCapabilities for obtaining a list of coverages offered together with an overall service description;
DescribeCoverage for obtaining information about a coverage without downloading it;
GetCoverage for downloading, extracting, and reformatting of coverages; this is the central workhorse of WCS.

WCS Extensions in part enhance GetCoverage with additional functionality controlled by further parameters, and in part establish new request types, such as:

WCS-T defining InsertCoverage, DeleteCoverage, and UpdateCoverage requests;
WCS Processing defining ProcessCoverages for submitting WCPS analytics code.

You can use http://localhost:8080/rasdaman/ows as service endpoints to which to send WCS requests, for example:

http://localhost:8080/rasdaman/ows?service=WCS&version=2.0.1&request=GetCapabilities

See example queries in the WCS systemtest which send KVP (key value pairs) GET request and XML POST request to Petascope.

5.4.1. Subsetting behavior¶

In general, subsetting in petascope behaves similarly to subsetting in gdal, with a couple of deviations necessary for n-D. Specifically, subsetting follows the next rules:

Slicing (geoPoint): the grid slice with index corresponding to the requested slicing geo point is returned. This is computed as follows:
```
gridIndex = floor((geoPoint - minGeoLowerBound) / axisResolution)
```
Trimming (geoLowerBound:geoUpperBound): the lower bound of the grid interval is determined as in the case of slicing. The number of returned grid points follows gdal:
- If axis resolution is positive (e.g. Long axis):
```
gridLowerBound = floor((geoLowerBound - minGeoLowerBound) / axisResolution)
numberOfGridPixels = floor(((geoUpperBound - geoLowerBound) / axisResolution) + 0.5)
gridUpperBound = gridLowerBound + numberOfGridPixels - 1
```
- If axis resolution is negative (e.g. Lat axis):
```
gridLowerBound = floor((geoUpperBound - maxGeoLowerBound) / axisResolution)
numberOfGridPixels = floor((geoLowerBound - geoUpperBound) / axisResolution) + 0.5)
gridUpperBound = gridLowerBound + numberOfGridPixels - 1
```
Note

If a trimming subset is applied on an axis with (geoUpperBound - geoLowerBound) / axisResolution < 0.5, then lower grid bound is translated by the slicing formula and upper grid bound is set to lower grid bound.

For example, a 2D coverage has Long (X) and Lat (Y) axes with CRS EPSG:4326. The resolution for axis Long is 10 and the resolution for axis Lat is -10. The geo bounds of axis Long are [0:180] and the geo bounds of axis Lat are [0:90].

Calculate slicing on Long axis by geo coordinates to grid coordinates:

- Long(0):          returns [0]
- Long(9):          returns [0]
- Long(10):         returns [1]
- Long(15):         returns [1]
- Long(20):         returns [2]
- Long(40):         returns [4]
- Long(49.99999):   returns [4]
- Long(50.0):       returns [5]

Calculate trimming on Long axis by geo coordinates to grid coordinates:

- Long(0:5):         returns [0:0]
- Long(0:10):        returns [0:0]
- Long(0:14.999):    returns [0:0]
- Long(0:15):        returns [0:1]
- Long(0:24.999):    returns [0:1]
- Long(0:25.0):      returns [0:2]
- Long(9,11): returns [0:0]

5.4.2. CIS 1.0 to 1.1 Transformation¶

Under WCS 2.1 - ie: with SERVICE=2.1.0 - both DescribeCoverage and GetCoverage requests understand the proprietary parameter OUTPUTTYPE=GeneralGridCoverage which formats the result as CIS 1.1 GeneralGridCoverage even if it has been imported into the server as a CIS 1.0 coverage, for example:

http://localhost:8080/rasdaman/ows?SERVICE=WCS&VERSION=2.1.0
    &REQUEST=DescribeCoverage
    &COVERAGEID=test_mean_summer_airtemp
    &OUTPUTTYPE=GeneralGridCoverage

http://localhost:8080/rasdaman/ows?SERVICE=WCS&VERSION=2.1.0
    &REQUEST=GetCoverage
    &COVERAGEID=test_mean_summer_airtemp
    &FORMAT=application/gml+xml
    &OUTPUTTYPE=GeneralGridCoverage

5.4.3. Polygon/Raster Clipping¶

WCS and WCPS support clipping of polygons expressed in the WKT format format. Polygons can be MultiPolygon (2D), Polygon (2D) and LineString (1D+). The result is always a 2D coverage in case of MultiPolygon and Polygon, and is a 1D coverage in case of LineString.

Further clipping patterns include curtain and corridor on 3D+ coverages from Polygon (2D) and Linestring (1D). The result of curtain clipping has the same dimensionality as the input coverage whereas the result of corridor clipping is always a 3D coverage, with the first axis being the trackline of the corridor by convention.

In WCS, clipping is expressed by adding a &CLIP= parameter to the request. If the SUBSETTINGCRS parameter is specified then this CRS also applies to the clipping WKT, otherwise it is assumed that the WKT is in the Native coverage CRS. In WCPS, clipping is done with a clip function, much like in rasql.

Further information can be found in the rasql clipping section. Below we list examples illustrating the functionality in WCS and WCPS.

5.4.3.1. Clipping Examples¶

Polygon clipping on coverage with Native CRS EPSG:4326, for example:

WCS:

Table 5.2 WCS-T Standard Parameters¶
Request Parameter	Value	Description	Required
SERVICE	WCS	service standard	Yes
VERSION	2.0.1 or later	WCS version used	Yes
REQUEST	InsertCoverage	Request type to be performed	Yes
INPUTCOVERAGEREF	{url}	URl pointing to the coverage to be inserted	One of inputCoverageRef or inputCoverage is required
INPUTCOVERAGE	{coverage}	A coverage to be inserted	One of inputCoverageRef or inputCoverage is required
USEID	new \| existing	Indicates wheter to use the coverage’s id (“existing”) or to generate a new unique one (“new”)	No (default: existing)

Table 5.3 WCS-T Proprietary Enhancements¶
Request Parameter	Value	Description	Required
PIXELDATATYPE	GDAL supported base data type (eg: “Float32”) or comma-separated concatenated data types, (eg: “Float32,Int32,Float32”)	In cases where range values are given in the GML body the datatype can be indicated through this parameter. Default: Byte.	No
TILING	rasdaman tiling clause, see wiki:Tiling	Indicates the array tiling to be applied during insertion	No

Table 5.4 WCS-T Standard Parameters¶
Request Parameter	Value	Description	Required
SERVICE	WCS	service standard	Yes
VERSION	2.0.1 or later	WCS version used	Yes
REQUEST	UpdateCoverage	Request type to be performed	Yes
COVERAGEID	{string}	Identifier of the coverage to be updated	Yes
INPUTCOVERAGEREF	{url}	URl pointing to the coverage to be inserted	One of inputCoverageRef or inputCoverage is required
INPUTCOVERAGE	{coverage}	A coverage to be updated	One of inputCoverageRef or inputCoverage is required
SUBSET	AxisLabel(geoLowerBound, geoUpperBound)	Trim or slice expression, one per updated coverage dimension	No

Table 5.5 Non-standard domain operations¶
Expression	Result
`imageCrsdomain($c, Long)`	`(0:3599)`
`imageCrsdomain($c, Long).lo`	`0`
`imageCrsdomain($c, Long).hi`	`3599`
`domain($c, Long)`	`(-180:180)`
`domain($c, Long).lo`	`-180`
`domain($c, Long).hi`	`180`

5. Geo Services Guide¶

5.1. OGC Coverage Standards Overview¶

5.1.1. Coverage Data¶

5.1.2. Coverage Services¶

5.2. OGC Web Services Endpoint¶

5.3. OGC Coverage Implementation Schema (CIS)¶

5.3.1. Coverage CRS¶

5.3.2. Range Type¶

5.3.3. Nil Values¶

5.3.4. Errors¶

5.4. OGC Web Coverage Service¶

5.4.1. Subsetting behavior¶

5.4.2. CIS 1.0 to 1.1 Transformation¶

5.4.3. Polygon/Raster Clipping¶

5.4.3.1. Clipping Examples¶

5.4.4. WCS-T¶

5.4.4.1. Inserting coverages¶

5.4.4.2. Updating Coverages¶

5.4.4.3. Deleting Coverages¶

5.4.5. Renaming a coverage¶

5.4.6. Update coverage metadata¶

5.4.7. INSPIRE Coverages¶

5.4.7.1. Create an INSPIRE coverage¶

5.4.8. Check if a coverage exists¶

5.4.9. GetCapabilities response extensions¶

5.5. OGC Web Coverage Processing Service (WCPS)¶

5.5.1. General¶

5.5.2. Polygon/Raster Clipping¶

5.5.2.1. Clipping Examples¶

5.5.3. Auto-ratio for spatial scaling¶

5.5.4. Non-scaled axes are optional¶

5.5.5. Extensions on domain functions¶

5.5.6. LET clause¶

5.5.7. min and max functions¶

5.5.8. Positional parameters¶

5.5.8.1. Example¶

5.5.9. Decode Operator in WCPS¶

5.5.9.1. Syntax¶

5.5.9.2. Example¶

5.5.10. Case Distinction¶

5.5.10.1. Syntax¶

5.5.10.2. Constraints¶

5.5.10.3. Evaluation Rules¶

5.5.10.4. Examples¶

5.5.11. CIS 1.0 to CIS 1.1 encoding¶

5.5.12. Query Parameter¶

5.5.13. Describe Operator in WCPS¶

5.5.13.1. Syntax¶

5.5.13.2. Semantics¶

5.5.13.3. Examples¶

5.5.13.4. Specific Exceptions¶

5.6. OGC Web Map Service (WMS)¶

5.6.1. GetMap extensions¶

5.6.1.1. Transparency¶

5.6.1.2. Interpolation¶

5.6.2. nD Coverages as WMS Layers¶

5.6.3. GetLegendGraphic request¶

5.6.4. Layer Management¶

5.6.5. Style Behavior¶

5.6.6. Style Management¶

5.6.6.1. Style Definition¶

5.6.6.2. Style Removal¶

5.6.6.3. Examples¶

5.6.7. Pyramid Management¶

5.6.8. Testing a WMS Setup¶

5.6.9. Errors and Workarounds¶

5.6.9.1. Cannot load new WMS layer in QGIS¶

5.7. Data Import¶

5.7.1. Introduction¶

5.7.2. Common Options¶

5.7.2.1. config section¶

5.7.2.2. input section¶

5.7.2.3. recipe section¶

5.7.2.3.1. Image pyramids¶

5.7.2.4. hooks section¶

5.7.3. Recipe map_mosaic¶

5.7.4. Recipe time_series_regular¶

5.7.5. Recipe time_series_irregular¶

5.7.6. Recipe general_coverage¶

5.7.6.1. Coverage parameters¶