5. Geo Services Guide¶

This guide covers the details of geo data management and retrieval in rasdaman. The rasdaman Array DBMS is domain agnostic; the specific semantics of space and time is provided through a layer on top of rasdaman, historically known as petascope. It offers spatio-temporal access and analytics through APIs based on the OGC data standard Coverage Implementation Schema (CIS) and the OGC service standards Web Map Service (WMS), Web Coverage Service (WCS), and Web Coverage Processing Service (WCPS).

Note

While the name petascope addresses a specific component we frequently use the name rasdaman to refer to the complete system, including petascope.

5.1. OGC Coverage Standards Overview¶

For operating rasdaman geo services as well as for accessing such geo services through these APIs it is important to understand the mechanics of the relevant standards. In particular, the concept of OGC / ISO coverages is important.

In standardization, coverages are used to represent space/time varying phenomena, concretely: regular and irregular grids, point clouds, and general meshes. The coverage standards offer data and service models for dealing with those. In rasdaman, focus is on multi-dimensional gridded (“raster”) coverages.

In rasdaman, the OGC standards WMS, WCS, and WCPS are supported, being reference implementation for WCS. These APIs serve different purposes:

WMS delivers a 2D map as a visual image, suitable for consunmption by humans
WCS delivers n-D data, suitable for further processing and analysis
WCPS performs flexible server-side processing, filtering, analytics, and fusion on coverages.

These coverage data and service concepts are summarized briefly below; for specific details on coordinate reference system handling see also CRS definition management. Ample material is also available on the Web for familiarization with coverages (best consult in this sequence):

hands-on demos for multi-dimensional coverage services provided by Jacobs University;
a series of webinars and tutorial slides provided by EarthServer;
a range of background information on these standards provided by OGC;
the official standards documents maintained by OGC:

WCS 2.0.1

WCS-T 2.0

WCPS 1.0

WMS 1.3.0

5.1.1. Coverage Data¶

OGC CIS specifies an interoperable, conformance-testable coverage structure independent from any particular format encoding. Encodings are defined in OGC in GML, JSON, RDF, as well as a series of binary formats including GeoTIFF, netCDF, JPEG2000, and GRIB2).

By separating the data definition (CIS) from the service definition (WCS) it is possible for coverages to be served throuigh a variety of APIs, such as WMS, WPS, and SOS. However, WCS and WCPS have coverage-specific functionality making them particularly suitable for flexible coverage acess, analytics, and fusion.

5.1.2. Coverage Services¶

OGC WMS delivers 2D maps generated from styled layers stacked up. As such, WMS is a visualization service sitting at the end of processing pipelines, geared towards human consumption.

OGC WCS, on the other hand, rpovides data suitable for further processing (including visualization); as such, it is suitable also for machine-to-machine communication as it appears in the middle of longer processing pipelines. WCS is a modular suite of service functionality on coverages. WCS Core defines download of coverages and parts thereof, through subsetting directives, as well as delivery in some output format requested by the client. A set of WCS Extensions adds further functionality facets.

One of those is WCS Processing; it defines the ProcessCoverages request which allows sending a coverage analytics request through the WCPS spatio-temporal analytics language. WCPS supports extraction, analytics, and fusion of multi-dimensional coverage expressed in a high-level, declarative, and safe language.

5.2. OGC Web Services Endpoint¶

Once the petascope servlet is deployed (see rasdaman installation guide) coverages can be accessed through service endpoint /rasdaman/ows.

Note

Endpoint /rasdaman/rasql, which by default is also available after deploying rasdaman, does not know about coverages and their services, but only knows domain-agnostic rasql.

For example, assuming that the service’s IP address is 123.456.789.1 and the service port is 8080, the following request URLs would deliver the Capabilities documents for OGC WMS and WCS, respectively:

http://123.456.789.1:8080/rasdaman/ows?SERVICE=WMS&REQUEST=GetCapabilities&VERSION=1.3.0
http://123.456.789.1:8080/rasdaman/ows?SERVICE=WCS&REQUEST=GetCapabilities&VERSION=2.0.1

5.3. OGC Coverage Implementation Schema (CIS)¶

A coverage consists mainly of:

domain set: provides information about where data sit in space and time. All coordinates expressed there are relative to the coverage’s Coordinate Reference System or Native CRS. Both CRS and its axes, units of measure, etc. are indiciated in the domain set. Petascope currently supports grid topologies whose axes are aligned with the axes of the CRS. Along these axes, grid lines can be spaced regularly or irregularly.

range set: the “pixel payload”, ie: the values (which can be atomic, like in a DEM, or records of values, like in hyperspectral imagery).

range type: the semantics of the range values, given by type information, null values, accuracy, etc.

metadata: a black bag which can contain any data: the coverage will not understand these, but duly transport them along so that the connection between data and metadata is not lost.

Further components include Envelope which gives a rough, simplified overview on the coverage’s location in space and time and CoverageFunction which is unused by any implementation known to us.

5.3.1. Coordinate Reference Systems in Coverages¶

Every coverage, as per OGC CIS, must have exactly one Native CRS. Sometimes definitions for such CRSs are readily available, such as with the EPSG registry where 2-D WGS84 is readily available under its code EPSG:4326. In particular spatio-temporal CRSs, however, are not always readily available, at least not in all combinations of spatial and temporal axes. To this end, composition of CRS is supported so that the single Native CRS can be built from “ingredient” CRSs by concatenating these CRSs into a composite one.

For instance, a time-series of WGS84 images would have the following Native CRS:

http://localhost:8080/def/crs-compound?
      1=http://localhost:8080/def/crs/EPSG/0/4326
     &2=http://localhost:8080/def/crs/OGC/0/AnsiDate

Coordinate tuples in this CRS represent an ordered composition of a geospatial CRS with Latitude followed by Longitude, as per EPSG:4326, followed by a temporal coordinate expressed in ISO 8601 syntax, such as: 2012-01-01T00:01:20Z.

Several ways exist for determining the Native CRS of coverage domain set:

in a WCS GetCapabilities response, check the wcs:CoverageSummary/ows:BoundingBox@crs attribute;

in a WCS DescribeCoverage response, check the @srsName attribute in the gml:domainSet;

in WCPS, use function crsSet(e) to determine the CRS of a coverage expression e;

Note

In a coverage also consider the axisLabels attributes giving the axis names as used in the coverage, in proper sequence as per CRS; the uomLabels attribute contains the units of measure for each axis.

The following graphics illustrates, on the example of an image timeseries, how dimension, CRS, and axis labels affect the domain set in a CIS 1.0 RectifiedGridCoverage.

Note

This handling of coordinates in CIS 1.0 bears some legacy burden from GML; in the GeneralGridCoverage introduced with CIS 1.1 coordinate handling is much simplified.

5.3.2. CRS Management¶

the Native CRS of a coverage is given by a URL, as per OGC convention. Resolving this URL should deliver the CRS definition. The OGC CRS resolver is one such service. Its implementation is running SECORE which is part of rasdaman community.

By providing the source code of the OGC resolver it is possible to deploy one’s own resolver under an own URL, such as http://rasdaman.org:8080/def/crs/EPSG/0/27700.

5.3.3. Range Type¶

Range values can be atomic or (possibly nested) records over atomic values, described by the range type. In rasdaman the following atomic data types are supported; all of these can be combined freely in records of values, such as in hyperspectral images or climate variables.

Table 5.1 Mapping of rasdaman base types to SWE Quantity types¶
rasdaman type	size	Quantity types
`boolean`	8 bit	unsignedByte
`octet`	8 bit	signedByte
`char`	8 bit	unsignedByte
`short`	16 bit	signedShort
`unsigned short` = `ushort`	16 bit	unsignedShort
`long`	32 bit	signedInt
`unsigned long` = `ulong`	32 bit	unsignedInt
`float`	32 bit	float32
`double`	64 bit	float64
`complex`	64 bit	cfloat32
`complexd`	128 bit	cfloat64

5.3.4. Nil Values¶

Nil (“null”) values, as per SWE, are supported by rasdaman in an extended way:

null values can be defined over any data type

nulls can be single values

nulls can be intervals

a null definnition in a coverage can be a list of all of the above alternatives.

Note

It is highly recommended to NOT define null values over floating-point numbers as this causes numerical problems well known in mathematics. This is not related to rasdaman, but intrinsic to the nature and handling of floating-point numbers in computers. If really desired, a floating-point interval should be defined around the desired float null value (this corresponds to interval arithmetics in numerical mathematics).

5.4. OGC Web Coverage Service¶

WCS Core offers request types:

GetCapabilities for obtaining a list of coverages offered together with an overall service description;

DescribeCoverage for obtaining information about a coverage without downloading it;

GetCoverage for downloading, extracting, and reformatting of coverages; this is the central workhorse of WCS.

WCS Extensions in part enhance GetCoverage with additional functionality controlled by further parameters, and in part establish new request types, such as:

WCS-T defining InsertCoverage, DeleteCoverage, and UpdateCoverage requests;

WCS Processing defining ProcessCoverages for submitting WCPS analytics code.

You can use http://localhost:8080/rasdaman/ows as service endpoints to which to send WCS requests, for example:

http://localhost:8080/rasdaman/ows?service=WCS&version=2.0.1&request=GetCapabilities

See example queries in the WCS systemtest which send KVP (key value pairs) GET request and XML POST request to Petascope.

5.4.1. Subsetting behavior¶

In general, subsetting in petascope behaves similarly to subsetting in gdal, with a couple of deviations necessary for n-D. Specifically, subsetting follows the next rules:

Slicing (geoPoint): the grid slice with index corresponding to the requested slicing geo point is returned. This is computed as follows:
```
gridIndex = floor((geoPoint - minGeoLowerBound) / axisResolution)
```

Trimming (geoLowerBound:geoUpperBound): the lower bound of the grid interval is determined as in the case of slicing. The number of returned grid points follows gdal:

+ If axis resolution is positive (e.g: ``Long`` axis):
     gridLowerBound = floor((geoLowerBound - minGeoLowerBound) / axisResolution)
     numberOfGridPixels = floor(((geoUpperBound - geoLowerBound) / axisResolution) + 0.5)
     gridUpperBound = gridLowerBound + numberOfGridPixels - 1

+ If axis resolution is negative (e.g: ``Lat`` axis):
     gridLowerBound = floor((geoUpperBound - maxGeoLowerBound) / axisResolution)
     numberOfGridPixels = floor((geoLowerBound - geoUpperBound) / axisResolution) + 0.5)
     gridUpperBound = gridLowerBound + numberOfGridPixels - 1

Note

If a trimming subset on an axis with:

(geoUpperBound - geoLowerBound) / axisResolution < 0.5

then, lower grid bound is translated by the slicing formula and upper grid bound is set to lower grid bound.

For example: a 2D coverage has Long (X) and Lat (Y) axes with CRS EPSG:4326. The resolution for axis Long is: 10 and the resolution for axis Lat is: -10. The geo bounds of axis Long are: [0:180] and the geo bounds of axis Lat are [0:90].

Calculate slicing on Long axis by geo coordinates to grid coordinates:

- Long(0):          returns [0]
- Long(9):          returns [0]
- Long(10):         returns [1]
- Long(15):         returns [1]
- Long(20):         returns [2]
- Long(40):         returns [4]
- Long(49.99999):   returns [4]
- Long(50.0):       returns [5]

Calculate trimming on Long axis by geo coordinates to grid coordinates:

- Long(0:5):         returns [0:0]
- Long(0:10):        returns [0:0]
- Long(0:14.999):    returns [0:0]
- Long(0:15):        returns [0:1]
- Long(0:24.999):    returns [0:1]
- Long(0:25.0):      returns [0:2]
- Long(9,11): returns [0:0]

5.4.2. CIS 1.0 to 1.1 Transformation¶

Under WCS 2.1 - ie: with SERVICE=2.1.0 - both DescribeCoverage``and ``GetCoverage requests understand the proprietary parameter OUTPUTTYPE=GeneralGridCoverage which formats the result as CIS 1.1 GeneralGridCoverage even if it has been imported into the server as a CIS 1.0 coverage, for example:

http://localhost:8080/rasdaman/ows?SERVICE=WCS&VERSION=2.1.0
    &REQUEST=DescribeCoverage
    &COVERAGEID=test_mean_summer_airtemp
    &OUTPUTTYPE=GeneralGridCoverage

http://localhost:8080/rasdaman/ows?SERVICE=WCS&VERSION=2.1.0
    &REQUEST=GetCoverage
    &COVERAGEID=test_mean_summer_airtemp
    &FORMAT=application/gml+xml
    &OUTPUTTYPE=GeneralGridCoverage

5.4.3. Polygon/Raster Clipping¶

WCS and WCPS support clipping of polygons expressed in the WKT format format. Polygons can be MultiPolygon (2D), Polygon (2D) and LineString (1D+). The result is always a 2D coverage in case of MultiPolygon and Polygon, and is a 1D coverage in case of LineString.

Further clipping patterns include curtain and corridor on 3D+ coverages from Polygon (2D) and Linestring (1D). The result of curtain clipping has the same dimensionality as the input coverage whereas the result of corridor clipping is always a 3D coverage, with the first axis being the trackline of the corridor by convention.

Below some examples are presented expaining the mimics for WCS.

Syntactically, clipping is expressed by adding a &CLIP= parameter to the request. If the SUBSETTINGCRS parameter is specified then this CRS also applies to the clipping WKT, otherwise it is assumed that the WKT is in the Native coverage CRS.

5.4.3.1. Clipping Examples¶

Polygon clipping on coverage with Native CRS EPSG:4326, for example:

Table 5.2 WCS-T Standard Parameters¶
Request Parameter	Value	Description	Required
SERVICE	WCS	service standard	Yes
VERSION	2.0.1 or later	WCS version used	Yes
REQUEST	InsertCoverage	Request type to be performed	Yes
INPUTCOVERAGEREF	{url}	URl pointing to the coverage to be inserted	One of inputCoverageRef or inputCoverage is required
INPUTCOVERAGE	{coverage}	A coverage to be inserted	One of inputCoverageRef or inputCoverage is required
USEID	new \| existing	Indicates wheter to use the coverage’s id (“existing”) or to generate a new unique one (“new”)	No (default: existing)

Table 5.3 WCS-T Proprietary Enhancements¶
Request Parameter	Value	Description	Required
PIXELDATATYPE	GDAL supported base data type (eg: “Float32”) or comma-separated concatenated data types, (eg: “Float32,Int32,Float32”)	In cases where range values are given in the GML body the datatype can be indicated through this parameter. Default: Byte.	No
TILING	rasdaman tiling clause, see wiki:Tiling	Indicates the array tiling to be applied during insertion	No

Table 5.4 WCS-T Standard Parameters¶
Request Parameter	Value	Description	Required
SERVICE	WCS	service standard	Yes
VERSION	2.0.1 or later	WCS version used	Yes
REQUEST	UpdateCoverage	Request type to be performed	Yes
COVERAGEID	{string}	Identifier of the coverage to be updated	Yes
INPUTCOVERAGEREF	{url}	URl pointing to the coverage to be inserted	One of inputCoverageRef or inputCoverage is required
INPUTCOVERAGE	{coverage}	A coverage to be updated	One of inputCoverageRef or inputCoverage is required
SUBSET	AxisLabel(geoLowerBound,geoUpperBound)	Trim or slice expression, one per updated coverage dimension	No

Table 5.5 WMS Subset Parameters for Different Axis Types¶
Axis Type	Subset parameter
Time	time=…
Elevation	elevation=…
Other	dim_AxisName=… (e.g dim_pressure=…)

5. Geo Services Guide¶

5.1. OGC Coverage Standards Overview¶

5.1.1. Coverage Data¶

5.1.2. Coverage Services¶

5.2. OGC Web Services Endpoint¶

5.3. OGC Coverage Implementation Schema (CIS)¶

5.3.1. Coordinate Reference Systems in Coverages¶

5.3.2. CRS Management¶

5.3.3. Range Type¶

5.3.4. Nil Values¶

5.4. OGC Web Coverage Service¶

5.4.1. Subsetting behavior¶

5.4.2. CIS 1.0 to 1.1 Transformation¶

5.4.3. Polygon/Raster Clipping¶

5.4.3.1. Clipping Examples¶

5.4.4. WCS-T¶

5.4.4.1. Inserting coverages¶

5.4.4.2. Updating Coverages¶

5.4.4.3. Deleting Coverages¶

5.4.5. Coverage Metadata Update¶

5.5. OGC Web Coverage Processing Service (WCPS)¶

5.5.1. General¶

5.5.2. Polygon/Raster Clipping¶

5.5.2.1. Clipping Examples¶

5.5.3. Auto-ratio for scaling X or Y axis in WCPS¶

5.5.4. Automatic domain extraction¶

5.5.5. LET clause in WCPS¶

5.5.6. Case Distinction¶

5.5.6.1. Syntax¶

5.5.6.2. Constraints¶

5.5.6.3. Evaluation Rules¶

5.5.6.4. Examples¶

5.5.7. CIS 1.0 to CIS 1.1 Transformation¶

5.6. OGC Web Map Service (WMS)¶

5.6.1. WMS GetMap: Special Functionality¶

5.6.1.1. Transparency¶

5.6.1.2. Interpolation¶

5.6.2. 3D+ Coverages as WMS Layers¶

5.6.3. WMS Layer Management¶

5.6.4. WMS Style Management¶

5.6.4.1. Style Definition Variants¶

5.6.4.2. WMS Style Removal¶

5.6.5. Testing a WMS Setup¶

5.6.6. Errors and Workarounds¶

5.6.7. WMS Pyramid Management¶

5.7. Data Import¶

5.7.1. Introduction¶

5.7.2. Recipes¶

5.7.2.1. Common Options¶

5.7.2.2. Mosaic map¶

5.7.2.3. Regular timeseries¶

5.7.2.4. Irregular timeseries¶

5.7.2.5. General coverage¶

5.7.2.5.1. Ingredient Sentences¶

5.7.2.5.2. Parameters¶

5.7.2.5.3. Expressions¶

5.7.2.6. Import from external WCS¶

5.7.2.7. Import Sentinel-1 Data¶

5.7.2.8. Import Sentinel-2 Data¶

5.7.2.9. Image pyramids¶

5.7.2.10. Creating your own recipe¶

5.8. Data export¶

5.9. rasdaman / petascope Geo Service Administration¶

5.9.1. Service Startup and Shutdown¶

5.9.2. petascope Configuration¶

5.9.3. Meta Database Connectivity¶

5.9.3.1. Postgresql (default)¶

5.9.3.2. HSQLDB¶

5.9.3.3. H2¶

5.9.4. petascope Standalone Deployment¶

5.9.5. petascope Serving Static Content¶

5.9.6. Logging¶

5.10. Geo Service Standards Compliance¶