SOS Server¶
- Author:
Jeff McKenna
- Contact:
jmckenna at gatewaygeomatics.com
- Last Updated:
2021-03-27
Introduction¶
SOS (Sensor Observation Service), currently an OGC discussion paper, is part of of the OGC’s SensorWeb Enablement (SWE) group of specifications. These specifications describe how applications and services will be able to access sensors of all types over the Web. Specifically, SOS provides an API for managing deployed sensors and retrieving sensor data.
SOS support is available in MapServer 4.10.0 or more recent. Note that no client tools currently exist in MapServer for SOS; you can try QGIS desktop and its plugins, or try using a JavaScript library such as OpenLayers as a client.
SOS support was implemented in MapServer to the guidelines of MapServer MS RFC 13: Support of Sensor Observation Service in MapServer.
This document assumes that you are already familiar with certain aspects of MapServer:
MapServer application development and setting up .map files.
Relevant Definitions¶
The following is taken from the SOS 1.0.0 specification:
- Observation
An observation is an event with a result which has a value describing some phenomenon.
- Observation Offering
An observation offering is a logical grouping of observations offered by a service that are related in some way.
- Observed Value
A value describing a natural phenomenon, which may use one of a variety of scales including nominal, ordinal, ratio and interval.
- Sensor
An entity capable of observing a phenomenon and returning an observed value. A sensor can be an instrument or a living organism (e.g. a person).
Setting Up an SOS Server Using MapServer¶
Install the Required Software¶
SOS requests are handled by the “mapserv” CGI program. The first step is to check that your mapserv executable includes SOS support. One way to verify this is to use the “-v” command-line switch and look for “SUPPORTS=SOS_SERVER”.
Example 1. On Unix:
$ ./mapserv -v
MapServer version 4.9 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP
OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=FREETYPE SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT
SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=THREADS INPUT=JPEG
INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE DEBUG=MSDEBUG
Example 2. On Windows:
C:\MS4W\Apache\cgi-bin> mapserv -v
MapServer version 4.9 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP
OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=FREETYPE SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT
SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=THREADS INPUT=JPEG
INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE DEBUG=MSDEBUG
Notitie
Windows users can install MS4W which supports SOS server.
If you don’t have SOS support in your MapServer build, then you must compile MapServer with the following in mind:
flag -DUSE_SOS_SVR is required
requires either -DUSE_WMS_SVR or -DUSE_WFS_SVR flags to be enabled
requires libxml2 and proj libraries
requires ICONV support (-DUSE_ICONV) on Windows
For more help with MapServer compilation see the appropriate HowTo: Unix / Windows
Configure a Mapfile For SOS¶
Each instance of SOS server that you setup needs to have its own mapfile. It is just a regular MapServer mapfile in which some parameters and some metadata entries are mandatory. Most of the metadata is required in order to produce a valid GetCapabilities output.
Here is the list of parameters and metadata items that usually optional with MapServer, but are required (or strongly recommended) for a SOS configuration:
MAP level:
Map NAME
Map PROJECTION
Map Metadata (in the WEB Object):
sos_title
sos_onlineresource
sos_srs
sos_enable_request
see the Reference Section of this document for a full list of metadata and descriptions
LAYER level:
Layer NAME
Layer PROJECTION
Layer METADATA
sos_offering_id
sos_observedproperty_id
sos_observedproperty_id
sos_describesensor_url
see the Reference Section of this document for a full list of metadata and descriptions
Onlineresource URL¶
The sos_onlineresource metadata is set in the map’s web object metadata and specifies the URL that should be used to access your server. This is required for the GetCapabilities output. If sos_onlineresource is not provided then MapServer will try to provide a default one using the script name and hostname, but you shouldn’t count on that too much. It is strongly recommended that you provide the sos_onlineresource metadata.
Here is a valid online resource URL:
http://my.host.com/cgi-bin/mapserv?map=mysos.map&
By creating a wrapper script on the server it is possible to hide the “map=” parameter from the URL and then your server’s online resource URL could be something like:
http://my.host.com/cgi-bin/mapserv?
This is covered in more detail in the “More About the Online Resource URL” section of the WMS Server document.
Notitie
It is strongly recommended to review the security steps for the MAP= call to the MapServer executable, by setting MS_MAP_PATTERN or MS_MAP_NO_PATH or hiding the MAP= parameter on public servers, as recommended in the document Limit Mapfile Access. All possible environment variables to secure your server are listed in Environment Variables.
Example SOS Server Mapfile¶
The following is an example of a bare minimum SOS Server mapfile. Note the comments for the required parameters.
MAP
NAME "SOS_DEMO"
STATUS ON
SIZE 300 300
EXTENT -66 44 -62 45
UNITS METERS
SHAPEPATH "./data/"
IMAGECOLOR 255 255 0
SYMBOLSET "./etc/symbols.sym"
IMAGETYPE png
WEB
IMAGEPATH "/ms4w/tmp/ms_tmp/"
IMAGEURL "/ms_tmp/"
METADATA
"sos_onlineresource" "http://127.0.0.1/mapserv?map=/sos/sos_test.map" ## REQUIRED
"sos_title" "My SOS Demo Server" ## Recommended
"sos_srs" "EPSG:4326" ## REQUIRED
"sos_enable_request" "*" ## REQUIRED
END
END
PROJECTION
"init=epsg:4326"
END
LAYER
NAME "test_sos_layer"
METADATA
"sos_procedure" "NS01EE0014" ## REQUIRED
"sos_offering_id" "WQ1289" ## REQUIRED
"sos_observedproperty_id" "Water Quality" ## REQUIRED
"sos_describesensor_url" "http://some/url/NS01EE0014.xml" ## REQUIRED
END
TYPE POINT
STATUS ON
DATA "sos_test"
PROJECTION
"init=epsg:4326"
END
CLASS
NAME "water quality"
STYLE
COLOR 255 0 0
SYMBOL "circle"
SIZE 8
END
END
END
END #map
Test Your SOS Server¶
GetCapabilities Request¶
The GetCapabilities request allows the clients to retrieve service metadata about a specific service instance. For an SOS service, it allows to identify such things as offerings and observed property available, as well as information on sensors that are used.
Using a web browser, access your server’s online resource URL to which you add the parameters “SERVICE=SOS&REQUEST=GetCapabilities” to the end, e.g.
http://my.host.com/cgi-bin/mapserv?MAP=mysos.map&SERVICE=SOS&REQUEST=GetCapabilities
If everything went well, you should have a complete XML capabilities document. Search it for the word “WARNING”… MapServer inserts XML comments starting with “<!–WARNING: “ in the XML output if it detects missing mapfile parameters or metadata items. If you notice any warning in your XML output then you have to fix all of them before you can try your server with an SOS client, otherwise things are likely not going to work.
Notitie
The SERVICE parameter is required for all SOS requests.
GetObservation Request¶
The GetObservation request is designed to query sensor systems to retrieve observation data in the form defined in the Observation and Measurement specification (O&M), and more information on this O&M spec can be found at https://www.ogc.org/standards/om. Upon receiving a GetObservation request, a SOS shall either satisfy the request or return an exception report.
The following is a list of the possible parameters for a GetObservation request:
request: (Required) value must be “GetObservation”.
service: (Required) value must be “SOS”.
version: (Required) value must be “1.0.0”.
offering: (Required) The Offering identified in the capabilities document.
- observedProperty: (Required) The property identified in the
capabilities document.
- responseFormat: (Required) The format / encoding to be returned
by the response.
- eventTime (Optional) Specifies the time period for which
observations are requested.
- procedure: (Optional) The procedure specifies the sensor system
used. In this implementation, the procedure is equivalent to be the sensor id that will be used when doing a DescribeSensor request.
- featureOfInterest: (Optional) In this implementation, this will
be represented by a gml envelope defining the lower and upper corners.
- Result: (Optional) The Result parameter provides a place to put
OGC filter expressions based on property values.
- resultModel: (Optional) Identifier of the result model to be
used for the requested data. The resultModel values supported by a SOS server are listed in the contents section of the service metadata (GetCapabilities). MapServer currently supports om:Observation and om:Measurement. om:Measurement provides a flat model of the geometry and attributes, similar to WFS GetFeature output. om:Observations provides a more compact definition which includes an XML header of the field names and definitions, followed by a “DataBlock” of delimited records (default is CSV delimited output). The default output is om:Measurement.
srsName: (Optional) srs (EPSG code) of the output response.
Here are some valid examples:
Example 1:
http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=GetObservation&service=SOS&Offering=WQ1289&
observedproperty=Water Quality&version=1.0.0&
responseFormat=text/xml; subtype="om/1.0.0"
Example 2:
http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=GetObservation&service=SOS&Offering=WQ1289&
observedproperty=Water Quality&eventtime=<ogc:TM_Equals><gml:TimePeriod>
<gml:beginPosition>1991-05-01</gml:beginPosition><gml:endPosition>
1993-02-02</gml:endPosition></gml:TimePeriod></ogc:TM_Equals>
&result=<Filter><Or><PropertyIsEqualTo><PropertyName>COLOUR
</PropertyName><Literal>180</Literal></PropertyIsEqualTo>
<PropertyIsEqualTo><PropertyName>COLOUR</PropertyName><Literal>200
</Literal></PropertyIsEqualTo></or></Filter>&version=1.0.0
&responseFormat=text/xml; subtype="om/1.0.0"
Example 3:
http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=GetObservation&service=SOS&Offering=WQ1289&
observedproperty=Water Quality&featureofinterest=<gml:Envelope>
<gml:lowerCorner srsName='EPSG:4326'>-66 43</gml:lowerCorner>
<gml:upperCorner srsName='EPSG:4326'>-64 45</gml:upperCorner>
</gml:Envelope>&version=1.0.0&
responseFormat=text/xml; subtype="om/1.0.0"
Example 4:
http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=GetObservation&service=SOS&Offering=WQ1289&
observedproperty=Water Quality&version=1.0.0&
responseFormat=text/xml; subtype="om/1.0.0"&resultModel=om:Observation
DescribeSensor Request¶
The DescribeSensor request gives the client the ability to retrieve the characteristics of a particular sensor and return the information in a SensorML xml document. In this implementation, MapServer does not generate the SensorML document but only redirect the request to an existing SensorML document.
The following is a list of the possible parameters for a DescribeSensor request:
request: (Required) value must be “DescribeSensor”
service: (Required) value must be “SOS”.
version: (Required) value must be “1.0.0”.
- procedure: (Required) This is the sensor id, which was specified
in the “sos_procedure” metadata.
- outputFormat: (Required) The format encoding to be returned by
the response.
Here is a valid example:
http://127.0.0.1/cgi-bin/mapserv.exe?map=D:/ms4w/apps/sos/sos_test.map&
Request=DescribeSensor&procedure=urn:ogc:def:procedure:NS01EE0014&
service=SOS&version=1.0.0&outputFormat=text/xml; subtype="sensorML/1.0.0"
Limitations / TODO¶
1. Have MapServer generate the SensorML document, instead of redirecting the request to an existing SensorML document.
Reference Section¶
The following metadata are available in the setup of the SOS Server mapfile:
Notitie
Each of the metadata below can also be referred to as ‘ows_*’ instead of ‘sos_*’. MapServer tries the ‘sos_*’ metadata first, and if not found it tries the corresponding ‘ows_*’ name. Using this reduces the amount of duplication in mapfiles that support multiple OGC interfaces since “ows_*” metadata can be used almost everywhere for common metadata items shared by multiple OGC interfaces.
Web Object Metadata¶
ows_allowed_ip_list (or sos_allowed_ip_list)
Description: (Optional) A list of IP addresses that will be allowed access to the service.
Example:
METADATA "ows_allowed_ip_list" "123.45.67.89 11.22.33.44" END
ows_denied_ip_list (or sos_denied_ip_list)
Description: (Optional) A list of IP addresses that will be denied access to the service.
Example:
METADATA "ows_denied_ip_list" "123.45.67.89 11.22.33.44" END
ows_language
Description: (Optional) Descriptive narrative for more information about the server. Identifier of the language used by all included exception text values. These language identifiers shall be as specified in IETF RFC 1766. When this attribute is omitted, the language used is not identified. Examples: “en-CA”, “fr-CA”, “en-US”. Default is “en-US”.
ows_schemas_location
Description: (Optional) (Note the name ows_schemas_location and not sos/_… this is because all OGC Web Services (OWS) use the same metadata) Root of the web tree where the family of OGC SOS XMLSchema files are located. This must be a valid URL where the actual .xsd files are located if you want your SOS output to validate in a validating XML parser. Default is http://schemas.opengis.net/sos/.
ows_updatesequence
Description: (Optional) The updateSequence parameter can be used for maintaining the consistency of a client cache of the contents of a service metadata document. The parameter value can be an integer, a timestamp in [ISO 8601:2000] format, or any other number or string.
sos_abstract
Description: (Optional) Descriptive narrative for more information about the server.
sos_accessconstraints
Description: (Optional) Text describing any access constraints imposed by the service provider on the SOS or data retrieved from this service.
sos_addresstype, sos_address, sos_city, sos_country, sos_postcode, sos_stateorprovince
Description: Optional contact address information. If provided then all six metadata items are required.
sos_contactelectronicmailaddress
Description: Optional contact Email address.
sos_contactfacsimiletelephone
Description: Optional contact facsimile telephone number.
sos_contactinstructions
Description: (Optional) Supplemental instructions on how or when to contact the individual or organization.
sos_contactorganization, sos_contactperson, sos_contactposition
Description: Optional contact information. If provided then all three metadata items are required.
sos_contactvoicetelephone
Description: Optional contact voice telephone number.
sos_enable_request (or ows_enable_request)
Description: Space separated list of requests to enable. The default is none. The following requests can be enabled: GetCapabilities, GetObservation and DescribeSensor. A “!” in front of a request will disable the request. “*” enables all requests.
Examples:
To enable only GetCapabilities and GetObservation:
"sos_enable_request" "GetCapabilities GetObservation"
To enable all requests except GetCapabilities
"sos_enable_request" "* !GetCapabilities"
sos_encoding_blockSeparator
Description: (Optional) For GetObservation requests using resultModel=om:Observation (SWE DataBlock encoding). Record separator to be used. Default is ‘\n’
sos_encoding_tokenSeparator
Description: (Optional) For GetObservation requests using resultModel=om:Observation (SWE DataBlock encoding). Token (field) separator to be used. Default is ‘,’
sos_fees
Description: (Optional) Fees information. Use the reserved word “none” if there are no fees.
sos_hoursofservice
Description: (Optional) Time period (including time zone) when individuals can contact the organization or individual.
sos_keywordlist
Description: (Optional) A comma-separated list of keywords or keyword phrases to help catalog searching.
sos_maxfeatures
Description: (Optional) The number of elements to be returned by the SOS server. If the not set all observations are returned
sos_onlineresource
Description: (Required) The URL that will be used to access this OGC server. This value is used in the GetCapabilities response.
See the section “Onlineresource URL” above for more information.
sos_role
Description: (Optional) Function performed by the responsible party. Possible values of this Role shall include the values and the meanings listed in Subclause B.5.5 of ISO 19115:2003.
sos_service_onlineresource
Description: (Optional) Top-level onlineresource URL.
sos_srs
Description: (Required) Contains a list of EPSG projection codes that should be advertized as being available for all layers in this server. The value can contain one or more EPSG:<code> pairs separated by spaces (e.g. “EPSG:4269 EPSG:4326”) This value should be upper case (EPSG:3978…..not epsg:3978) to avoid problems with case sensitive platforms.
sos_title
Description: (Recommended) A human-readable name for this Layer.
Layer Object Metadata¶
- ows_allowed_ip_list
Same as ows_allowed_ip_list in the Web Object.
- ows_denied_ip_list
Same as ows_denied_ip_list in the Web Object.
sos_describesensor_url
Description: (Required) This metadata item is only a temporary measure until the describe sensor is generated from MapServer. Right now when a DescribeSensor request is sent with a procedure (sensorid), it will redirect it to the url defined by this metadata item.
In MapServer 5.0, it is possible to use variable substitution on the url. For example “sos_describesensor_url” "http://foo/foo?mysensor=%procedure%"will substitute the %procedure% in the metadata with the procedure value coming from the request.
"sos_describesensor_url" "http://some/url/NS01EE0014.xml"
sos_enable_request (or ows_enable_request)
Description: Space separated list of requests to enable. The default is none. The following requests can be enabled: GetCapabilities, GetObservation and DescribeSensor. A “!” in front of a request will disable the request. “*” enables all requests.
Examples:
To enable only GetCapabilities and GetObservation:
"sos_enable_request" "GetCapabilities GetObservation"
To enable all requests except GetCapabilities
"sos_enable_request" "* !GetCapabilities"
sos_[item name]_alias
Description: (Optional) An alias for an attribute’s name that will be returned when executing a GetObservation request.
sos_[item name]_definition
Description: (Optional) An associated definition (usually a URN) for a component, that will be returned when executing a GetObservation request. Default is "urn:ogc:object:definition"
sos_[item name]_uom
Description: (Optional) An associated unit of measure URN) for a component, that will be returned when executing a GetObservation request. Default is "urn:ogc:object:uom"
sos_observedproperty_authority
Description: (Optional) An associated authority for a given component of an observed property
sos_observedproperty_id
Description: (Required) ID of observed property, possibly in number format.
sos_observedproperty_name
Description: (Optional) Name of observed property, possibly in string format.
sos_observedproperty_version
Description: (Optional) An associated version for a given component of an observed property
sos_offering_description
Description: (Optional) Description of offering.
sos_offering_extent
Description: (Optional) Spatial extents of offering, in minx, miny, maxx, maxy format:
"sos_offering_extent" "-66, 43, -62, 45"
The logic for the bounding box returned as part of the offering is the following:
note that it is a mandatory element that needs an espg code and lower/upper corner coordinates
looks for the espg parameter in the first layer of the offering (this could be an ows/sos_srs or a projection object with the epsg code (mandatory)
looks for sos_offering_extent. If the metadata is not available, the extents of all layers in the offering will be used to compute it.
Here is an example result from a GetCapabilities request:
<gml:boundedBy> <gml:Envelope> <gml:lowerCorner srsName="EPSG:4326">-66 43</gml:lowerCorner> <gml:upperCorner srsName="EPSG:4326">-62 45</gml:upperCorner> </gml:Envelope> </gml:boundedBy>
sos_offering_id
Description: (Required) ID of offering, possibly in number format.
sos_offering_intendedapplication
Description: (Optional) The intended category of use for this offering.
sos_offering_name
Description: (Optional) Name of offering, possibly in string format.
sos_offering_timeextent
Description: (Optional) Time extent of offering, in the format of “begin/end”. Here is an example:
"sos_offering_timeextent" "1990/2006"
If end is not specified it will be set to now. Here is an example result from a GetCapabilities request:
<sos:eventTime> <gml:TimePeriod> <gml:beginPosition>1990</gml:beginPosition> <gml:endPosition>2006</gml:endPosition> </gml:TimePeriod> </sos:eventTime>
sos_procedure
Description: (Required) Normally a sensor unique id. One per layer:
"sos_procedure" "NS01EE0014"
Notitie
sos_procedure can also be a list, separated by spaces, i.e.:
"sos_procedure" "35 2147 604"
All sos_procedure links from layers in the offerings will be outputed together, such as the following taken from a GetCapabilities response:
<procedure xlink:href="urn:ogc:object:feature:Sensor:3eTI:csi-sensor-1"/> <procedure xlink:href="urn:ogc:object:feature:Sensor:3eTI:csi-sensor-2"/>
sos_procedure_item
Description: (Required if sos_procedure is not present): See section 5 for more details
"sos_procedure_item" "attribute_field_name"
sos_timeitem
Description: (Optional) Name of the time field. It will be used for queries when a GetObservation request is called with an EVENTTIME parameter. It is layer specific and should be set on all layers.
"sos_timeitem" "TIME"
Layer Metadata API¶
If sos_metadataurl_href is not defined, MapServer will provide a link to the Layer Metadata API for the given layer. See the Layer Metadata API documentation for more information.
Use of sos_procedure and sos_procedure_item¶
In MapServer 5.0 SOS support has been upgraded to use a new metadata called sos_procedure_item. The value for sos_procedure_item is the field/attribute name containing the procedure values. The use of this metadata as well as the sos_procedure is described here per type of request (refer to https://github.com/MapServer/MapServer/issues/2050 for more description):
It should be noted that, for very large datasets defined only with sos_procedure_item, this may result in costly processing, because MapServer has to process attribute data. It is advised to setup and manage datasets accordingly if dealing with large observation collections.
GetCapabilities¶
if sos_procedure is defined, use it
if not look for sos_procedure_item : procedure values are extracted from the layer’s attribute specified by this metadata. Not that this can be time consuming for layers with a large number of features.
if none is defined return an exception
DescribeSensor¶
if sos_procedure is defined, use it
if not look for sos_procedure_item : procedure values are extracted from the layer’s attribute specified by this metadata
if none is defined return an exception
GetObservation¶
Both sos_procedure and sos_procedure_item can be define. Here are the cases:
- case 1only sos_procedure is defined.
Use this metadata to match the layer with the procedure value sent in the request
When outputting the <member/procedure> output the value of the metadata
Notitie
If more than one procedure is defined per LAYER object, output observations will have incorrect sos:procedure values, because there is no way to map procedures to observations. This is where sos_procedure_item should be used (i.e. when more than one procedure makes up a LAYER object).
- case 2: only procedure_item is defined.
Use the sos_procedure_item and do a query on the layer to match the procedure with the layer.
When outputting the <member/procedure> use the procedure_item as a way to only output the attribute value corresponding to the feature.
- case 3: both are defined.
check in sos_procedure to match the procedure with the layer.
When outputting the <member/procedure> use the procedure_item as a way to only output the attribute value corresponding to the feature.