OGR Output

Author:

Frank Warmerdam

Contact:

warmerdam at pobox.com

Author:

Jeff McKenna

Contact:

jmckenna at gatewaygeomatics.com

Last Updated:

2022-04-25

Introduction

OGR output support was added to MapServer 6.0. It provides an output driver to produce feature style output suitable as a return result from WMS GetFeatureInfo or WFS GetFeature requests. OGR feature output depends on MapServer being built against the GDAL/OGR library. The OGR output driver should be enabled in MapServer 6.0 or newer when INPUT=OGR appears in the version string.

OUTPUTFORMAT Declarations

Details of OGR output formats allowed are controlled by an OUTPUTFORMAT declaration. The declarations define the OGR format driver to be used, creation options specific to that driver, and more general instructions to MapServer on how to package multi-file results and whether to try and build the result on disk or in memory.

Examples:

OUTPUTFORMAT
  NAME "CSV"
  DRIVER "OGR/CSV"
  MIMETYPE "text/csv"
  FORMATOPTION "LCO:GEOMETRY=AS_WKT"
  FORMATOPTION "STORAGE=memory"
  FORMATOPTION "FORM=simple"
  FORMATOPTION "FILENAME=result.csv"
END

OUTPUTFORMAT
  NAME "OGRGML"
  DRIVER "OGR/GML"
  FORMATOPTION "STORAGE=filesystem"
  FORMATOPTION "FORM=multipart"
  FORMATOPTION "FILENAME=result.gml"
END

OUTPUTFORMAT
  NAME "SHAPEZIP"
  DRIVER "OGR/ESRI Shapefile"
  FORMATOPTION "STORAGE=memory"
  FORMATOPTION "FORM=zip"
  FORMATOPTION "FILENAME=result.zip"
END

OUTPUTFORMAT
  NAME "SPATIALITEZIP"
  DRIVER "OGR/SQLITE"
  MIMETYPE "application/zip"
  FORMATOPTION "DSCO:SPATIALITE=YES"
  FORMATOPTION "STORAGE=memory"
  FORMATOPTION "FORM=zip"
  FORMATOPTION "FILENAME=result.db.zip"
END

OUTPUTFORMAT
  NAME "application/json"
  DRIVER "OGR/GEOJSON"
  MIMETYPE "application/json"
  FORMATOPTION "FORM=SIMPLE"
  FORMATOPTION "STORAGE=memory"
END

OUTPUTFORMAT
  NAME "OGRFLATGEOBUF"
  DRIVER "OGR/FlatGeoBuf"
  FORMATOPTION "STORAGE=filesystem"
  FORMATOPTION "FORM=simple"
  FORMATOPTION "FILENAME=result.fgb"
  FORMATOPTION "LCO:VERIFY_BUFFERS=NO"
END

The OGR format driver to be used is determined by the name appearing after "OGR/" in the DRIVER argument. This name should match one of the formats listed as supported for the "-f" argument to ogr2ogr in the ogr2ogr usage message.

The IMAGEMODE for OGR output is FEATURE, but this is implicit and does not need to be explicitly stated for OGR output driver declarations.

The OGR renderer will support the following FORMATOPTION declarations:

DSCO:*

Anything prefixed by DSCO: is used as a dataset creation option with the OGR driver. See the OGR web page for the particular format driver to see layer creation options available.

LCO:*

Anything prefixed by LCO: is used as a layer creation option. See the OGR web page for the particular format driver to see layer creation options available.

FORM=simple/zip/multipart

Indicates whether the result should be a simple single file (simple), a mime multipart attachment (multipart) or a zip file (zip). "zip" is the default.

STORAGE=memory/filesystem/stream

Indicates where the datasource should be stored while being written. "filesystem" is the default.

If "memory" then it will be created in /vsimem/ - but this is only suitable for drivers supporting VSI*L which we can't easily determine automatically.

If "filesystem", then a directory for temporary files (specified using WEB TEMPPATH or MS_TEMPPATH) will be used for writing and reading back the file(s) to stream to the client.

If "stream" then the datasource will be created with a name "/vsistdout" as an attempt to write directly to stdout. Only a few OGR drivers will work properly in this mode (ie. CSV, perhaps kml, gml).

FILENAME=name

Provides a name for the datasource created, default is "result.dat" for single file or multipart form, or "result.zip" for zip form. In the case of zip form, the filename can be of the format basename.ext.zip where basename.ext will be used as the actual OGR datasource name (i.e. generally the filename that will be put inside the zip file).

USE_FEATUREID=true/false

Starting from MapServer v7.0.2. Defaults to false. Include feature ids in the generated output, if the ows_featureid metadata key is set at the layer level. The featureid column to use should be an integer column. Useful if you need to include an "id" attribute to your geojson output. Use with caution as some OGR output drivers may behave strangely when fed with random FIDs.

LAYER Metadata

The OGR output driver utilizes several items from the LAYER level METADATA object. Some of these were originally intended for GML output or are primarily intended to support WFS.

wfs_getfeature_formatlist

(Optional) A comma delimited list of formats supported for WFS GetFeature responses. The OUTPUTFORMAT NAME values should be listed.

"wfs_getfeature_formatlist" "OGRGML,SHAPEZIP,CSV"
wfs_additional_files_in_output

(Optional) A comma delimited list of filenames that should be included as additional files in the response to a WFS GetFeature request for this layer, for output formats whose FORM FORMATOPTION is "zip" or "multipart". This is typically used to add technical information, metadata or licensing information. The filenames can also be directory names, in which case all the files contained in the directory will be taken into account. Relative filenames will be evaluated against the mapfile directory, and the SHAPEPATH value if defined. Filenames can also be URLs (starting with http:// or https://). For that latter capability, note that this requires OGR to be compiled with Curl support.

"wfs_additional_files_in_output" "license.pdf,my_layer_metadata.xml"
gml_include_items

(Optional) A comma delimited list of items to include, or keyword "all". You can enable full exposure by using the keyword "all".

"gml_include_items" "all"

You can specify a list of attributes (fields) for partial exposure, such as:

"gml_include_items" "Name,ID"

The new default behaviour is to expose no attributes at all.

gml_[item name]_alias

(Optional) An alias for an attribute's name. The resulting file will refer to this attribute by the alias. Here is an example:

"gml_province_alias" "prov"
gml_[item name]_type

(Optional) Specifies the type of the attribute. Valid values are Integer|Long|Real|Character|Date|Time|DateTime|Boolean.

Catatan

Long is to be used for 64-bit integers, starting with MapServer 7.0.1. If MapServer is built against GDAL 2.0 or later, Long will be translated as a OGR 64-bit integer. For earlier versions, it will be translated as a OGR double-precision floating point value.

Catatan

Time and DateTime have been added in MapServer 8. And since MapServer 8, Date semantics is a date, without time, whereas in previous versions, it was used indifferently for Date, Time or DateTime.

gml_[item name]_width

(Optional) Specifies the width of the indicated field for formats where this is significant, such as Shapefiles.

gml_[item name]_precision

(Optional) Specifies the precision of the indicated field for formats where this is significant, such as Shapefiles. Precision is the number of decimal places, and is only needed for "Real" fields.

gml_types

(Optional) If this field is "auto" then some input feature drivers (ie. OGR, and native shapefiles) will automatically populate the type, width and precision metadata for the layer based on the source file.

"gml_types"   "auto"
ows/wfs_geomtype

(Optional, metadata shared with WFS server GML output) Set the geometry type of OGR layers created from this MapServer LAYER. One of "Point", "LineString", "Polygon", "MultiPoint", "MultiLineString", "MultiPolygon", "GeometryCollection", "Geometry", or "None". Most are fairly obvious, but "Geometry" can be used to represent a mix of geometry types, and "None" is sometimes suitable for layers without geometry. Note that layers which are a mix of polygon and multipolygon would normally have to be described as "Geometry". To produce 2.5D output append "25D" to the geometry type (ie. "Polygon25D"). Note that Z values are only carried by MapServer if built with USE_POINT_Z_M support.

"ows_geomtype"  "Polygon"

MAP / WEB Metadata

wms_feature_info_mime_type

In order for WMS GetFeatureInfo to allow selection of OGR output formats, the mime type associated with the OUTPUTFORMAT must be listed in this metadata item.

"wms_feature_info_mime_type" "text/csv"
wfs_additional_files_in_output

(Optional) A comma delimited list of filenames that should be included as additional files in the response to a WFS GetFeature request, for any layer of the mapfile, for output formats whose FORM FORMATOPTION is "zip" or "multipart". This is typically used to add technical information, metadata or licensing information. The filenames can also be directory names, in which case all the files contained in the directory will be taken into account. Relative filenames will be evaluated against the mapfile directory, and the SHAPEPATH value if defined. Filenames can also be URLs (starting with http:// or https://). For that latter capability, note that this requires OGR to be compiled with Curl support.

"wfs_additional_files_in_output" "license.pdf"

Geometry Types Supported

In MapServer we have POINT, LINE and POLYGON layers which also allow for features with multiple points, lines or polygons. However, in the OGC Simple Feature geometry model used by OGR a point and multipoint layer are quite distinct. Likewise for a LineString and MultiLineString and Polygon an MultiPolygon layer type.

To work around the mismatches between the MapServer and OGR geometry models, there is a mechanism to specify the geometry type to be used when exporting through OGR. This is the "wfs/ows_geomtype" metadata item on the layer. It may be one of one of "Point", "LineString", "Polygon", "MultiPoint", "MultiLineString", "MultiPolygon", "GeometryCollection", "Geometry", or "None".

If this item is not specified, then "Point", "LineString" or "Polygon" will be used depending on the TYPE of the LAYER. In cases of mixed geometry types (ie. polygons and multipolygons) the geometry type should be set to "Geometry" which means any geometry type.

"ows_geomtype" "Geometry"

In order 2.5D support (geometries with Z coordinates) to be enabled, the "25D" suffix must be add to the value of the "ows_geomtype" metadata item (i.e. "Polygon25D"), and MapServer must be built with USE_POINT_Z_M support.

Catatan

Empty geometries are supported from the MapServer 8.0 release.

Empty or null geometries can also be output through OGR. When using the GeoJSON format these are returned as "geometry": null. The CSV format outputs them as an empty value.

To ensure empty features are not removed by a spatial filter use the following setting in the LAYER METADATA:

"wfs_use_default_extent_for_getfeature" "false"

Attribute Field Definitions

For OGR output it is highly desirable to be able to create the output fields with the appropriate datatype, width and precision to reflect the source feature definition.

It is possible to set the gml_[item]_type, gml_[item]_width and gml_[item]_precision metadata on the layer to provide detailed field definitions:

METADATA
  "gml_ID_type"        "Integer"
  "gml_ID_width"       "8"
  "gml_AREA_type"      "Real"
  "gml_AREA_width"     "15"
  "gml_AREA_precision" "6"
  "gml_NAME_type"      "Character"
  "gml_NAME_width"     "64"
  ...

However, doing this manually is tedious and error prone. For that reason some feature sources (at least OGR, Shapefiles, POSTGIS and ORACLESPATIAL) support a mechanism to automatically populate this information from the source datastore. To accomplish this specify:

"gml_types"            "auto"

If no effort is made to set type, width and precision information for attribute fields, they will all be treated as variable length character fields when writing through OGR.

Return Packaging

One of the challenges returning generalized feature formats is that many such formats consists of multiple files which must be returned in the result. There are three approaches taken to this based on the FORM FORMATOPTION in the OUTPUTFORMAT declaration.

simple

In this case a single result is returned. This is suitable for format drivers that produce a single file. The return result will have the mimetype listed in the OUTPUTFORMAT declaration. Note that if the OGR driver actually returns multiple files, only the primary one (the one with a name matching the filename passed into the OGR CreateDataSource call) will be returned. The return result will have a suggested filename based on the FILENAME FORMATOPTION.

multipart

In this case all the files produced are returned as a multipart mime result. In this case the MIMETYPE of the OUTPUTFORMAT is ignored. All component files are returned with a mime type of "application/binary" and the whole package is "multipart/mixed".

zip

In this case all the files produced are bundled into one .zip file and this zip file is returned with a mimetype of "application/zip". The OUTPUTFORMAT MIMETYPE is ignored.

One caveat with "zip" results is that this option is only available if the GDAL/OGR version is 1.8 or newer (or a 1.8 development later than approximately Oct 15, 2010). Earlier versions of GDAL/OGR lacked the zipping capability needed.

Test Suite Example

The MSAutoTest test suite contains a test case for use of OGR Output from WFS. The mapfile is at:

The comments at the start of the file have a variety of sample requests that can be run against the map, as long as [MAPFILE] is replaced with the mapfile name. They requests should be run against mapserv sitting in the msautotest/wxs directory.