MAP SERVICES

The geospatial raster files are generally voluminous in memory going in GBs or even higher than 4GB (BIGTIFF format). It has always been a challenge to access these files including vector files throught network in a customized manner. Open Geospatial Consortium (OGC) is an international standard organisation originated in 1994 which has standardized more than 30 Map Services so that they can be accessed by a wide variety of client applications through network. Some of these services are as follows:

  • WMS: Web Map Service
  • WFS: Web Feature Service
  • WCS: Web Coverage Service
  • CSW: Catalog Service for the Web
  • WMTS: Web Map Tile Service
  • WTS: Web Terrain Service
We will discuss the WMS here.

WMS (WEB MAP SERVICE)

A Web Map Service (WMS) is a standard protocol for serving (over the network) georeferenced images which a map server generates using data from a GIS database. Here we will learn about Map Server 4 Windows (MS4W) as a WMS Server later in this page.

This service is made available through http (HyperText Transfer Protocol) using get method. Some WMS servers support the post method too. The WMS service fetches image tiles generally in JPG, PNG or GIF format generated from the following parameters received from the client end.

  • SERVICE: Its value is WMS.
  • VERSION: It can be 1.0.0, 1.1.0 or 1.3.0.
  • REQUEST: GetMap, GetCapabilities or GetFeatureInfo.
  • LAYERS: It is decided by the client which layers to display.
  • TRANSPARENT: True or False. Whether we want to make the blank space left by the image to be transparent or opaque (Optional).
  • STYLES: It defines the style in the displayed layers. This can be blank if displaying only raster images (Optional).
  • FORMAT: This can be image/png, image/jpg or image/gif. Generally image/png is used.
  • SRS: It represents the Spatial Reference system of the viewer. Generally its value is EPSG:4326.
  • CRS: It represents the Coordinate Reference System. This is mandatory in version 1.3.0. Generally its value is CRS:84.
  • BBOX: It represents the boundary box of the requested image. Its format is xmin,ymin,xmax,ymax.
  • WIDTH: It is the width of the requested image in pixels. Generally, it value is 256.
  • HEIGHT: It is the height of the requested image in pixels. Generally, it value is 256.
  • DPI: It represents dots per inch. Its value is client-dependent. Generally its value is 96 (Optional).
  • BGCOLOR: It represents the color to be used as the background (non-data) pixels of the map represented in hexadecimal prefixed by 0x (Optional).

In case we use MS4W as our WMS Server, then a mandatory parameter called MAP is also to be added along with the above mentioned parameters. For example a WMS request looks like this

http://localhost/cgi-bin/mapserv.exe?map=F:/image.map&VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&BBOX=90.5,45.5,91.5,46.5&SRS=EPSG:4326&WIDTH=256&HEIGHT=256&LAYERS=World:BlueMarbel&STYLES=&FORMAT=image/png&DPI=96&TRANSPARENT=TRUE

MS4W (MAP SERVER FOR WINDOWS)

MS4W WMS Server is available with Apache Web Server with built-in PHP (Hypertext Preprocessor) support which is easy to handle. You can download the latest MS4W from its official site. Or in short download version 3.2.3 from here.

Follow the below mentioned procedure to install MS4W in your PC.

  • Extract the MS4W zip file and copy it into C drive.
  • Run apache-install.bat from C:\ms4w folder as Administrator.
  • This will install a service called Apache MS4W Web Server. You can check by entering services.msc command in command prompt.
  • Now start the Apache MS4W Web Server service. Sometimes service does not start if port on which Apache starts (80) is pre-occupied.
  • If this shows error then change port number from C:\ms4w\Apache\conf\httpd.conf file around line number 46 after Listen keyword.
  • Try starting the Apache MS4W Web Server service again. Now it should not show any error.

Now you need to install MapServer Map Manager to create Map files from rasters/vectors/tile index shape file. You would have observed in the WMS request example above that there is an additional parameter called map which has not been explained yet. This is the main file which we will create now and learn its various components.

After you install the MapServer Map Manager, you can start it from the shortcut available on Desktop. You will find that this application has the option of adding 6 types of files namely vector, raster, tile, WMS, new layer and layers from Map file. We will be concentrating only on first three types.

You can start the MapServer Map Manager and load a raster layer into it. You can also load multiple raster images into same viewer as shown below.

MapServer Map Manager RasterMapServer Map Manager Rasters
Single raster displayMultiple raster display

Now you save this viewer as a .map file. If you open this .map file into a text editor, you will find similar text as mentioned below. Please note here # is used to comment the text following it.

MAP
  DEFRESOLUTION 96
  EXTENT -16.4569256756757 -45 61.4569256756757 0
  FONTSET "C:\\Program Files (x86)\\MapManager\\templates\\font.list"
  IMAGETYPE "png24"
  NAME "MS"
  RESOLUTION 96
  SHAPEPATH ""
  SIZE 1026 593
  STATUS ON
  SYMBOLSET "C:\\Program Files (x86)\\MapManager\\templates\\symbols.sym"
  UNITS DD

  OUTPUTFORMAT
    NAME "png24"
    MIMETYPE "image/png"
    DRIVER "AGG/PNG"
    EXTENSION "png"
    IMAGEMODE RGB
    TRANSPARENT FALSE
  END # OUTPUTFORMAT

  OUTPUTFORMAT
    NAME "print"
    MIMETYPE "image/png"
    DRIVER "AGG/PNG"
    EXTENSION "png"
    IMAGEMODE RGB
    TRANSPARENT FALSE
  END # OUTPUTFORMAT

  PROJECTION
    "proj=longlat"
    "datum=WGS84"
    "no_defs"
  END # PROJECTION
  LEGEND
    KEYSIZE 20 10
    KEYSPACING 5 5
    LABEL
      SIZE MEDIUM
      OFFSET 0 0
      SHADOWSIZE 1 1
      TYPE BITMAP
    END # LABEL
    STATUS OFF
  END # LEGEND

  QUERYMAP
    SIZE -1 -1
    STATUS OFF
    STYLE HILITE
  END # QUERYMAP

  SCALEBAR
    INTERVALS 4
    LABEL
      SIZE MEDIUM
      OFFSET 0 0
      SHADOWSIZE 1 1
      TYPE BITMAP
    END # LABEL
    SIZE 200 3
    STATUS OFF
    UNITS MILES
  END # SCALEBAR

  WEB
    METADATA
	  wms_enable_request "*" 				# NOT GENERATED BY MAPSERVER MAP MANAGER
	  wms_srs "EPSG:4326 EPSG:4269" 			# NOT GENERATED BY MAPSERVER MAP MANAGER
      "distance_units"	"MS_METERS"
      "mapmanager_version"	"1.0.0.0"
    END # METADATA
  END # WEB

  LAYER
    DATA "D:\\BlueMarbel\\TrueMarble.250m.21600x21600.E3.tif"
    EXTENT 0 -45 45 0
    METADATA
      "coordsys_name"	"Longitude-Latitude / WGS 84"
    END # METADATA
    NAME "TrueMarble.250m.21600x21600.E3"
    PROJECTION
      "proj=longlat"
      "datum=WGS84"
      "no_defs"
    END # PROJECTION
    STATUS ON
    TILEITEM "location"
    TYPE RASTER
    UNITS METERS
  END # LAYER

END # MAP

You can see in the above map file that two extra lines have been added where are not generated by MapServer Manager. The first line wms_enable_request "*" have been added to allow WMS request of any type (GetMap, GetCapabilities or GetFeatureInfo) by using wildcard *. The second line wms_srs "EPSG:4326 EPSG:4269" has been used to advertise the projections in which servers are able to serve data using EPSG projection codes seperated by space.

The second line represents the viewver boundary box coordinates while makeing the map file in MapServer Map Manager. You can even write it as -180,-90,180,90 for all raster data having SRS as EPSG:4326. The 11th line displays UNITS in DD i.e, Degree Decimals. In case the SRS of the raster data is UTM this UNITS will be in METERS. You can see a single LAYER tags in this map file. In case of multiple raster/vector/tile files, there will be multiple LAYER tags. Inside this LAYER tag, NAME is important because it the name of the layer which the WMS URL requires. In case you want to load more than one layer in a WMS request, in layers parameter write name of all layers seperated by comma.

Similarly, you can load the vectors in shapefile format into MapServer Map Manager.

You can even change the point/line/polygon display style from the Label & Style tab. Like rasters, you can load multiple vectors or a combination of rasters and vectors.

Follow the same process as discussed above to generate the map file and add two additional line to enable this map file to be used in WMS request.

Sometimes we want to load a set of all the images kept in a folder/subfolders. For this requirement we make a tile index shapefile as we discussed in GDAL tools. Then this tile index shape file is loaded in MapServer Map Manager as a normal vector file and map file is generated as shown in below image. Please note that bm is a tile index shape file made from 32 raster data. It is not a raster data file.

MapServer Map Manager Vector
MapServer Map Manager Tile

Sometimes you will find that while loading a multispectral image(s), its preview is shown only in signle band cast in which the image may look green/red/blue tinted. In such situation use SCALE=AUTO. Also add RESAMPLE=BILINEAR for avoiding the pixelation of raster during excessive zoom on data.

There may be situations that the images kept in a folder for which a tile index shapefile is to be generated are having different Coordinate Systems. Remember that the map file of one tile index shapefile having multiple images, we have only one PROJECTION tag under LAYER tag. The coordinate system shown in this PROJECTION tag will be one. The raster data that are in this projection system will be visible, rest all will be ignored.

This situation can be handled by creating the tile index shapefile with one more column having EPSG value of each raster data using the following command

gdaltindex -t_srs EPSG:4326 -src_srs_name epsg_no D:/Raster/rastertindex.shp D:/Raster/*.tif

Now you can add AUTO keyword in PROJECTION tag under LAYER tag along with TILESRS indicating the column name having epsg values of each raster data as shown below:

LAYER
.
.
TILESRS "epsg_no"
PROJECTION
      AUTO
END # PROJECTION
.
.
END #LAYER
	

USING MAP FILE IN WMS

Now your MS4W installation and complete and map file is ready in MapServer Map Manager, you are ready to use this map file as a WMS service.

You would recall that we mentioned that while using MS4W apart from all the parameters mentioned in a standard WMS protocol MAP parameter is mandatory in MS4W. We will be using the location of this map file as MAP parameter i.e, F:\MAPFILES\earth.map and LAYER NAME (in the above map file example layer name is TrueMarble.250m.21600x21600.E3) as layers in WMS URL (Universal Resource Locator). The URL will look below if the MS4W is running on port no 80:

http://localhost/cgi-bin/mapserv.exe?map=F:\MAPFILES\earth.map&VERSION=1.1.1&REQUEST=GetMap&SERVICE=WMS&LAYERS=TrueMarble.250m.21600x21600.E3&SRS=EPSG:4326&WIDTH=256&HEIGHT=256&BBOX=20,-40,40,-20&FORMAT=image/png

If you receive an image tile or a tranparent tile, it means your WMS is server is running. If you receive a file named mapserv.exe, then download it and open it in your text Editor (preferably use Notepad++). You can find where you have gone wrong!!!.

SPATIAL DATABASES

A spatial database also known as a geodatabase is a database which has the ability to store and query vector objects such as points, lines and polygons. Some spatial databases can also store the raster data.

Following are some well known databases which have got spatial features:

  • Postgres SQL with PostGIS
  • Oracle Spatial
  • IBM DB2
  • Microsoft SQL Server
  • MySQL
  • SpatiaLite

For complete list of databases use can refer WikiPedia.

Postgres SQL with PostGIS Database is considered to the best because it is free, open-source, customizable as per individual requirements and most of the open-source GIS applications have a built in support for this database.

If we compare the commercial Databases, Oracle Spatial seems better. Some may prefer Microsoft SQL Server depending upon their requirements.

Now we will learn some Oracle Spatial concepts. Basic SQL knowledge is pre-requisite else following details may look Greek & Latin.

You can download Oracle Database from its official site. You need a registration for downloading. This database is free till you use it for self learning purpose. For Commercial use you need to buy a licence.

ORACLE SPATIAL

In Oracle spatial, the geometry columns can have three types of datatypes as mentioned below:

1. SDO_GEOMETRY: SDO stands for Spatial Data Object. This datatype has the ability to store points, lines, curves, polygons, multipoints, mutilines, multicurves, and multipolygons. We will be using only this datatype in our further topics.

2. SDO_PC: PC stands for Point Clouds. This is used for visualization, analysis and Management of 3D LiDAR/Laser scanners Topography where a large number of inputs are received in a short span of time. Oracle Spatial has the ability to store approximately 500,000 points in a second depending the Server configuration. This data cannot be stored in sdo_geometry because of the restrictions on the number of vertices it can have. For two dimentional data, sdo_geometry can save only 524,288 vertices.

3. SDO_TIN: TIN stands for Triangulated Irregular Network. It is a vector-based topological data model used to represent terrain surfaces. TINs are composed of a network of irregularly spaced triangles.

LIDARsdo_pc

Point Clouds

SDO_GEOMETRY

In this secion we will learn to create sdo_geometry column in Oracle Spatial table. We will also see how to save a point and polygon geometry in this table and raise spatial query to get the output.

SDO_GEOMETRY is written as sdo_geometry (SDO_GTYPE, SDO_SRID, SDO_POINT, SDO_ELEM_INFO, SDO_ORDINATES).

SDO_GEOMETRY , as you can see the above, consists of five components which are explained below:

1. SDO_GTYPE: It represents geometry type like point, line, rectangle, circle etc. It is represented in number. For example point in 2D is represented by 2001 and polygon by 2003.

2. SDO_SRID: This is Spatial Reference ID. It is represented by number. For example EPSG:4326 is represented by 8307.

3. SDO_POINT: This represents exclusively a point geometry attributes in the form of SDO_POINT_TYPE(x,y,z) where x represents longitude, y latitude and z altitude. For example a point having latitude 25.5 and longitude 95.5 is written as sdo_point_type(95.5, 25.5, null). If SDO_POINT is not null, then SDO_ELEM_INFO & SDO_ORDINATES have to be null.

4. SDO_ELEM_INFO: It indicates how to interpret the ordinates stored in the SDO_ORDINATES attribute in the form SDO_ELEM_INFO_ARRAY(triplet set of numbers). You will see the usage in the below examples.

5. SDO_ORDINATES: It stores the coordinate values that make up the boundary of a spatial object in the form SDO_ORDINATE_ARRAY(x1,y1,x2,y2,...). It is this SDO_ORDINATES which has the restriction of storing only 524,288 vertices.

For example:

A point having latitude 25.5 and longitude 95.63 can be written as

sdo_geometry(2001, 8307, sdo_point_type(95.63, 25.5, null), null, null).

A polygon having 4 corner coordinates (24.4, 78.2), (24.5, 82.3), (22.3, 82.5), (22.1, 78.3) can be written as

sdo_geometry(2003, 8307, null, sdo_elem_info_array(1, 1003, 1), SDO_ORDINATES(78.2, 24.4, 82.3, 24.5, 82.5, 22.3, 78.3, 22.1, 78.2, 24.4))

GEO-CRAWLER

What is a geo-crawler?

Suppose you have some/many storage locations where you have a collection (in millions) of rasters kept in seperate locations/folders depending on certain common characterstics like countrywise images or all LANDSET data in one folder etc. The question arises how will you organise these data so that you/your clients are able to search all rasters/vectors at a perticular location and preview/load it in your/their browser.

I perticularily mentioned browser because you may have clients at different locations who want to perform the same query and preview the raster before downloading/ordering. Previewing is important rather than first downloading and checking the raster quality because these raster files are quite voluminous in memory and may eat up large network bandwidth. So one of the best options will be through web browsers. You can have desktop applications which can perform the same job through network but the type of query remains the same.

You can visit Geospatial Data Extraction or EarthExplorer to practically see how a geo-crawler can help a client to search a data.

A Geo-Crawler is an application which reads the header information of raster files, extracts geospatial/non-geospatial information and saves it in Spatial Database (Oracle in this case). GDAL has the requisite libraries which can read these header infomations. There are certain applications which can display this raster boundaries on a global map like Oracle Fusion Middleware Mapviewer along with Oracle Maploader and create images preview with the help of MS4W WMS Server.

Geo-Crawler in Oracle Spatial database using Java.

This requires a pre-requisite knowledge of Java. You can use an IDE like Netbeans or Eclipse for it. I will use Netbeans.

This Geo-crawler requires all the Environmental Variables which we discussed in GDAL Setup. So either you can set these in your System Environmental Variables or set it locally when the crawler starts through batch (.bat) file. When you create a Java Application project in Netbeans, you have to add a gdal.jar in its classpath, which you will find in C:\gdal\bin\gdal\java folder. In this folder you will also find four .dll files namely gdalconstjni.dll, gdaljni.dll, ogrjni.dll and osrjni.dll, these files have to be pasted in your Java (JDK/JRE) bin folder which is used by your project or completed application.

Now create a spatial table in Oracle Database as follows:

	--Create Spatial Table
	CREATE TABLE rasterTable (sl_no NUMBER, rasterlocation VARCHAR2(500) UNIQUE, rasterName VARCHAR2(100) NOT NULL,
	geo_polygon sdo_geometry, projection VARCHAR2(50), datum VARCHAR2(50), Band VARCHAR2(50), 
	rasterWidth NUMBER, rasterHeight NUMBER);
	
	--Register Spatial Table in Geom Metadata
	INSERT INTO user_sdo_geom_metadata VALUES ('rasterTable', 'geo_polygon', SDO_DIM_ARRAY 
	(SDO_DIM_ELEMENT('Long', -180, 180, 0.00001), SDO_DIM_ELEMENT('Lat',   -90,  90, 0.00001)), 8307);

	--Create Spatial Index
	CREATE INDEX rasters_SIDX on rasterTable(geo_polygon) indextype is MDSYS.SPATIAL_INDEX;
	

IMPORTANT: Whenever creating a Spatial Table in Oracle Database, the table details like table name and spatial column name has to entered in user_sdo_geom_metadata and indexing is to be created as shown above.

Following are the methods which will be used to create the Geo-Crawler:

The follwing method extract the coordinates of a raster corner pixel.

	static double[] GDALInfoReportCorner(Dataset hDataset, String corner_name, double x, double y)

    {
        double[] coordinates = new double[2];
        double dfGeoX, dfGeoY;
        String pszProjection;
        double[] adfGeoTransform = new double[6];
        CoordinateTransformation hTransform = null;

        hDataset.GetGeoTransform(adfGeoTransform);
        {
            pszProjection = hDataset.GetProjectionRef();

            dfGeoX = adfGeoTransform[0] + adfGeoTransform[1] * x
                + adfGeoTransform[2] * y;
            dfGeoY = adfGeoTransform[3] + adfGeoTransform[4] * x
                + adfGeoTransform[5] * y;
        }

        if (pszProjection != null && pszProjection.length() > 0)
        {
            SpatialReference hProj, hLatLong = null;

            hProj = new SpatialReference(pszProjection);
            if (hProj != null)
                hLatLong = hProj.CloneGeogCS();

            if (hLatLong != null)
            {
                hTransform = new CoordinateTransformation(hProj, hLatLong);
                hLatLong.delete();
            }

            if (hProj != null)
                hProj.delete();
        }

        /* -------------------------------------------------------------------- */
        /*      Transform to latlong and report.                                */
        /* -------------------------------------------------------------------- */
        if (hTransform != null)
        {
            double[] transPoint = new double[3];
            hTransform.TransformPoint(transPoint, dfGeoX, dfGeoY, 0);
            coordinates[0] = transPoint[0];
            coordinates[1] = transPoint[1];
        }

        if (hTransform != null)
            hTransform.delete();

        return coordinates;
    }
	

The following codes will open a raster file to get all four corner and center coordinates:

	double[][] coord = new double[5][2]; //Double array to save all corners/centre coordinates
		
	Dataset hDataset; // Declare a Dataset. Raster file will be read in this form only.
		
	String rasterLocation = "D:\\Rasters\\rsterImage.tif";
		
	gdal.AllRegister(); // It is mandatory to register the GDAL before using it to open a raster file
		
	hDataset = gdal.Open(rasterLocation, gdalconstConstants.GA_ReadOnly); // Open raster in read-only mode
            
	//Corner Coordinates            
        coord[0]= GDALInfoReportCorner(hDataset, "Upper Left ", 0.0, 0.0);
        coord[1]= GDALInfoReportCorner(hDataset, "Lower Left ", 0.0, hDataset.getRasterYSize());
        coord[2]= GDALInfoReportCorner(hDataset, "Lower Right", hDataset.getRasterXSize(), hDataset.getRasterYSize());
        coord[3]= GDALInfoReportCorner(hDataset, "Upper Right", hDataset.getRasterXSize(), 0.0);
        coord[4]= GDALInfoReportCorner(hDataset, "Upper Left ", 0.0, 0.0);
	

The following codes will extract the datum and projection from the raster

	String proj= null;
	String dat = null;
	if (hDataset.GetProjectionRef() != null)
            {
                SpatialReference hSRS;
                String pszProjection;
                pszProjection = hDataset.GetProjectionRef();
                hSRS = new SpatialReference(pszProjection);
                //System.out.println(hSRS.GetAttrValue("AUTHORITY", 1));// this will return EPSG value
                if (hSRS != null && pszProjection.length() != 0)
                {
                    String[] pszPrettyWkt = new String[1];
                    hSRS.ExportToProj4(pszPrettyWkt); // this saves the coordinate system in PROJ4 format
                    proj = (pszPrettyWkt[0].substring(pszPrettyWkt[0].indexOf("+proj=")+6, 
					(pszPrettyWkt[0].substring(pszPrettyWkt[0].indexOf("+proj"))+6).indexOf(" ")));
                    dat = ((pszPrettyWkt[0].substring(pszPrettyWkt[0].indexOf("+datum=")+7)).substring(0, 
					(pszPrettyWkt[0].substring(pszPrettyWkt[0].indexOf("+datum=")+7)).indexOf(" ")));
                }
                else
                    System.out.println(hDataset.GetProjectionRef());
                hSRS.delete();
            }
	

The following code will read the number of bands in a raster and the image size in pixels.

	int band = hDataset.getRasterCount(); // Number of bands
	double rasterWidth = hDataset.getRasterYSize(); // Raster width in pixel
	double rasterHeight = hDataset.getRasterXSize(); // Raster height in pixel
	

When you finish the job a reader a raster geospatial information, you need to delete the Dataset using hDataset.delete(); to flush the memory.

Now you have the required data, you can save it in the form of polygon (using coorner coordinates) as mentioned above into Spatial Database. Now spatial query will solve the problem of searching a raster data at a location among a collection of million raster files as shown below:

	SELECT * FROM rastersTable WHERE SDO_RELATE(geo_polygon, SDO_GEOMETRY(2001, 8307, 
	SDO_POINT_TYPE(95.5,24.5,null), NULL, NULL), 'mask=CONTAINS') = 'TRUE';
	
where geo_polygon is the name of spatial column which stores the raster boundaries and 24.5, 95.5 is the latitude and longitude of the location for which rasters data are searched. Similarily query can also be performed for raster within rectangle, any polygon or circle by changing the MASK and SDO_GEOMETRY values.

For more information, you can download and refer to Oracle Spatial Developers' Guide for Oracle Spatial 11g.

QUICK LINKS
Coordinates

Earth Shape

Ellipsoid & Geoid

Datum

Projection

UTM

Rasters

Vectors
GDAL

GDAL Setup

GDAL Rasters

GDAL Vectors

GDAL Tools
WMS

MS4W

Spatial DB

Oracle Spatial

SDO_GEOMETRY

Geo-Crawler
Web Clients

Openlayers

OL3

Closest Distance Map

Openlayers v2.x

ThreeJS

Terrain Viewer

Bluemarble (250m tiff)

JS Tools

Geographiclib

JS2Shapefile

CSV2Shape


GeoSpatialEarth.in Copyright © All rights reserved.