Category Archives: GIS

Configuring Python and GeoDjango for Amazon Web Services Deployment

Quick run down of configuring Geodjango to run on amazon web service’s Elastic Beanstalk (EBS). This assumes you have a separate database instance up and running and that you’ve setup the project to use Elastic Beanstalk (i.e. eb init). Configuration files are broken into the tasks they handle and are read automatically by Elastic Beanstalk.

  1. Create two directories in the root of the project (.ebextensions).
  2. Under .ebextensions add the following files:
    • commands.config – Executes shell commands to deploy static files and migrate database changes. Enables the enterprise linux yum repository so that you can install the GIS libraries through yum (very cool).
    • django.config – default settings for Elastic Beanstalk.
    • environment.config – Setup any environmental variables that you want.
    • packages.config – Tells EBS what packages to install.
  3. commands.config:
    container_commands:
    01collectstatic:
    command: "source /opt/python/run/venv/bin/activate && python manage.py collectstatic --noinput"
    02migrate:
    command: "source /opt/python/run/venv/bin/activate && python manage.py migrate"
    leader_only: true
    03installs:
    command: "sudo yum-config-manager --enable epel"
  4. django.config:
    option_settings:
    aws:elasticbeanstalk:container:python:
    WSGIPath: kyp_backend/wsgi.py
  5. environment.config
    option_settings:
    - option_name: DB_NAME
    value: [db_name]
    - option_name: DB_USERNAME
    value: [username]
    - option_name: DB_PASSWORD
    value: [password]
    - option_name: DB_HOSTNAME
    value: [amazon host name]
    - option_name: DB_PORT
    value: [port]
    - option_name: HOST_ENVIRONMENT
    value: [development/test/production]

  6. packages.config:
    packages:
    yum:
    postgresql94-devel: []
    geos: []
    binutils: []
    gdal-devel: []
    proj: []
    gdal: []

OGR/GDAL Examples

Generate a shapefile from a postgresql table
ogr2ogr -f “ESRI Shapefile” [Shapefile Name].shp PG:”dbname='[DB Name]’ host='[Host Name]’ port='[Port Number]’ user='[User Name]'” [Table Name]

Format Types

  • ESRI Shapefile
  • MapInfo File
  • TIGER
  • S57
  • DGN
  • Memory
  • BNA
  • CSV
  • GML
  • GPX
  • LIBKML
  • KML
  • GeoJSON
  • Interlis 1
  • Interlis 2
  • GMT
  • GPKG
  • SQLite
  • ODBC
  • WAsP
  • MSSQLSpatial
  • PostgreSQL
  • PCIDSK
  • DXF
  • Geoconcept
  • GeoRSS
  • GPSTrackMaker
  • PGDump
  • GPSBabel
  • SOSI
  • GFT
  • GME
  • CouchDB
  • ODS
  • XLSX
  • ElasticSearch

SDE Examples for Oracle

Export from database to SDE export file
sdeexport -l [SOURCE_TABLE],[GEOMETRY FIELD] -f .\[OUTPUT FILE NAME].sdx -i sde:oracle11g -u [USERNAME] -p [PASSWORD]@[TNS NAME]

Export from database to shapefile
sde2shp -o init -l [SOURCE_TABLE],[GEOMETRY FIELD] -f [OUTPUT FILE NAME].shp -t polygonZ -a all -i sde:oracle11g –
u [USER] -p [PASSWORD]@[TNS NAME]

Import to database from SDE export file
sdeimport -o CREATE -t [OUTPUT_TABLE_NAME] -f [INPUT_FILE_NAME] -g RTREE -k [DBTUNE KEYWORD] -i sde:oracle11g -u [USER] -p [PASSWORD]@[TNS NAME]

Delete table from database including SDO/SDE metadata.
sdetable -o delete -t [TABLE_NAME] -u [USER] -p [PASSWORD]@[TNS NAME] -i sde:oracle11g

Drop and Create Spatial Index
DROP INDEX “[SCHEMA]”.”[INDEX_NAME]”;

CREATE INDEX “[SCHEMA]”.”SIDX_[TABLE_NAME]” ON “[SDE]”.”[TABLE_NAME]” (“[SHAPE_FIELD]”)
INDEXTYPE IS “MDSYS”.”SPATIAL_INDEX” ;

Oracle Spatial Examples

Get project information (note table alias required).
select c.[GEOMETRY FIELD].sdo_srid
from [TABLE] c;

Get the geometry type
select c.[GEOMETRY_FIELD].sdo_gtype
from [TABLE_NAME] c;

Interpret results with:
*00 UNKNOWN_GEOMETRY
*01 POINT
*02 LINE or CURVE
*03 POLYGON
*04 COLLECTION
*05 MULTIPOINT
*06 MULTILINE or MULTICURVE
*07 MULTIPOLYGON

Ensure table is registered in Oracle Spatial

select *
from mdsys.sdo_geom_metadata_table;

Get spatial tolerance of table
SELECT sdo_dimname, sdo_tolerance
FROM TABLE(
select DIMINFO
from user_sdo_geom_metadata
WHERE table_name = ‘[TABLE_NAME]’
) ;

Create point geometry from x/y values
select
mdsys.SDO_GEOMETRY(
2001, — point data
[SRID],
mdsys.SDO_point_type([X Ordinate], [Y Ordinate], NULL),
NULL,
NULL )
from dual;

Create Spatial Index
CREATE INDEX “[Index Name]” ON “[Table Name]” (“[Geometry Field]”)
INDEXTYPE IS “MDSYS”.”SPATIAL_INDEX”;

Add Geometry Metadata

Insert into MDSYS.USER_SDO_GEOM_METADATA (TABLE_NAME,COLUMN_NAME,DIMINFO,SRID) values (‘[Table_Name]’,'[Geometry_Field_Name]’,MDSYS.SDO_DIM_ARRAY(MDSYS.SDO_DIM_ELEMENT(NULL,-180,180,0.001),MDSYS.SDO_DIM_ELEMENT(NULL,-90,90,0.001)),4269);

Determine Spatial Relationship Between Geometries
SELECT [ID_Field]
, sdo_geom.relate(
a.[Geometry_Field]
, ‘determine’
, b.[Geometry_Field]
, [Tolerance]
)
FROM
[Table_Name_1] a, [Table_Name_2] b;

Testing Coordinate Transformation in PostGIS

I spend a lot of time thinking about which projection to use for a certain project,. Sometimes picking the perfect projection bites me when things change (such as the area of interest expands). In my recent case I have been avoiding reprojecting my data because I haven’t done this before in PostGIS and I’ve been convinced that this will be a major undertaking. After a minute’s worth of research I realize I may have been dead wrong. In fact, the process seems so easy and quick that I am leery. So here it is, my attempt at convincing myself that reprojecting in PostGIS is an inconsequential task.

First I will create some test data using a table in UTM 17 N meters (EPSG:26917).

PointsToReproject

Let’s prove this worked:

SELECT 'Table_SRID' as INFO,find_srid('public', 'ProjTest', 'geom')
UNION ALL
SELECT st_AsText(geom), st_SRID(geom) FROM "ProjTest";

TableInfoBefore

Check!

Next we’ll do the transformation to US National Atlas Equal Area

ALTER TABLE "ProjTest"
ALTER COLUMN geom TYPE geometry(Point,2163)
USING ST_Transform(geom,2163);

Then check the results:
TableInfoAfter

Check!

Finally let’s display it in QGIS:

First I’ll leave the project projection in QGIS to UTM 17N, looks as expected.
DisplayAfter26917

Just to double check, I’ll change the project projection to 2163. Surprisingly things look crazy. Notice that the offset is not predictable, meaning that it might not be a datum shift.
DisplayAfter2163

Interestingly, projecting “on the fly” to other coordinate systems does not reproduce this offset. So, perhaps it is the Google Physical web service that I am using that is having problems with reprojecting to US National Atlas Equal Area and not the reproduction of the points. To test that, we will reproject one of the coordinates back to UTM from US National Atlas Equal Area using gdal.


gdaltransform -s_srs EPSG:2163 -t_srs EPSG:26917
1480246.71738073 -990001.583001807

Results:
Original = 250230.930254224, 3854029.95272766
GDAL Transformation = 250230.930251404, 3854029.95272553

Looks like the reprojection was successful, just need to figure out the problems with reprojecting the base map.

In summary, reprojecting data in PostGreSql is trivial. The work can be done in place and scripts are very succinct. I am extremely pleased with this implementation, thanks guys!

GeoForst Lite, Adding a New Property – Part 2

Introduction

Last time we downloaded and configured the software and data sources we needed to start adding data to the project. If you missed it, just visit Getting Started with GeoForst Lite.

If your familiar with GIS, this post won’t be worth much, but skimming through will get you familiar with some of the datasets. In this post we are focusing on adding data to the Land_Area table which acts as a container for properties.

Alternately, if you aren’t familiar with GIS, this will give you enough knowledge to add a property to the map. There are links to some QGIS tutorials in the post that are worth visiting.

Helpful Items

There are a few things that your should have or know before you start, if possible. Here are a few that may help you on your project:

  • The ability to find your property on a map.
  • A platt map, or some other map of the boundaries and corners of your property.
  • Intimate knowledge of your land.

Adding a property

The first thing we want to do is add the boundary of your property, so we have an area to work within. Before we do that we need to find the property somehow in the map. This is where the osmSearch plugin will help. To make the osmSearch panel visible go to the View menu View > Panel > osmSearch. You can dock this anywhere you like, or not, your choice.

Adding osmSearch panel.

For this turn on the Google Physical layer that we added through the OpenLayers plugin and type in the city and state of the location into the osmSearch panel and hit Search. The location will then appear in the results field. Clicking on the result will pan you to the location you chose. Unfortunately it will not zoom, so you’ll need to choose an appropriate scale, 1:500,000 is a good place to start.

Note: osmSearch will highlight the feature you searched for in red, which can be annoying once you’ve zoomed in to the location. Dismiss it by clicking the “x” in the right hand corner of the search text box.

Zoom to place

At this point you should be able to find your property and zoom into so that the entire property is just visible. You probably want to turn off the Google Physical layer and turn on the Bing Aerial layer once you start zooming in, you’ll have to feel that transition out for yourself.

zoomedAerial

Adding a Property to the Map

Next will will edit the layer named “Land Areas” to add the property boundary. To do this select the layer, right click on it, and select toggle editing. At this point the layer is ready for editing.

toggleEditing

Click on the add feature button on the editing toolbar and begin delineating your property by digitizing the boundaries. For more information on editing layers in QGIS, visit digitizing an existing layer.

addFeature

Digitize your property boundary and add the attributes as shown below.
Adding Property Attributes

Stop editing, and save your edits. You should have something like this:
Newly added property.

Save the project before you close down QGIS and next post we will add some forest stands.

GeoForst Lite, Getting Started – Part 1

Introduction

GeoForst Lite is a standalone program consisting of three parts, QGIS, a local datastore, and a QGIS map.

Installation

  1. Download QGIS version 2.x and install.
  2. Download GeoForst Lite.
  3. Unzip GeoForst Lite and move it to any directory you choose. Your home or documents directory is a fine place for it.

Adding Plugins

At this point you should have the ability to start the QGIS. There are a couple of ways to make this easy. Let’s step through the process.

Begin by opening the QGIS application, it should open to a blank project. At this point we need to add a couple plugins to make life easy. To do this, select the Plugins menu and select Manage and Install Plugins….

pluginsMenu

Now we want the good stuff, so let’s enable the experimental plugins and we are ready to get our plugins.

experimental

Okay lets get two plugins, the OpenLayers plugin and the osmSearch plugin. OpenLayers will let us display a variety of excellent base layers such as Google Terrain and Bing Imagery. This is really fantastic stuff. The osmSearch plugin lets you find locations by place name, which beats finding a location by pan and zoom alone. Search for those Plugins and install them.

Installing the OpenLayers plugin…
OpenLayers

Finally let’s add the OSM place search plugin…
osmSearch

At this point the setup is complete. Open the GForst_Lite.qgs file that you unzip and placed on your file system and there will be three layers under the External Layers group.

ExternalLayers

Note: The OpenLayers plugin layers need a kick in the pants to scale correctly when first added. Zoom in or pan a bit to force it to redraw. It’s wonky, but a small price to pay for a great addition to QGIS.

Next up

Adding Your Property to GeoForst.