Wednesday, September 28, 2005

Closing some loose bugs

Looking for PPC packages for review I hit a few road blocks and a good find. Fugawi has a demo available but it doesn't include the PDA version in it. Teletype was recently acquired and also doesn't provide trial versions and finally TomTom Navigator only offers upgrades for existing customers in its website.

UTM based PocketQuads from ProSurv offers 30-day trials and targets surveyors. Its companion Windows PC application is Prosurv Imaging. If you are interested in visiting Yellowstone & Grant Teton there is a special edition for these parks with corresponding topo maps.

These packages support USGS DRG's and let you import NGS Datasheets for their benchmarks data. You need to use the Windows version (Imaging) to break down a large .tiff image file into smaller .jpeg images before transferring them to the PocketPC.

Prosurv Imaging can load .tiff files (NAD 27 or 83 from USGS) with corresponding .tfw in the same directory. The idea is that you have a job and its corresponding image data set to their respective UTM zones. There is no default project, you need to explicitly create one. I tried my best to follow the directions, converted NAD coordinates to UTM zones using Jeeep.com and after several tries I finally got a map to display in the PocketPC but it wasn't calibrated to the right area.

What I don't get is why do I need to provide extra information if a .tfw (which is supposed to provide enough georeference) is already being used. In any case these packages seen to expect users that know what they are doing and when you try "unexpected" actions you will run into one too many bugs particularly with the PPC version. So I gave up trying to make the georeference images work. Plus both packages could make use of a less colorful and a bit more standard user interface.

The GPS parsing (which seemed amazingly fast unless it is only 2D) is provided by Frason Software the same developer of quite interesting GPS and map related packages like a Coordinates conversion tool and GPS Gate, a package that allows you to share the same COM port used by your GPS card to more than one application through the use of virtual COM ports (trial version available for download). It is worth a click.

The one find was ArcPad from ESRI, a company that is almost synonymous with Geographical Information Systems (GIS). I'm still waiting for the Evaluation CD (ArcGis Desktop) to create my own maps so I will hold a bit on a review of this package. Plus GIS opens a whole new chapter in this area and deserves some good research and care.

Saturday, September 17, 2005

Google Maps takes coordinates

I just tried entering latitude and longitude coordinates at Google Maps or something like "37 57 11.98 N 121 05 15.76 W" and sure enough there was a zoomed image of the location. The more precision you provide, the more zoom you get.

In fact, just found from a FAQ that is probably based in the Google Maps API that you can also use coordinates in a URL like for example:

http://maps.google.com/maps?q=37.57+-121.05

Friday, September 16, 2005

Geocaching, datum and geoids

Looking for a good definition of "datum" I ended up going after benchmarks placed by the National Geodetic Survey in bridges, railroad tracks and other undisturbed locations around U.S.

I was trying to convince myself that this would help verify how accurate my GPS card is but at the end it is quite fun just to go after these "treasure hunts". GeoCaching keeps a database and a photo gallery of benchmarks and their locations. You can search for benchmarks by ZIP or their Point/Permanent ID (PID) and log your findings.

You can also look up the NGS database thru several ways including a dynamic map that displays the locations of these marks. Look for entries with Adjusted in their description and make sure the marker is a benchmark disk (and not a rivet, for example).

For each of them you can check the corresponding datasheet describing its PID, NAD 83 coordinates, type of marker and how to find it. Give yourself a good quiet time to read these directions, they can be quite overwhelming. Obviously, a compass will help you figure out which way is what.

World Geodetic System 1984 (WGS 84)

Going after a good and clear definition might take you for a pretty good spin. And that's exactly what I ended up doing while trying to figure out what exactly WGS 84, NAD 83 and datum meant.

A good descripition for WGS 84 and a great explanation of geodetic datum can be found at the WGS website in the "WGS 84 Implementation Manual", a training document for navigational systems available there.

"There are many geodetic reference datums in use throughout the world providing references for the charting of particular areas. Each datum has been produced by fitting a particular mathematical Earth model (ellipsoid) to the true shape of the Earth (geoid) in such a way as to minimize the differences between the ellipsoid and the geoid over the area of interest. [...] These different datums and ellipsoids produce different latitude and longitude grids and hence, different sets of geographical coordinates."

And later in the same chapter:

"Ground-derived coordinates (Latitude and longitude) are determined with measurements and calculations on mathematical reference models. These models represent the shape of the Earth in a particular geographic region and are called geodetic datums."

"Ellipsoidal Datum = Cartesian Datum + Shape of Earth Ellipsoid"

A position is defined by its coordinates in three axes (x, y and z). Each of these reference points uses "Earth’s centre of mass" as its geocentric origins. Those benchmarks (some provide horizontal references, other vertical) and their corresponding measurements tells us how much above (or below) the geoid they are.

Source: WGS 84 Implementation Manual
Source: WGS 84 Implementation Manual.

"The Z-axis coincides with the mean rotational axis of the earth. The mean equatorial plane perpendicular to this axis forms the (X-Y) plane. The (X-Z) plane is generated by the mean meridian plane of Greenwich. The latter is defined by the mean rotational axis and the zero meridian of the BIH (Bureau International de l´Heure) adopted longitudes ("mean" observatory of Greenwich)."

North American Datum 1983 (NAD 83)

You might also be wondering if there is difference between WGS 84 and NAD 83. Both are based on ellipsoid models but according to this document, you might find differences among references made with both systems. But not greater than one meter according to it.

The idea is that everyone in the world makes use of the same reference model. US had a previous datum (NAD 27) that is now considered outdated and unfit for use with GPS. According to the NGS FAQ, NAD 83 is the "horizontal control datum for the US, Canada, Mexico and Central America". (If you are wondering NAVD 88 is the vertical control datum or water and tidal levels for US.)

There is also discussion about in the future to make use of the International Terrestrial Reference Frame coordinated by the International Earth Rotation Service.

But after all this, nothing like looking for caches with a good GPS receiver...

Sunday, September 11, 2005

Degrees, Minutes & Seconds

While calibrating images to add its corresponding geographic references you need to provide a specific point in the surface of the Earth. There are at least three ways of describing them:

  • Degrees (DD.DDDD)
  • Degrees, Minutes (DDMM.MMMM)
  • Degrees, Minutes and Seconds (DDMMSS.SS)
Where

1 degree = 60 minutes = 3,600 seconds

or 1° = 60' = 3,600"

  • To convert from Degrees to Degrees and Minutes

122.37° = 122° (0.37 * 60') = 122° 22.2'

  • To convert from Degrees and Minutes to Degrees, Minutes and Seconds

122° 22.2' = 122° 22' (0.2 * 60") = 122° 22'12"


If you want to go the other way around, like converting seconds to minutes you need to multiple your seconds by 1'/60" (from above 1' = 60') so

12" x 1' / 60" = 12 x 1' / 60 = 12/60 x 1' = 0.2'


Similar process to convert minutes to degrees

22.2' x 1° / 60' = 22.2 x 1° / 60 = 22.2/60 x 1° = 0.37°

You can also use some JavaScript-based calculators if this looks too much.

Wednesday, September 07, 2005

Digital Raster Graphics

Among the limitations of the shareware/demo version of OziExplorer is the inability to import map images in formats that provide georeferencing information like tfw (tif), jgw (jpeg), and GeoTiff file formats.

If you go to the USGS website at http://topomaps.usgs.gov/drg you will find out about DRG's:

"A digital raster graphic (DRG) is a scanned image of a U.S. Geological Survey (USGS) standard series topographic map, including all map collar information. The image inside the map neatline is georeferenced to the surface of the earth and fit to the Universal Transverse Mercator projection. The horizontal positional accuracy and datum of the DRG matches the accuracy and datum of the source map. The map is scanned at a minimum resolution of 250 dots per inch."

If you are looking for images from CA you can download DRG's, which in fact translates to a .tiff file and the corresponding .tfw georeference at the California Spatial Information Library.

There select the MapSurfer Interface. Zoom to the area of interest, click the red information button, drag a rectangle in the area of the map you are looking for and wait for a list of options to be displayed at the bottom of the window.

Now click the link for the maps you want to download. You will find among them 1:250K, 1:100K and 1:24K Series on NAD 27 and NAD 83 datum in untrimmed or trimmed files. Download both the .tif and .tfw extensions.

In OziExplorer Trial (not the Shareware/Demo version), select File | Import Map... Single DRG File. Point to the directory where you saved both the .tif and the .tfw files. Remember which NAD datum you picked and select the correponding option in the import combo box.

After this you will have a .map file in the formatted supported by OziExplorer opened for use. The colors from the resulting image appear washed out, showing the extra work the Topo! guys had to get better results from the original maps. It won't have street names either. But you do have a much higher resolution and several more zoom levels.

Here you will find some more information regarding DRG's files like what exactly is this trimming business:

"The standard DRG images include map collar information as displayed on a regular USGS paper map. Trimmed DRG images have been clipped to the neatline of the quad, allowing for easier image mosaicking and edgematching."

OziExplorer & OziExplorerCE

OziExplorer and its companion WindowsCE product like Aussies are from DownUnder.

This package contains pretty much every single feature a GPS user would probably require from a software product. It does have some funny ways about doing things, but it also has a very long history of added features, bug fixes and support for whatever is out there.

I learned about more Datum (probably Data if plural) that I ever wanted to know about, lots of map providers and its different formats, quick and easy getting started tutorials and even the internal file formats used by the product. And this package leaves the feeling that you will need a lot of time to exhaust its possibilities.

OziExplorer installed easily in a WinXP box with a CF card adapter to the PCMCIA port. (The GM-270 from Holux and its COM driver was sitting at COM6 all the time, I only had to run the Holux utility GPS Viewer and scan the ports to find it there.) Ozi let me configure COM6 and have it as a moving map.

For CF cards you can use the "NMEA Only" setting and discover that Ozi includes support for Garmin, Magellan, Eagle units and pretty much every single GPS receiver under the Sun.

But first you need to load a .map file (more on this in a minute), then establish the connection with the GPS unit. The status bar will show which kind of sentence the unit is transmitting, if a fix was obtained and the status of the link.

The trial version only run for one hour without interruptions and you cannot upload/download data to a GPS unit; load or save waypoints or events or save map images in bitmap.


Creating .map files

The idea was to use the pair OziExplorer & OziExplorerCE with the PDA as a moving map. But the .map file has to be created first in the Windows version. And then converted to the ozfx3 format (or ozf2 for previous versions) with a utility available at its website.

When you install OziExplorer in your Windows box you will be asked if you want two icons to be created by the installer: one for the demo/shareware version and another for the trial.

The trial includes most of the options available for registered users (US$85 currently). The Shareware version has restricted usage, like for example maps can only be imported from .bmp images.

The Trial will let you import maps from several image formats (among others .jpeg and .tiff) including some pretty obscure types. I decided to try it out using .bmp images exported from Topo! The idea is to georeference the image by creating a .map file with two reference points (up to nine in the registered version).

First select File | Open and then "Calibrate image" to load a .bmp image file. Set the reference points by clicking in the image location and selecting the Point 1 tab in the right panel for the first reference point. Type in the Latitude and Longitude for that point. Do the same for the second reference point. Hit the Save button and you will get a .map file.

Ozi map formats

Download the img2ozf utility from the Optional Extras link in the website and install in your PC. These are conversion utilities that will take a .map as input and generate map files in the format used by OziExplorerCE.

After selecting the .map files and corresponding .bmp images, click the Convert Maps button. This will start the ozfx3 conversion.

In the same utility select the next tab, Copy Maps to PDA, hit the refresh button and the new _ozf.map files should be displayed. Hit the Copy Maps to PDA button and the ozfx3 files will be transferred to a connected PDA.

Voila! You got georeferenced maps in your PDA.

OziExplorerCE costs US$30 and according to the website, to fully utilize this product you need a registered copy of both products. The trial version lasts for two weeks.

Among its limitations the evaluation copy of the OziExplorerCE will interrupt its communication with the GPS unit while in moving map mode and a "Demo version" watermark shows up in the maps.

OziExplorerCE

I like the option for Sirf-based GPS receivers where OziExplorerCE let you choose a fix for the elevation bug from these chips. I wasn't aware of the bug but I did notice some weird values (like negative altitude) a couple of times with other packages.

It might not be a perfect solution but it gives probably better values. In fact it tends to increase the altitude value when you are going downhill under poor satellite visibility conditions.

With the Demo version you can log your track (default behavior with GPS connection active) and save it to memory or disk. It does find your georeferenced maps for the position you are in.

The demo will pop up a screen to let you know that you are using a demo version and turn the GPS connection off. But it does have a lot of room for control, like letting you configure the color of tracks and providing your own schemes.

This is probably the most feature rich package available for Windows CE devices.

After tracking a route, you can upload the TrackLogFile from the PocketPC to the Data directory of OziExplorer in the PC. Use the File menu to open the previous .map image you used for that track. Use the "File | Load from file | Load Map File" and "Load Track from File" for that.

With the corresponding .map file loaded, select View | Tracks | Track Control (or hit Alt-T). Select the Track Log File you want to replay. Now click the button with a plus sign and a down arrow for more options. Select Track Replay and hit the button with the red dot on the Track Replay Control to start the replay.

As a photographer friend once said "with Ozi you will typically use for trip planning and afterwards for analysis of the waypoints/track data. You can also use Ozi on a laptop in the car, connected with GPS - Ozi will show you where you are at your maps. This is way cool. Sometimes we use this when we are at unknown territories and wish to know what is [there] before us".

Saturday, September 03, 2005

Playing with Log files

If you select the Data tab of GPSDash, Pocket Topo! and some other GPS packages you will be able to watch a continuous stream of data sent by your GPS card.

Something that looks like this:

$GPGGA,185621.155,3659.6803,N,12157.7357,W,1,07,1.5,12.9,M,,,,0000*2E $GPGSA,A,3,23,20,16,13,27,04,24,,,,,,2.9,1.5,2.6*38 $GPRMC,185621.155,A,3659.6803,N,12157.7357,W,0.00,,030905,,,A*67 $GPGGA,185622.155,3659.6803,N,12157.7357,W,1,07,1.5,12.9,M,,,,0000*2D $GPGSA,A,3,23,20,16,13,27,04,24,,,,,,2.9,1.5,2.6*38 $GPGSV,3,1,09,23,69,007,32,20,55,178,33,16,45,070,37,13,43,313,34*70 $GPGSV,3,2,09,27,29,251,35,24,18,243,31,04,15,285,31,25,06,058,26*72 $GPGSV,3,3,09,03,05,123,*46

These are NMEA Sentences, the standard proposed by the National Marine Electronics Association and used by GPS devices and vendors to describe geographical positions.


To understand this talk you need to grab a copy of the NMEA 0183 standard specification or check the web for FAQ's.

For example, the first line above

$GPGGA,185621.155,3659.6803,N,12157.7357,W,1,07,1.5,12.9,M,,,,0000*2E

Uses one of the available sentence types GGA (Global Positioning System Fix Data) and if I'm reading this right, it seems to be telling me among other things that:

This measurement was taken at 18:56:21 UTC (Universal Time Coordinates); at latitude 36 degrees 59.680' North and longitude 121 degrees 57.735 West with a GPS Fix obtained from data of 7 satelites, at 12.9 meters above sea level.

The other sentences will give you different types of information. $GPGSV for example will tell you which are the satellites in view and $GPGSA gives you data on GPS DOP (Dillution of Position) and active satellites.

Notice that some GPS receivers can generate log files in a different standard like Sirf, maker of GPS chips.

"Hacking GPS" from Kathie Kingsley-Hughes has among other things a chapter that describes NMEA sentences and the use of VisualGPS.

VisualGPS

With a package that supports reading and writing of log files you can track your trips and generate logs with data received from your GPS receiver to replay it later.

Some of these packages provide ways to generate and analise GPS data in NMEA format. Among them you will find GpsDash described in previous posts and VisualGPS with free versions available for the PocketPC and Windows.

VisualGPS CE won't show you a moving map to help you track your current position but it provides textual information about it and detailed status of the satellites data.

As soon as a fix is obtained, start a new log file [Log | Start] by selecting where VGPS should save it. Go about your trail and select Log | Stop when you are done.

Surveys

You can replay the log file with VisualGPS in the Pocket PC or transfer it to its companion package in a Windows box. Visual GPS has a pretty interesting Survey window that you can use to analise your GPS data.


Check this site for some really cool "drawings" made with data from GPS log files.

GPSVisualizer

A quick way to generate some interesting "drawings" is provided by GPS Visualizer. You upload one or more log files, select the background for your map, output format and the site will generate a combined image of your tracks with image from satellites or USGS maps.