Surveys

21.04.2017 - escada

The best way to collect data for OpenStreetMap is by surveying an area. With surveying, we mean walking or driving around and taking notes of the features or objects you encounter. For newcomers it might be difficult to start with their first survey. There are as many methods as there are mappers, but it might help to get started when you have read at least one workflow. That is why I wrote this article. I will also explain how you can prepare your survey with data from external sources.

I have been conducting surveys for the past 6 years. Every weekend I take the dogs out on a walk. During those walks I make notes. So I combine two activities, I have never gone out just for the sake of collecting data to be mapped.

One of the benefits of having the dogs with me, is that I was never stopped and questioned about my activities, unlike some other mappers.

Our Dogs, the one in the middle is still an apprentice, the two on the outside already accompany me for 6 years of surveying.

Equipment

Lets start by listing the gadgets I use to collect data. My choice is largely influenced by the fact that it needs to work while I handle the dogs. In other words, it should be easy to operate the device with 1 hand; I do not want to carry it in my hand all the time and I do not want to stop for long periods.

Now the actual equipment:

Garmin Dakota 10, Nikon D7100, LG G3s

Garmin Dakota 10

I started contributing to OpenStreetMap after buying this GPS device. From day one I use it to collect data. It can be used to gather a trace of the paths you followed, as well as for creating waypoints, which are small text notes for items you want to add to the map later on. The traces and the waypoints can be read by JOSM, the editor I use to map (aka improve the map, aka drawing items).

The device is small, has a touchscreen and hangs around my neck with a lanyard when I do not need to type or look at the screen. I also find it reliable. It is powered by 2 AA batteries, so it is easy to have a spare pair with me to avoid getting out of power.

A minor drawback is that you need a cable to connect it to a computer and exchange data.

As I am colour blind, I load the leisure-styled maps on the device. Those maps can be found on bbbike. Of course, those maps are OpenStreetMap-based. You can download them for any place on earth.

Nikon D7100 DSLR

In the early days, I only used the Garmin and did all my data collection with it. But typing on the screen is time consuming, so one day I decided to take my camera with me and take pictures. And I liked it a lot. It is a great way to document the places you visit.

The main benefits of this camera are it is fast, can still be operated with 1 hand, and the most needed controls are accessible via handles or switches. E.g. I can turn a rind on the lens to zoom in and out, while the camera hangs around my neck. The zoom length is indicated with marks on the lens, so even before lifting the camera to eye-height, I can prepare it. I do not like the switches and menus you find on compact cameras, or even worse the 2-finger approach required by smartphones. That does not work well with only 1 hand available (remember the other one holds the leashes of dogs).

A drawback for some might be that the camera has no build in GPS, so the photos are not georeferenced when you download them to your computer. This means that you do not know where the pictures are taken. But there is a workaround for that (see further down). However, it will never be possible to know the direction in which a picture was taken. This can easily be overcome by taking several pictures.

Smartphone with OsmAnd

I run the navigation app OsmAnd on my smartphone, an LG G3s. I place the smartphone in a holder that I can tie around my forearm. This allows me to consult the screen without using my hands. The plastic of the holder make it harder to read in certain light conditions though. In order to keep the rotation of the map stable, even when I move my arm, I use the RotationControl app.

OsmAnd is used as the main navigation aid and to track my walks. So at the end I have 2 tracks, one from the Garmin and one from OsmAnd.

Drawback with this setup is that I cannot take pictures with the smartphone, but I find it undoable to switch from OsmAnd navigation to picture taking anyhow. You have to confirm several dialogs and the switch between the apps is terribly slow. I think it can easily take up to 30 seconds to make a single picture.

Mapillary and OpenStreetCam

Mapillary and OpenStreetCam are popular apps to continuously take pictures of your journeys. Both apps allow mappers to use the pictures for mapping purposes.

I do not use them during my walks. I cannot operate the smartphone all the time to continuously take pictures, I do not want to end up with hundreds of useless pictures (because almost the same as the previous picture) and it would require a dedicated smartphone. You cannot use those apps and OsmAnd at the same time. I can imagine that it is more useful for cyclists or car drivers.

For those that are interested in using this method to capture photos, take a look at Levente Juhász’ blog post on Mapillary.com.

## Preparing a Walk

The Route

I typically make walks between 10 and 14 kilometers, which I plan at home. There are a number of ways to do this, but you can always download a GPX-trace from sites such as

• GPSies
• Wikiloc
• Dogsfriendly – Belgium only
• wandelroutes.org - Flanders only
• or you could plan your own walk along one of the many networks on Wandelknooppunt.be
• Of course you could also use the walks found on Waymarked Trails which shows all hiking routes mapped in OpenStreetMap

Since I want to contribute to the walking networks mapped in OpenStreetMap, I typically chose Wandelknooppunt.be and look for missing routes on Waymarked trails. The walk should be fun for the dogs as well, so I try to find a mix between little villages (a lot to map) and forests and fields (more fun for the dogs and hopefully more beautiful scenery, but less to map).

The GPX-file with the route is downloaded to my computer and later on placed on the Garmin and on the smartphone.

Heritage POIs

As mentioned in Heritage in Flanders and Crowd-Sourced Projects, I am interested in mapping immovable heritage in Flanders. In order to know the location of those objects along (or near) my planned route, I developed the following workflow.

• Visit Wikipedia and download the webpage that lists the items in a particular villages
• Run a Python script to convert the webpage into a format understood by OsmAnd
• Place the result of the Python script on the smartphone

Let’s take a detailed look at each of those steps.

Wikipedia pages with lists of immovable heritage

The Dutch Wikipedia lists all immovable heritage in Flanders, grouped per village. For large towns, there might be multiple pages.

List of immovable heritage in Rumst on Wikipedia

Each line typically contains the name or description of the item, an address, the (part) village the coordinates and an image. Lines without coordinates are useless for our purpose.

As said, I download this type of pages and save them as .html (e.g Rumst.html) on my local disk.

Running the Python script

The Python script can be run from the command line.

python ../Python/wikipedia2osmandgpx.py -f Rumst.html

which will then generate an osmand_Rumst.gpx file in the same folder as the html file. This gpx-file contains 1 waypoint per row in the html-file. An excerpt from that file

for those two lines of the Wikipedia page:

the generated waypoints are

<wpt lat="51.0775" lon="4.425">
		<name>OK-Dorpswoning Markt 13</name>
		<desc>14005 Dorpswoning - Markt 13</desc>
		<type>heritage</type>
</wpt>
<wpt lat="51.0786111111" lon="4.45277777778">
		<name>NOK-B-XIX-Arbeidershuizen Mechelsesteenweg 71</name>
		<desc>14007 XIX-Arbeidershuizen - Mechelsesteenweg 71</desc>
		<type>heritage</type>
</wpt>

As you can see the location of the item is placed in the lat and lon attributes of the wpt-tag. The name contains the description and the address, as well as an indication whether there is an image (OK) or not (NOK for Not OK). The description shows the identification number from Onroerend Erfgoed.

This osmand_Rumst.gpx file is ready to be placed on your smartphone.

The Python script has a few limitations which makes it unsuitable for a fully automated setup on a server where waypoint files can be generated on demand.

• It cannot handle Wikipedia pages with multiple tables for different villages such as the page for Kaprijke and Lembeke.
• It has problems with some strangely encoded characters
• It cannot download the html page itself, because Wikipedia disallows this.
• At this moment it does not look at OpenStreetMap data, which means that you might end up surveying items that are already fully mapped. Since I am one of the few mappers working on this type of features, this is not really a problem.

Placing the files on the smartphones

I place the places with the GPX-trace for the route and the waypoints for the immovable heritage in the tracks folder of OsmAnd. The path depends on the version of OsmAnd you are running and how you configured it.

Now I can toggle the display of those files via the OsmAnd UI. This is done as follows:

1) In the menu, choose “Configure Map”

2) Then “GPX Track…”

3) Then enable your GPX-trace and one or more files with waypoints

4) Press ok

return back to your map, et voilà, the trace of the route and all immovable heritage in the are are visible.

The thick red line represents the route to follow, the red circles with white stars, the heritage items.

Clicking one of the circles, shows the detailed information on the heritage item

Starting the Survey

When I start the walk, I create a waypoint on the Garmin at the same time I make a photo with the camera. This allows me to synchronize the times later on.

The Waypoint file on the Garmin will then contain a text similar to

823772</ele><time>2017-04-16T11:25:03Z</time><name>SYNC</name><sym>City (Medium)</sym></wpt>

During the walk I just make a lot of photos. Each time I see something I want to map later on I take a picture. Sometimes multiple pictures, e.g. an overview and a close up to read a label.

Processing the photos.

Assigning locations to photos

Back home, I download the photos to my laptop. Also the traces from the Garmen and OsmAnd are places on the laptop. I keep those files on my Dropbox.

In order to assign the location to the images, I use geotag, this is a Java application and can run on all platforms.

I first load all images and the trace made by the Garmin. I select the first picture made during the survey and set the time of the SYNC waypoint. This corrects the time recorded by the camera to the one recorded by the Garmin. Now I can let the program calculate the coordinates for each image.

The images are then stored on Smugmug in my xian account, so they are save against disk crashes. Feel free to use those images for mapping purposes or documentation on the wiki.

Now I can start mapping, but that is another story.

Feel free to contact me at marc dot gemis at gmail dot com with any question regarding this article.