Most of our partners already have an existing mapping system. The (currently) only possible way to ‘switch’ to TransforMap as a mapping platform is to switch to OpenStreetMap (OSM), if the dataset is suitable for OSM. Data therefore has to be added to OSM. When adding large datasets in bulk to OSM, we call it an import.
OpenStreetMap has very strict guidelines on imports. But don’t be afraid and read on, when done the right way imports can be a huge benefit for all.
There are prerequisites that must be fulfilled:
- The owner of the data must hold ALL rights on it and be able to release it as Public Domain (CC-0)
- → they MUST NOT be geolocated with Google!
- The import must be guided/carried out by an experienced TransforMapper/OpenStreetMapper.
- Data must be suitable for OSM (only Points of Interests visible on the ground for a longer time (e.g. no events)
- Be aware that data in OSM is
- Worldwide accessible, and analyzeable in bulk, also commercial use explicitely allowed.
- Subject to change (anyone can edit it) with no guarantee for availability.
- Will be kept up to date not only by you, but also by 2 million OpenStreetMappers.
To start an import, you can do it yourself if you are an experienced OpenStreetMapper, or contact @species.
How to prepare your data for an import:
- check the licence of EVERY field, especially coordinates
- data format: csv, xls or any geo-format (except kml, where we charge extra due to its complexity )
- Charset: UTF-8!
- max length of field: 254 chars.
- no formatting, no line breaks, HTML, etc allowed!
- its columns should contain:
- address splitted in city, postal code, street/hamlet, housenumber columns
- coordinates (WGS84) (if already available as open data)
- name of POI
- own classification (or better OSM tag)
We would appreciate the following additional information columns:
- contact data (if available, each single entry an own column):
- webseite email, social network sites
- tel, fax (int. Format +49 xxxx xxxx-xx)
- contact name
- a free text field “description” can be used for additional, unstructured information
- opening hours (see osm spec here )
- wheelchair accessibility
- founding year
- smoking allowed?
You can help us by preparing you POI with our TransforMap Taxonomy tags:
- needs: “provides”/“topic”
- OSM “main tag”
Step-by-step How-to for Import
in a spreadsheet software:
- Check charset, convert to utf-8
- check and convert tel-nrs to int. format
- check alignment of columns if using csv
- create a mapping of import columns to osm key/values
- convert column heads to keys and cell contents to values
- check and correct address fields for Country, Postcode, City, Street, House-Nr.
- delete unused columns
If coordinates are missing, use script (TODO link here) to geolocate via Nominatim (OSM geocoder). For addresses not present in OSM, check open data portals and local ordinance survey. Use of single addresses is often permitted.
if you assume that a lot of your points are already present in OSM, it maybe adviseable to do a spatial comparison in QGIS:
Load file in QGIS for further processing and matching to exiting OSM POIs.
Download matching existing OSM-POIs for area (amenity,shop,leisure, … maybe use geofabrik.de shapefile), open in QGIS
create buffer on OSM-objects:
- vector → geoprocessing → buffer, distance 0.00018
- vector → analysis → points in polygon: select all new POIS inside buffers
- rightclick on layer → save selection as “both” (shapefile, or better geojson)
- rightclick on layer → Attribute Table: Invert Selection
- rightclick on layer → save selection as “not-yet-in-osm” (shapefile, or better geojson)
- maybe (if points of your dataset have the same coordinates, use Plugin “Shift Points” with minimal distance (0.00001), save dataset.
For import and converting to OSM format, use JOSM:
- open file to import (“not-yet-in-osm”) in JOSM
- select all (CTRL-A)
- add tag “TODO=YES”
- correct tag keys that where malformed because of shape or geojson conversion
- and now, for each point:
- check if coordinates match a feature on aerial and /or osm data - e.g. houses. POIs cannot lie on:
- in the middle of roads
- in lakes
- … find the most reasonable place and move the POI here - EVERY point has to be moved 1-5m, to avoid overlapping
- if done, remove the TODO=YES tag
- move on to the next, e.g. search for TODO=YES and pick the first one
- for points where a location cannot be found, move it to another layer and add tag fixme=yes
- at first you create a new layer in the “layers” panel
- C-X, switch to other layer (set the green check to it), C-V
- switch back to working layer
- note: do not download any existing OSM data into these layers, use a new one for comparision!
- when done:
- save the done file as ready-to-import.osm
- save the file with the “fixme” POIs. for these POIs, ask the data source about the POIs
- are the coordinates correct?
- is the aerial too old? (this can be spottet if a start_date on the POI is supplied)
consult the local(!) OSM mailinglist - (and imports@ if O.K. locally ) for the import, present the ready-to-import.osm, following the import guidelines.
- if no objections (~1 Week):
- create a OSM user for the specific import
- add description text and contact info (email, link to TM proposal site)
- for bigger imports, add the created import description page to user info
- add this user to “users currently mapping” on the proposal site
- open the file to import in JOSM, make a good changeset comment involing links to wiki pages, upload
- for the “both”-file, use the ~same process, but merge tag data into existing features.
- create a OSM user for the specific import
If you want to create a map out of it, use the code on github, where our demo maps are based on.
For sure you will have questions, post them as reply in this topic! Our answers will then used to improve this howto too