Ubuntu, Gpsdrive, Mapnik, Postgress, Postgis, Openstreetmap HOWTO Copyright R K Scanlon June 2008 Version: 0.1.1 Date: 06 June 2008 Introduction This howto will provide you with the steps needed to install gpsdrive with mapnik and Openstreetmap data. You will then have the capability to generate the map in gpsdrive on the fly. It is necessary to follow all steps in sequence to get it working correctly. The final installation is a maximal install ie it has all possible options installed. If you do not want the optional parts just leave out those marked as optional. This can be regarded as a full manual install. Other than using synaptic it is completely done from a command line (terminal). Conventions used in this document Command line prompts: $> user prompt (yourself or postgres) #> root prompt (su) There are 10 sections to this how to: 1. Install Ubuntu 2. Setup gpsd 3. Setup PostgreSQL 4. Setup Postgis 5. Install osm2pgsl 6. Install iconv 7. Install Mapnik 8. Install Gpsdrive 9. Getting it all running together 10. Test Gpsdrive Assumptions I am using Ubuntu 9.04 you may elect to use another linux distribution. However I can not guarantee that this process will work on any other distribution. Some of the steps may be redundant for your distro but if you have problems then these may be a place to start. I also assume you know how to use synaptic or apt-get in Ubuntu. If using another distro then you may have another package installer. This is a clean install. If you already have your linux distribution in place then start from 1.3 I also created a directory /usr/local/src/gpsdrive to place all downloaded software to. So when you see download a file etc this is where it is downloaded to. Likewise when unpacking a zip or tar file this is where it is done. Whereever you see username in this document it means the user that will be running gpsdrive. this is usually your username. Where instructed to change to root (su) or sudo then you need to enter the root password when prompted. Software Ubuntu 9.04 Postgres 9.3.1 Postgis 1.3.1 Mapnik 0.5.1 Gpsdrive from svn gpsd BEFORE YOU START. This is a relative easy setup but at any point along the way if you get errors find out why and correct them before proceding further. If left uncorrected then you will not have everything working correctly at the end and will end up with a bigger mess to correct. Steps 1. Install Ubuntu 8.04, other required libraries and perl modules 1.1 Use your preferred medium to install Ubuntu from. I have done install from both CD and USB Pen drive. 1.2 After installation has completed and you have a normal desktop login do a complete update. 1.3 Then install the following packages via synaptic or apt-get (these will be the current updated packages): Package Required for Must have/optional Postgres Postgres, OpenStreetMap must have Python-pygresql Postgres, Mapnik, Gpsdrive(scripts) must have Postgis Mapnik, Openstreetmap must have Cmake Gpsdrive(build) must have Gpsd Gpsdrive must have Gpsd-clients Gpsd (testing) optional (but best to have) Gpsbabel optional build-essential Gpsdrive(build) must have libxml2-dev osm2pgsql(build) must have libgeos-dev osm2pgsql(build) must have libpq-dev osm2pgsql(build) must have libbz2-dev osm2pgsql(build) must have libboost-thread mapnik must have libboost-thread-dev mapnik must have libboost-filesystem mapnik, Gpsdrive must have libboost-filesystem-dev mapnik, Gpsdrive must have libboost-regex mapnik must have libboost-regex-dev mapnik must have libboost-iostreams mapnik must have libboost-iostreams-dev mapnik must have libboost-serialization mapnik must have libboost-serialization-dev mapnik must have libboost-python mapnik must have libboost-python-dev mapnik must have libboost-program-options mapnik optional (but best to install) libboost-program-options-dev mapnik optional (but best to install) libpng-dev mapnik must have libjpeg62-dev mapnik must have libfreetype6-dev mapnik must have libltdl mapnik must have libgdal1 mapnik optional (but best to install) libgdal1-dev mapnik optional (but best to install) libgtk2-dev gpsdrive must have libfontconfig gpsdrive must have libgda3 gpsdrive must have libgda3-common gpsdrive must have libgda3-dev gpsdrive must have libgda3-odbc gpsdrive must have libgda3-postgres gpsdrive must have libgda3-sqlite gpsdrive must have gettext gpsdrive must have libdbi-perl must have libdbd-pg-perl must have libdbd-sqlite-perl must have subversion all must have NOTE: Many of these will install dependencies as you go and I always find synaptic the best for this. NOTE: Some of these may already be installed. 1.4 Create working directory for downloads etc. #> mkdir /usr/local/src/gpsdrive #> chown username.username /usr/local/src/gpsdrive 1.5 Download the following: 1.5.1 openstreetmap icons as debian packages from: #> cd /usr/local/src/gpsdrive #> wget http://www.gpsdrive.de/debian/pool/main/openstreetmap-map-icons_7698_all.deb NOTE: This is all of the icons if you only want some then the other files are there as well NOTE: It should be possible to add this as a repositry to synaptic and update automatically 1.5.2 Install openstreetmap icons: $> su #> dpkg -i openstreetmap-map-icons_7698_all.deb #> exit 1.5.3 World boundaries (51MB) and coastlines (~200MB): $> wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz $> wget http://hypercube.telascience.org/~kleptog/processed_p.zip $> tar -zxvf world_boundaries-spherical.tgz $> unzip processed_p.zip $> mv coastlines/* world_boundaries/ $> rmdir coastlines This completes the installation of all the necessary libraries etc. 2. Setup gpsd NOTE: This should be setup automatically so really just need to test if it's working correctly and there is really no point in going any further without getting this working. NOTE: You need your gps for this. 2.1 Edit the file /etc/default/gpsd You may need to edit this file as by default gpsd is not started automatically. $>su #>gedit /etc/default/gpsd The file should read: START_DAEMON="true" DAEMON_OPTS="-n" DEVICES="/dev/gps" The last line has to be your gps device it could be at /dev/ttyUSB0 for a USB gps but it will depend on your gps and installation. 2.2 Plug your gps in (I use a usb gps mouse). 2.3 Run: &> gpspipe -r This should produce a lot of nmea sentences which indicates your gps and gpsd is working correctly. -C to exit. NOTE: If you want to use gpsdrive with gps input do not proceed beyond here until you have this working. 3. Setup PostgreSQL 3.1 Create Database You need to be authenticated as the PostgreSQL super user in order to execute many of the commands below. You can not login as this user. Instead as root enter: $> su #> su - postgres As the postgres user run the following to create the database: NOTE: Replace username with the user that will be using mapnik. $> createuser username Shall the new role be a super user(y/n) y $> createdb -E UTF8 -O username gis $> createlang plpgsql gis 4. Setup PostGIS 4.1 As the postgresql super user: NOTE: Replace username with the user that will be using mapnik. $> psql -d gis -f /usr/share/postgresql-9.3-postgis/lwpostgis.sql $> echo "ALTER TABLE geometry_columns OWNER TO username; ALTER TABLE spatial_ref_sys OWNER TO username;" | psql -d gis $> exit #> exit 5. Install osm2pgsql and the osm data 5.1 Using svn get osm2pgsql from: $> cd /usr/local/src/gpsdrive $> svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/ 5.2 Compile and install $> cd /usr/local/src/gpsdrive/osm2pgsql $> make 5.3 Download osm data NOTE Before downloading the whole of the world data (>3GB) consider how much of this you really need. I downloaded only the data for my country as at this time I have no need for other places. $> cd /usr/local/src/gpsdrive For the whole world $> wget http://planet.openstreetmap.org/planet-latest.osm.bz2 For extracts see: http://wiki.openstreetmap.org/index.php/Planet.osm#Extracts 5.4 Put the osm data into the database $> cd /usr/local/src/gpsdrive/osm2pgsql $> ./osm2pgsql -m -d gis ../planet-latest.osm.bz2 or $> ./osm2pgsql -m -d gis ../your_downloaded_data_file.osm or you can limit the data by using a bounding box $> ./osm2pgsql --bbox minlon,minlat,maxlon,maxlat -m -d gis your_down_loaded_data_file.osm minlat = minimum latitude minlon = minimum longitude maxlat = maximum latitude maxlon = maximum longitude Negative for latitude south and longitude west. This took about 3 minutes for my country and I understand that for the world it takes several days. See http://wiki.openstreetmap.org/index.php/Mapnik#Tuning_the_database and http://wiki.openstreetmap.org/index.php/Mapnik#Loading_data if you intend loading the whole world and/or encounter problems with loading the data. This is done at this stage rather than using the script ("mapnik-osm-updater.sh") provided by gpsdrive as it gives you a chance to correct any possible errors early in the stage. 6. Install iconv 6.1 Download iconv $> cd /usr/local/src/gpsdrive $> wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.12.tar.gz 6.2 Unpack iconv, compile and install $> cd iconv $> ./configure $> make #> su #> make install #> exit 7. Install mapnik 7.1 Download mapnik $> cd /usr/local/src/gpsdrive $> wget http://download.berlios.de/mapnik/mapnik_src-0.5.1.tar.gz 7.2 Unpack mapnik, compile and install $> tar -zxvf mapnik_src-0.5.1.tar.gz $> cd mapnik-0.5.1/ $> python scons/scons.py PGSQL_INCLUDES=/usr/include/postgresql PROJ_INCLUDES=/usr/include PROJ_LIBS=/usr/lib XMLPARSER=libxml2 $> su #> python scons/scons.py install PGSQL_INCLUDES=/usr/include/postgresql PROJ_INCLUDES=/usr/include PROJ_LIBS=/usr/lib XMLPARSER=libxml2 #> exit 7.3 Update /etc/ld.so.conf Mapnik is installed in /usr/local/lib so you need to add that to /etc/ld.so.conf and run ldconfig. $> su #> echo "/usr/local/lib" >> /etc/ld.so.conf #> ldconfig #> exit 7.4 Move world boundaries/coastlines to mapnik directory $> su #> cd /usr/local/src/gpsdrive #> cp -a world_boundaries /usr/local/lib/mapnik #> exit 7.5 Check Mapnik is working (optional) This check is just to make sure that mapnik will render correctly from the data you have entered and is not necessary for gpsdrive to run. However it does provide you with a check that all is well upto this point. 7.5.1 Download scripts $> cd /usr/local/src/gpsdrive $> svn co http://svn.openstreetmap.org/applications/rendering/mapnik $> cd mapnik 7.5.2 Edit set-mapnik-dev Edit the "set-mapnik-env" script to fit your local file names etc. All the settings are documented inside this script. Replace all "~" with "/usr/local/src/gpsdrive". Replace "osm" with "gis" 7.5.3 Create osm.xml Create the osm.xml file from osm-template.xml: $> source ./set-mapnik-env $> ./customize-mapnik-map >$MAPNIK_MAP_FILE 7.5.4 Add a symlink $> ln ../world_boundaries . 7.5.5 Edit generate_(tiles|image).py Set a bounding box and zoom levels in generate_(tiles|image).py $> ./generate_image.py This produces one image of the bounding box you set and is an ideal way to check that everything to this point is working correctly. or $> ./generate_tiles.py WARNING: This can take a long time to run and I do not recommend using it unless you need the tiles generated. 8. Install gpsdrive 8.1 Using svn get gpsdrive $> cd /usr/local/src/gpsdrive $> svn co http://svn.gpsdrive.cc/gpsdrive/trunk gpsdrive-svn 8.2 Compile and install $> cd gpsdrive-svn/build $> ./build_make.sh $> make $> su #> make install #> exit 9. Getting it all running together 9.1 way.txt $> touch ~/.gpsdrive/way.txt 9.2 Some symlinks and a directory to create $> su #> mkdir -p /usr/lib/mapnik/0.5 #> ln -s /usr/local/lib/mapnik /usr/local/share #> ln -s /usr/share/icons/map-icons/ /usr/local/share/icons #> ln -s /usr/local/lib/mapnik/* /usr/lib/mapnik/0.5 #> ln -s /usr/local/share/gpsdrive /usr/share #> exit 9.3 A script to run $> cd /usr/local/src/gpsdrive/gpsdrive-svn $> scripts/gpsdrive-update-mapnik-poitypes.pl 10 Test gpsdrive 10.1 Run from a terminal to check infomation sent to standard output. $> gpsdrive Anything other then the following may indicate incorrect operation. DB: Using waypoints from local user file. DB: Using waypoints from OpenStreetMap database. Read 576 POI-Types from geoinfo.db You should be able to select mapnik mode from Map Control. Assuming you have added the osm data then the map should be dynamically generated. That's it you should now have a working gpsdrive/mapnik/openstreetmap setup. Bibliography http://www.gpsdrive.de http://wiki.openstreetmap.org/index.php/Mapnik http://www.mapnik.org FAQ Q: How can I change the colours used by/with mapnik? A: The colors for mapnik are set in the file ~/.gpsdrive/osm.xml which is created from /usr/share/gpsdrive/osm-template.xml on first run. Q: When trying to access the database for poi's it keeps giving an error. I've narrowed this down to the fact there is no column poi in the table planet_osm_points. What should have put this column into the database table? A: When GpsDrive is compiled with mapnik support, and the postgis database for mapnik is available, GpsDrive can use the Points of Interest stored there. Normally the database should be filled with the script "mapnik-osm-updater.sh", which will also create the 'poi' column necessary to use this feature. However the column 'poi' can also be created by using "gpsdrive-update-mapnik-poitypes.pl" at any time. Q: How can I access all my mysql poi's from previous gpsdrive installation? A: As of version 2.10pre5 GpsDrive stores its waypoints in an sqlite data-base. the mysql database used before is not necessary anymore. To get your waypoints from that mysql database to the new structure, you can export your data to a GPX file with 'poi-manager_mysql.pl -e -f FILE.GPX', which can then be imported with 'poi-import.pl -f FILE.GPX'. Take care, the import script just throws the waypoints into the database without any dupe checking! Q: Are the POI in the postgres osm database or somewhere else? A: There are two databases used for storing points: All user created data and temporary points are stored in an sqlite file in the users home directory (~/.gpsdrive/waypoints.db), which will be created, if it's not existing there. If mapnik is available, the OpenstreetMap points stored in the postgis database can also be used (Access is readonly!). This is also possible, when the displayed map is not rendered from the mapnik database (Mapnik mode off). Q: How is the script gpsfetchmap.pl related to mapnik? A: With this script, you can download bitmap images to use as maps. If mapnik mode is enabled, the maps to display are rendered live directly from the postgis database on your system. You don't have to download anything with gpsfetchmap.pl to use this feature. Please have a look at mapnik-osm-updater.sh, to see how to create this database and fill it with data from the OpenStreetMap project. Q: The poi's are showing but the icons used are incorrect or missing. A: Run gpsdrive-update-mapnik-poitypes.pl to update the poi's in the PostgreSQL database and the icons will then render correctly with the currently selected theme. Q: On startup of gpsdrive I get "Cannot init mapnik. Mapnik support DISABLED: Could not create datasource. No plugin found for type 'shape' in layer 'world-1'". How do I fix this? A: See section 9.2 above. Q: In gpsdrive using mapnik the position is north/south of where it should be. If you are north of the equator the position appears south of where it should be and if your south of the equator the position appears north of where it should be. A: Rerun section 5.4 without the -m option ie "./osm2pgsql -d gis ../planet-latest.osm.bz2" Q: Running "psql -d gis -f /usr/share/postgresql-9.3-postgis/lwpostgis.sql" gives errors that file liblwgeom.so can not be found. A: Check that there are not two copies of this file. There may be one in /usr/lib and /usr/local/lib the correct location is /usr/lib. Change Log 0.1.1 06 June 2008 Updated names of some required packages in 1.3 Removed 1.4 perl modules now installed in 1.3 Renumbered 1.5-1.6 Cleaned up formating in 1.5-1.63 Corrected error in 1.6.3 Added iconv and renumbered step 6 - 9 Corrected error in 7.4.1 Added edit details in 7.4.2 Added new 7.4.4 and renumber after Added more symlinks to 9.2 Added to FAQ 0.1 04 June 2008 Initial draft