Locatieserver: about parcels and roadnumbers

For those interested in dutch OpenData: our national OpenData service PDOK has a Solr based geocoding service available.

Since this week it is possible to search for ‘parcel’ codes to find cadastral parcels, but also to search on so called ‘hectometer-paaltjes': the little green number signs you see when you drive the dutch highways.

So in QGIS with the ‘PDOK services plugin’ you can use different filters to search for such objects.

Below a screenie with both a search and one of the reference maps from PDOK:

pdokservicesplugingeocoderopentopo

Happy QGISsing :-)

AmsterdamTimeMachine

Via twitter: AmsterdamTimeMachine.nl.

Jan Hartman’s and WebMappers hard work of georeferencing a set of Old Amsterdam maps: http://amsterdamtimemachine.nl.

6 XYZ-Map services with maps old as 1625 to have a look into history, off course also to be loaded in QGIS :-)

Wanna see ‘the red light district’ in 1625?

debroen

Or see Dutch 17th century glory on a map (well, 135 degrees rotated :-) ):

vbercenrode

Or see that there was also a ‘Waag’ (“weigh house”) on Dam Square?

vberckenrodededam

Go grab the xyz-urls from http://amsterdamtimemachine.nl or download this zip containing a QGIS project file.

The zip contains a .qgs file which you can load as project with both QGIS 2.18 and QGIS 3.x NOW. But also a file ‘amsterdamtimemachine.nl.xml’ which you can load as a set of XYZ-services IF you have a QGIS newer then 3.1 (so you either run a nighly, or wait for 3.2 to come out)..

Happy QGISsing…

Coding a QgsLocator (Plugin)

What is a Locator (plugin)

Some months ago, Nyall Dawson silently dropped a nice widget into the lower left corner of your QGIS screen:

locatorbel

People familiar with QtCreator (the Qt-development environment) should recognize it as a QtCreator Locator look-a-like: a way to (very) quickly search in your project for words, classes, bookmarks, help topics, files etc etc.. It is actually a multipurpose tool to ‘quickly’ search for something, or ‘start’ something. See the original Qt documentation.

Nyall had the brilliant idea to add something like that to QGIS too… and he did. In the code it is called a QgsLocator and it can load/register all kind of so called ‘QgsLocatorFilters’.
Out of the box by starting to type in the search field, you search for: Actions, Processing Algorithms, Spatial Bookmarks, Active Layer Features, Project Layers and Project Layouts.
Just start to type ‘bel…’ and you will see there are a lot of buff(er) related Processing Algorithms available.

Then by clicking on one of the algorithms, you actually start it in processing, super fast, super cool.
Or if you have a world map loaded like in the image and click on ‘Belgium’ you zoom to that ‘feature’.

BUT the coolest thing is, that as a (python) developer you can ‘easily’ implement such a QgsLocatorFilter yourself.

The thing that came up with me first is: ‘Wow, what about using this for a fast searching geocoder…’

So, let’s try…

How to create one

The crux is, to implement/code a so called ‘QgsLocatorFilter’ (see the QgsLocatorFilter api docs, the PyQGIS api or the cpp headerfile).

YOUR implementation of this piece of code is actually the work horse of your Locator Filter. It will take the user input, do something with it (in our case: throw it to an Online Geocoder Api), get back the results, create ‘QgsLocatorResult’s from that, which then will be shown to the user AND describe what happens when a user double clicks on the result.

Off course you need some ‘glue’ to put it in QGIS after this:
– a plugin which you can put in the QGIS plugin repository so a user can download your filter
– the ‘registering’ of your filter to QGIS so it is ‘picked up’ and shown in the Locator search widget
(see the nominatim locator plugin code)

Some geocoders (notably the Google one), need a personal api key to work.
When you left click on the little magnifier-icon a menu will appear in which you can select ‘Configure’

configuremenu

That will bring you to the Locator tab in the Options dialog:

locatoroptions

There you can choose to always enable a Locator Filter, or fully disable it.

As you can see in the right, there is also an (now inactive) configure button. If your locator returns True as a result of calling hasConfigWidget(), then that button becomes active and you can for example implement your own Geocoder configuration window.

An example geocoder: Nominatim_Locator_Plugin

Nominatim (from the Latin, ‘by name’) is “a tool to search OSM data by name and address and to generate synthetic addresses of OSM points (reverse geocoding)”. You can read the OSM wiki or the OSM api information.

I have created a plugin called the ‘Nominatim_Locator_Plugin’ which implements this QgsLocatorFilter interface. Install it in QGIS by enabling the experimental plugins, search for ‘Nominatim’ and install the ‘Nominatim Locator Filter’ plugin. Or … have a look into the python code.

As you can see in the ‘fetchResults’ we wait until we have at least 2 characters, and then we start searching.

One special case with the Nominatim service is that it is a free service, but you are NOT allowed to use it as a ‘suggest’ service. That is: only search a full place name. To handle this, I added the gesture that you have to finish your place- or address-term with a ‘space’, and THEN the request is fired.

The cool thing with Nominatim and OpenStreetMap is that it handles all kind of languages. If you type the dutch word of Statue of Liberty: ‘vrijheidsbeeld’, it will just show up:

vrijheidsbeeld

Some technical details

Which methods you have to define you can find in qgslocatorfilter.h.

One thing to be aware of is that the searching/retrieving/showing of the results is all done in multiple threads! I thought to be clever and tried to use asynchronous GET requests to the services, but ended up fighting ‘thread troubles’. So my tip: just use synchronous HTTP for your requests.

Which brings me to which library to use for the HTTP traffic. In QGIS 2 plugins you see people use the Requests-module pretty often, or plain httplib2 or implement their own NetworkAccessManagers.
Disadvantage of using external modules or home-brew solutions is that often QGIS-configurations like Proxy Settings or Network Timeout values are ignored…

For my experiments I used this module: networkaccessmanager.py.
The nice thing is that it tries to be a thin Python wrapper around QGIS’s own QgsNetworkAccessManager (which in this case is actually a thin wrapper around Qt’s QnetworkAccessManager).
It uses the Proxy settings that a User configured and if needed it can use QGIS’s authorisation module so the HTTP traffix is authenticated against the users own systems.

This boundless solution is a great one, but it would be even cooler if the (rather naive) QgsNetworkAccessManager Cpp implementation could be extended to be more Python/User friendly. So it is easier to handle redirects, timouts, headers etc. And like in the boundless module you could choose between synchronous (blocking) or asynchronous (non-blocking) calls.

I created a QEP (QGIS Enhancement Proposal) for it, and hope others are interested and chime in.

Future Ideas

Off course FOSS is never ‘finished’, as in the good sense of being ‘never finished’ :-)

New ideas always pop up when you are coding:

– use a Nominatim configuration screen, to for example only search for certain osm tags (what about: “find all pubs in current mapcanvas view”, best to be done with a map of your current surroundings :-) )

– use the accept-language option of Nominatim (use current QGIS language) in the query, so you can use ‘Эйфелева башня’ or ‘eiffeltoren’ and get your results back in your preferred language.

eiffeltowerrussion

– a Locator for Google Maps api searching. You will need a key for that one, so create a configuration Window for it too (actually: done :-) )

– the PDOK locatieserver interface, our national geocoder service (actually: done :-) )

– YOUR national geocoder service

So, I hope we get you interested in Locators and will see your plugins appear at plugins.qgis.org

Ah, and some last tips for Python QGIS / PyQGIS programmers:

Latest Python API docs (thanks Denis): https://qgis.org/pyqgis/master/

https://qgis.org/api/api_break.html (all QGIS2.x – QGIS3.x api breaks and fixes)

New version of dutch PDOK services plugin

A short post in Dutch, to let us dutchies know of a new version of the ‘PDOK services plugin’ which eases the use of our national OWS services.

If you want you can install it, and for example view the different soli types of The Netherlands:

De Bodemkaart 1:50000 als WMS en WFS
De Bodemkaart 1:50000 als WMS en WFS

Another nice service is the 25cm Aerial Map of The Netherlands (Luchtfoto 2016 Ortho 25cm RGB).

Happy QGIS’ing all!

PDOK services plugin update

Mostly interesting for dutchies :-)

A short post that there is a new 0.10 version of the pdokservicesplugin (http://plugins.qgis.org/plugins/pdokservicesplugin/),

Some layers have been removed and some are new. The total number of layers is now 6723 (coming from 5298).

WMS layer windspeed at 100m height
WMS layer windspeed at 100m height

Most important changes:

Removed:

– brtachtergrondtijdelijk
– brtachtergrondgrijstijdelijk
– top10nl (nu: top10nl2)
– Noordzee Kabels en Leidingen
– Noordzee Wingebieden

New:

– AHN3 WMS, WFS en WCS
– Adressen (INSPIRE geharmoniseerd)
– CBS Wijken en Buurten 2014
– Historische Rivierkaarten
– Kadastrale Kaart
– NAPinfo
– Potentiekaart omgevingswarmte
– Potentiekaart reststromen
– Potentiekaart restwarmte
– Verspreidingsgebied habitattypen
– Windsnelheden 100m hoogte

Update your plugin via the plugin-manager in QGIS.

2016_histrivieren

Change predefined scales plus new PDOK services

New PDOK services

This post is mostly interesting for dutch readers, as our national OWS service ‘PDOK’ added some new services. And we made them (5000 layers) available via the PDOK services plugin.

Change predefined scales

But I also want to show that you can change the predfined scales that you see in the Scale part of the statusbar in QGIS. By default these go from 1:500 till 1:1.000.000.

Selection_202

But if you are a user who almost always uses your national WMTS service layers als background layers, you will have more QGIS fun when you use your OWN scales there.
Because using the QGIS scales on a PDOK WMTS service, you will see this:

wrongscale

…unsharp tiles because the dutch WMTS tileschema is not in line with the ‘oldfashioned’ scales that QGIS shows you…

But as said: you can determine the scales that you see in the dropdown yourself by loading a simple XML file. For PDOK I created two example xml files which you can download here as a zip file. Unzip it and it will show you two xml files.
Via Settings/Options select the Map Tools tab and remove all excisting scales by using the red minus symbol button. You can now ‘load’ one of the scale xml configurations, and the dropdown will now look like this:

Selection_204.

Using these scales, optimized for those WMTS service, you will have crisp tiles:

rightscale.

In this way you can (as a heavy WMTS user) make it easier to view your services on the optimal scales. You can even remove some scales which you never use!

Shot version: choose YOUR best scale, Happy QGIS-ing!

QGIS goodies

Just a short post in case you missed it.

Since some time QGIS is coorporating with spreadshirt.com, so we can open ‘virtual shops’ all over the world to sell QGIS t-shirst, caps and mugs.

So: check your size, and go via this page to the nearest QGIS-shop and make your friends jealous with our great QGIS shirts :-)

shop_tshirt

Oh, and the QGIS project receives a small percentage of this money to buy refreshments for during the hackfests…

Mozilla Stumbler layer: loading TMS and XYZ tilelayers in QGIS

Mozilla Location Service (MLS) layer

Personally I’m very interested in the Mozilla Location Service (MLS), I wrote an earlier article about it on my work (Zuidt.nl) blog.
Selection_102

Key in this project is that both locations of wifi points and cell towers are crowdsourced and put in a open database, so everybody can use that data to determine his/her position on earth based on some info you get from your laptop or cell phone (without GPS).

This is only possible if as much people as possible start uploading this kind of locations, and as you can see on Mozilla’s overview map it is getting on steam now.

This map layer, showing all blue dot’s on places where locations have been recorded, is a ‘normal’ XYZ tile layer (epsg:3857) generated once a day by Mapbox if I’m correct. After reading Mishari Muqbil blogpost about how to add this ‘blue dot’ layer in OsmAnd android application, I was curious if I could also load this layer in QGIS.

After some searching I found the very nice ‘TileLayerPlugin’ from Minoru Akagi who also is the author of the beautifull Qgis2threejs plugin (have a look into it if you haven’t done so yet!).

So if you want to load xyz map tile layers, like OpenStreetMap, or this Mozilla Stumbler layer in QGIS, do the following:

– via the plugin manager, search for ’tilelayer’ and install the TileLayer Plugin.
– open it’s dialog (initially in the Web menu), and click the Settings button in it and point to some directory on your hard disk for the ‘External layer definition directory’
– in that directory, as you can read in the README at github, you have to put one .tsv file per xyz layer. Note that the values should REALLY be tab-separated, not space separted!
– for the MozStumbler file you need (one!) line like this (tab separated!):
MLSstumber MLSstumber https://d17pt8qph6ncyq.cloudfront.net/tiles/{z}/{x}/{y}.png 1 0 13 -180 -85.0 180.0 85.0
To find the exact cloudfront-url you need, have a look into this json file: in https://location.services.mozilla.com/map.json

As soon as you have those bits in place you can see the blue dots. have a look, the coverage of my hometown is growing:

Selection_103

Arcgis rest service and other layers

But wait… isn’t that the Arcgis / arcgisonline World Topo Map I see there? Yes, it is, Adding one of these lines:
ArcgisWorldTopo esri http://server.arcgisonline.com/ArcGIS/rest/services/World_Topo_Map/MapServer/tile/{z}/{y}/{x}.png 1 0 16 -180 -85.0 180.0 85.0
or
ArcgisWorldImagery esri http://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}.png 1 0 16 -180 -85.0 180.0 85.0
makes it possible to see other arcgisonline mapserver rest tile services in QGIS.

In this way it is also possible to load OpenStreetmap Tiles in QGIS, save:
OpenStreetMap © OpenStreetMap contributors http://tile.openstreetmap.org/{z}/{x}/{y}.png 1 0 19
as osm.tsv and see OpenStreetmap in it’s full glory.

I’ve put some tsv files in this zip: http://qgis.nl/tilelayers.zip for you to try out.

Crowdsourced Street Level Photos

Another post of Mishari Muqbil made me aware of another crowd sourcing project: street level photo’s: http://www.mapillary.com/map

While still young, it’s a nice initiative to bring ‘power to the people’ instead of to the big companies :-)