Map of pollution in Flanders

The map (see live map here) shows the pollution on the level of the municipality for Flanders. The pollutants for which data are available are nitrogen dioxide (NO2), ozone (O3) and small particles (PM10/PM2.5). These data are retrieve from Aircheckr. An API is available to retrieve these open data, originally provided by the Belgian government.

Several (front-end) tools are used to visualize the data. The map on Flanders with municipalities is shown using jQuery with the plugin Mapael that is based on raphael.js. To retrieve the data from the database, an API was created using Python with Flask and Flask restful.

The schema given below shows the complete (near) real-time streaming data pipeline. The data are retrieved from the exteral API (aircheckr) and send to Apache Flume which stores the data in Hadoop for future batch processing and analytics and also to Apache Kafka. A consumer of Kafka is written in Python and the result is stored in Apache Kudu. At last an API makes the data available for visualization or further processing.

Streaming Data Pipeline

Links

Aircheckr:  http://www.aircheckr.com

Irceline: http://www.irceline.be/nl/documentatie/open-data

Mapael: https://www.vincentbroute.fr/mapael/

Raphael.js: http://dmitrybaranovskiy.github.io/raphael/

Flask: http://flask.pocoo.org/