From Transit to Urban Mobility, Part 1: Origins

16 May 2016 by Dzianis Dudnik

Getting from A to B used to be as simple as, well, ABC. Until recently, commuters took the train if there was one, and the bus if there wasn’t. But as the process of urbanisation speeds up and new forms of transport - trams, bike-sharing and car-sharing schemes to name a few - spring up to get more people around larger cities, knowing how best to do that has got harder than ever.

Luckily, the pace of technological progress has kept up with this urbanisation. It’s up to us to utilise these advances correctly.

Services which provide transit planning assistance have existed for a long time. Whatever mode of transport you are using, you will probably find a service which will build you the optimal route on their service. But to compare forms of transport, you would usually need to consult several apps or websites. How tedious. And what if the best route comes from a combination of modes of transport? Perhaps you should take a car share to a train station, the train to the city centre and then switch to the bike to cover the last couple of miles. The proprietary apps for each service won’t help you with this.

What we need is a service which aggregates every possible solution and gives the user the big picture at a glance to help make the right decision. Enter the Urban Mobility Team.

The first challenge we at HERE faced was that there was no Urban Mobility Team. Well, there was a Team, but it wasn’t Urban Mobility. It was Transit.

What the Transit Team Could Do

Before we go into how we could turn the Transit Team into an Urban Mobility Team, it will be helpful to see what we started from.

The Transit Team provided the following RESTful services: Public Transit time-table and real-time routing, station board with next departures and transit station proximity search, all of that is available for more than 1000 cities around the globe, updated on a daily basis. A fine set of tools to start with, but not enough to reach the level of service we aspire to.

From the very high level, our Transit platform consisted of two major parts:

1. Data collection and processing chain: produces and enriches data from Transit agencies and creates packages of data for a particular city or region or, in some cases, the whole country. These packages are then sent to our routing engine clusters to be accessed and served by the Meta Router. A story for another day.

2. The Meta Router API back-end This is what we’ll focus on for the rest of this article.

The Meta Router

After going through the load balancer, all requests from our customers are handled by a component called Meta-Router, a Python application sitting behind nginx and using Gunicorn as HTTP server.

The most important APIs that the Meta Router provides are ’Transit Routing Service', 'Departure Board Service’ and 'Station Search Service’. Of these, the one we depend on the most is the Transit Routing Service.

Transit Routing Service

Here's what happens when you issue a request to get a route from A to B:

The Meta-Router’s main task is to define which routing engine cluster should be called to calculate an actual route, an apparently easy task except for the fact that in many cases we do not own the most full and quality data for a particular region so an external API might be called. Secondly, we might have multiple sources which are able to provide a route for the same origin and destination. So we have to select the best one and use others as a fall-back solution.

Every incoming request has the coordinates of origin and destination points, so that will help us find the routing engine cluster that has the data we need. Each Meta-Router instance maintains an in-memory Grid which is aware of all the available data-packages and external APIs and which geographical region they cover. (It’s called Grid because every geographical region is represented with a list of 4-character geohash strings, each representing a cell within certain bounding box.)

To find which regions cover both origin and destination points, the Meta Router calculates the geohash string from input coordinates and looks for corresponding entries in the Grid. We could get more than one data source covering a given area, so we need to rank them according to the assumed routing quality and detail. The way it's done is not very straightforward and we are still working the best way to do that.

From our ranked list of regions, each pointing to a specific routing engine cluster or an external API, it is sufficient to get the first successful routing result from a routing engine or external API and return it as a final result.

The Meta-Router starts querying corresponding routing engines sequentially, falling back to the next one in case of failure. And if no routing engine was able to provide an answer, we consult the ultimate fall-back: estimated routing provided by the Here Location Platform (unfortunately, this has very little time-table information).

Departure Board Service

This works in a very similar manner. Since every routing engine has all the time-table information, the process of getting the next departures for a given station is the same as described above. Though apparently simple, matters are complicated somewhat by the need to merge stations from different data-sources, find real-time departure information and, again, integrate external APIs.

Station Search Service

This is powered by the ElasticSearch. If you are familiar with this product, you probably know that it provides lots of great features on top of Lucene like configuring search clusters and a very powerful Query DSL. (If you are not, get familiar! It’s great!)

Our search API is nicely translated into ElasticSearch queries using features like geo-distance filtering and fuzzy term search. Filling search clusters with data is also easy: we upload updated station data in JSON format during every data deployment and let the cluster take care of replication, redundancy and indexing.

So, that's our platform on a high level: where we stand at the moment. We receive incoming request, find the routing engine or external API providing the best data, forward the request and get the result.

In the next part we'll see how to make a transition from Transit to Urban Mobility. Stay tuned!