Stream Processing

The HERE platform supports Apache Flink framework for running streaming pipelines. We offer two different modules to do that, Flink Connector and Flink Support.

Note

The Flink Connector currently supports stream layer only. Support for all other layer types will be added in the next releases.

HERE strongly suggests to use Flink Connector whenever possible as this allows to make use of the full power of Apache Flink framework.

Flink Connector and Flink Support modules can be combined.

Flink Connector implements the standard Flink interfaces that allow you to create source Tables for reading and sink Tables for writing to stream layers.

As a result, you can use both the relational APIs that Flink supports, which are Table API and SQL. Moreover, you can convert a Table to a DataStream and use the Flink DataStream API.

This means, your business logic does not need to contain any HERE-specific function calls.

For detailed explanation of Flink Connector please see Flink Connector.

Flink Support is a HERE-proprietary implementation of using data from HERE platform catalogs and layers with Apache Flink framework. The distribution of processing jobs to workers is done by Flink but the data model is a HERE-proprietary format. There is no SQL-like interface, so you cannot select and filter data using Table API.

In the end, FlinkDataClient is a wrapper for regular DataClient making sure that all asynchronous communication is syncronized to make it usable with Flink Source and Sink interfaces. There are no additional functions for tabular access.

There are plans but no defined dates yet to retire Flink Support module in long term.

results matching ""

    No results matching ""