Dependencies for Stream Pipelines

To run an application that uses the Data Client Library within a stream pipeline (Flink), use the flink-support module as a dependency to your project. Don't add any other modules to your project. Adding other modules causes dependency conflicts (with Akka) and your application will not work in the Flink environment.

Note: Akka libraries

Flink depends on Akka libraries, which causes dependency conflicts with the Data Client Library. To resolve these conflicts, the Data Client Library relocated all Akka related dependencies in the internal namespace dataclient_akka. To avoid conflicts, Flink users should only access classes exposed by FlinkDataClient.

The flink-support module is only released as a fat JAR containing all Data Library modules and some shared libraries. This is required to avoid conflicts with flakka (the Flink custom build of the Akka library), which is incompatible with the Akka version that Data Client Library is using.

The module flink-support is only compatible with Flink 1.10.1

To add the flink-support module to your project, add the following dependencies:

Maven
sbt
<dependencies>
        <dependency>
            <groupId>com.here.platform.data.client</groupId>
            <artifactId>flink-support_2.11</artifactId>
            <version>0.13.48</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-scala_2.11</artifactId>
            <version>1.10.1</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-clients_2.11</artifactId>
            <version>1.10.1</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-streaming-scala_2.11</artifactId>
            <version>1.10.1</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-table-planner_2.11</artifactId>
            <version>1.10.1</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>
libraryDependencies ++=
        Seq(
          "com.here.platform.data.client" %% "flink-support" % "0.13.48",
          "org.apache.flink" %% "flink-scala" % "1.10.1" % "provided",
          "org.apache.flink" %% "flink-clients" % "1.10.1" % "provided",
          "org.apache.flink" %% "flink-streaming-scala" % "1.10.1" % "provided",
          "org.apache.flink" %% "flink-table-planner" % "1.10.1" % "provided",
          "org.apache.hadoop" % "hadoop-client" % "2.7.3 % "compile"
        )

results matching ""

    No results matching ""