# Select a Streaming Connector: Kafka or HTTP

Stream layers are implemented as Kafka clusters.

To read from or write to a stream layer, your application must use one of the following connector types:

• Direct Kafka is the preferred connector type since it directly communicates with the underlying Kafka cluster. It is the default connector.
• HTTP Connector is an HTTP wrapper on top of Kafka, and therefore implies a communication overhead.

If required, you can provide a different connector type for the producer and the consumer. For example, your application can use Direct Kafka to read from a stream layer and the HTTP connector to write to a different stream layer.

### Note

If your application is using the Data Client Library to access data in the Marketplace, it must use the HTTP connector so that your data consumption can be calculated.

## Direct Kafka

To use the Direct Kafka connector type, you must provide the following dependency:

Maven
sbt
<dependencies>
<dependency>
<groupId>com.here.platform.data.client</groupId>
<artifactId>kafka-support_2.12</artifactId>
<version>1.18.79</version>
</dependency>
</dependencies>
libraryDependencies += "com.here.platform.data.client" %% "kafka-support" % "1.18.79"

Note that, if your application uses flink-support, you do not need to add this dependency as the dependency is already included in flink-support. For more details, refer to the Dependencies for Stream Pipelines section.

To enable Direct Kafka support, you must provide the following configuration:

here.platform.data-client.stream.connector {
producer = "kafka-connector"
consumer = "kafka-connector"
}


Producers and consumers can be configured independently. For more information, see the Data Client configuration file in the Configuration section.

## HTTP Connector

If you cannot use Kafka Direct, for example because your application needs to access data in the Marketplace or is running behind a proxy, use the HTTP connector.

If you cannot use Kafka Direct, for example because your application is running behind a proxy, use the HTTP connector. To use it, the following configuration must be set:

here.platform.data-client.stream.connector {
producer = "http-connector"
consumer = "http-connector"
}


Producers and consumers can be configured independently. For more information, see the Data Client configuration file in the Configuration section.

### Note: Kafka Consumer Group Name

If your application uses the HTTP Connector type, the connector will define a unique Kafka consumer group name for a given combination of credentials, catalog HRN and layer ID. As a result, if you run two applications that use the same credentials, and consume data from the same stream layer, any given message will be consumed by only one application. Specify a unique kafka consumer group in each application to avoid this behaviour.