The Data Archiving Library assists with archiving messages ingested via a stream layer. This library provides a data agnostic workflow that you can customize to produce and archive stream data in different formats including Protobuf, Avro and Parquet. The Data Archiving Library is included in the HERE Data SDK.
Along with this library, we provide examples showing how to archive SDII messages in each of these data formats and how to index your data by ingestion time, area (HERE Tile) and event type. You may alter these examples to index your data by a different set of attributes. Together, the Data Archiving Library and the examples provide you with an archiving pipeline solution that you can use to store your stream data for efficient querying customized to your requirements.
The use of the Data Archiving Library requires a basic understanding of some concepts and services related to the HERE platform:
For the terms and conditions covering this documentation, see the HERE Documentation License.
Data privacy is of fundamental importance to HERE and our customers. We practice data minimization and don’t collect data we don’t need. And we promote pseudonymity for data subjects wherever a service does not require personal information to function. We employ privacy by design in services we develop. We strive to go beyond mere regulatory compliance and make privacy an integral part of our corporate culture. We believe that our approach to privacy is vital to earning and retaining the trust of our customers – and the bedrock of our future success as a data-driven location platform.
For more information on how data privacy is of fundamental importance to HERE and our customers, see the HERE Privacy Charter.
For more information on data security and durability best practices, see the Data API.