Getting Started
All new HERE platform users should go through the Getting Started document.
To get started using pipelines, you need to be able to use the HERE platform. This means that you need to be able to use the HERE platform software and libraries. To develop new applications using the HERE platform requires a few more tools. The following things are required:
- You must have an account. Typically this involves requesting an account from your company's Admin. For more information, see the Identity & Access Management Guide.
- Review the HERE platform documentation to get familiar with the design and use of the HERE platform.
- Explore the platform portal - After you log in, you will be directed to the platform portal. Explore the platform portal and become familiar with its features.
- Manage your profile - Click on your Name in the top right of the platform portal screen to access your profile. Here you can find your User-ID and the Groups that you belong to.
Security
The HERE platform is a cloud-based platform providing several microservices and resources. Each microservice and each resource (data catalog, pipelines, schemas, etc.) are protected and require authentication and authorization to be used. Within the HERE platform, you will need credentials to access these services. Learn how to set up your teams and credentials in the Identity & Access Management Guide.
Setting Up a Development Environment
Follow these steps:
- Get your credentials set up as mentioned above or refer to Get your Credentials.
- Configure Your Environment.
Web Portal
If you need information about working with pipelines through the Web Portal, see Using the Web Portal for Pipelines for more information.
Pipeline Tasks
There are three main tasks involved in using HERE platform pipelines.
- Designing and implementing a data processing pipeline as a JAR file.
- Deploying the pipeline JAR file to the HERE platform pipeline.
- Running and managing jobs on a Pipeline Version on the pipeline.
The first task is the most technically ambitious. It involves designing a data processing workflow and implementing it inside of a pipeline JAR file. See the Working with Pipelines article for more information.
The second and third tasks can both be accomplished in three possible ways.
You can use the platform portal and its GUI. For more details, see Using the platform portal UI for Pipelines.
You can use the CLI. For more details, see the Command Line Interface Developer Guide.
- You can use your own custom application to interact directly with the Pipeline Service REST API. For more details, see the Pipeline API Reference.
Deploying a Pipeline
- What it means to Deploy a Pipeline
- To Deploy a Pipeline via the platform portal, see Deploying a Pipeline via Web Portal.
- To Deploy a Pipeline via the CLI, see Deployment and the OLP CLI User Guide.
- To Deploy a Pipeline via the API, see Deploying a Pipeline via API.
- To Configure a Pipeline, see Interfacing with the Pipeline Process.
Pipeline Monitoring and Alerts
- For Pipeline Monitoring information, see the Metrics and Logs User Guide.
- For information about Pipeline Logging Levels, see Pipeline Logging.
- To use Grafana for alerts, see Pipeline Monitoring.
Batch Pipelines
Stream Pipelines
Pipeline Operations
- Running a Pipeline
- Constructing a HERE platform Request
- Using the Command Line Interface (CLI)
- Using the platform portal UI for Pipelines
- For how to upgrade a Running Pipeline, see the Pipeline Lifecycle or Upgrading a Pipeline by Portal
- To run a Sample Pipeline, follow the instructions for HERE Workspace for Java and Scala Developers
Designing a Pipeline
- Developing Pipelines: An Overview
- Maven Archetypes
- For instruction on building a New Pipeline Project, visit HERE Workspace for Java and Scala Developers.
Design Data Manipulation for a Pipeline
See any of these documents:
- Data Processing Library Developer Guide
- Data Inspector Library Developer Guide
- Location Library Developer Guide
- Data Client Library Developer Guide
- Data API Developer Guide
- Data API