HERE platform projects are resource containers specific to your organization that allow you to group and manage access to platform resources. Projects let you easily manage access control and track resource usage, both via the platform portal and command line interface (CLI).
We recommend that you use projects to manage all your platform resources. This guide walks you through the mostly CLI-based steps to migrate resources for a use case—including catalogs, schemas, pipelines and associated pipeline templates—into a platform project, if those resources were not originally created in a platform project. Once your catalogs for a use case are in a project, you can still share them with all or specific projects in your organization so that they can be reused for other use cases.
Home project: This typically means the project in which a resource was created. For this migration guide, it means the project to which you are adding or moving resources. This project now becomes the "home project" for those resources. A resource can only have one home project, but catalogs can be shared to other projects in your organization so that they can be reused in multiple use cases. Storage for a catalog is logged to the home project, and data I/O for using the catalog is logged to the project that's using it.
Adding vs. moving resources: This guide refers to adding catalogs to projects. This means that you are adding the catalog to a project, but not removing any permissions associated with the catalog outside the context of the project. You can choose to remove those permissions later and purely manage the resource in the project, but "adding" rather than completely "moving" the catalog into a project helps mitigate the risk of breaking anything as you migrate your use case to a project. With pipelines, the guide refers to moving them rather than adding them, that is, they will not be accessible outside of project context once moved.
Link any HERE catalogs used in your use case to the project you created in step 1 using the platform portal or the
olp project resource link availability list and
olp project resources link CLI commands.
To link a HERE catalog from the platform portal, go to the Projects Manager, click the project to which you want to add a catalog, and select the Resources tab. Click Add catalog and choose to Link a catalog. A list of available HERE catalogs will be displayed.
IMPORTANT: Link the "HERE Optimized Map for Visualization Plus" catalog (hrn:here:data::olp-here:here-optimized-map-for-visualization-plus-2) to your project so Data Inspector continues to work seamlessly. This will allow you to visually inspect layers of catalogs in your projects (Data Inspector uses the Optimized Map for Visualization to provide the background map).
IMPORTANT: Ensure at least one app has manage access to your input and output catalogs before you add the catalogs to projects so that you can continue to manage access outside of projects. Once you add your catalogs to projects, you won't be able to manage access to them outside the project via the portal but can continue to do so via the CLI, which requires an app to authenticate.
Add your input and output catalogs to the project you created in step 1 using the
olp resource project add CLI command.
Permissions outside of the project will not be impacted, that is, any pipelines that use these catalogs will continue to run.
Access to schemas associated to a catalog will be available in the project scope without any additional work required. However, you can also add schemas you created to a project using the
olp resource project add CLI command.
What if you make a mistake? Catalogs and schemas created outside of a project can be removed from a project in order to help you correct a mistake in this step of the process. Just use the
olp resource project remove CLI command. If the catalog or schema is shared to other projects before it's removed from a project, this action revokes its availability to the other projects. If the catalog or schema is shared and also linked to another project before it's removed from its home project, this action revokes those links.
Add members (users, groups, apps) to your project using the
olp project access grant CLI command or the platform portal.
To add a member to a project using the platform portal:
IMPORTANT: Add run-as ID (aka runtime credentials) for pipeline versions as member(s) of the project.
Full access to all resources in the project will be granted to these project members by default. You can optionally restrict access to resources by a project member by applying existing HERE or custom-created policies using the CLI. The CLI Projects Workflows explain how to set up granular access to the project resources through project policies. The Project Policy Commands allow you to create and manage custom policies.
Restrict access to catalogs by run-as ID (aka runtime credentials) if desired (optional), but make sure they have the minimum required rights of reading from input catalog(s) and writing to output catalog(s).
This step describes how to move a pipeline created outside of a project into a project. For batch pipelines, you may prefer to simply recreate the pipeline, cancel the versions outside the project, then activate the versions in the project, as described in steps 6-8 below.
Start by checking the pipeline versions associated with the pipeline you want to move to determine if the 5 most recent (the maximum supported) or fewer should be moved. You can do this check using the
olp pipeline version list CLI command.
Next, check to see if other pipelines are using pipeline templates used by pipeline versions you need. Use the
--report parameter of the
olp pipeline move CLI command. If they are, create separate pipeline templates, either for use in this pipeline or the other pipelines.
Now you're ready to move the pipeline, specified pipeline versions, and associated pipeline templates to the project using the
olp pipeline move CLI command.
IMPORTANT: The pipeline job will continue to run with the run-as ID (aka runtime credentials) using existing permissions to catalogs outside of the project from before the move. Verify the pipeline job is running as expected. The pipeline should be re-deployed with a new job to switch it to use the run-as ID (aka runtime credentials) and catalogs in project scope, and report the usage in project scope. To re-deploy, you can pause and resume the running pipeline version. You can also deactivate and activate the pipeline version if there is no processing state to be preserved.
olp pipeline move command revokes permissions attached to the group that was associated to the pipeline outside of project context, without impacting the permissions inside the project. This means you can now only access the pipeline through the project.
Pipeline versions that were not included in the move cannot be reactivated after the move, because their associated templates were not moved.
What if you make a mistake? Use the
--revert-to-group parameter of the
olp pipeline move CLI command to remove the pipeline, pipeline versions, and associated pipeline templates from the project and add them back to the group that they were in before the migration. Note that the rollback option is not available if any of the following are true:
Recreate batch pipeline(s) in the project (including pipeline template, pipeline, and first pipeline version) using the CLI or platform portal. Ensure that you specify project scope, regardless of the interface. If you use the platform portal and use the Pipelines tab instead of the Projects Manager, ensure that the project is selected in the drop-down menu in the top right-hand side of the screen.
Update automated deployment access to resources to be in the context of your new project by specifying the project in the
--scope parameter with CLI commands, in the
here.token.scope optional parameter of the credentials file, or by specifying a default project for your app using the
olp app scope CLI command. Read more about the order in which these three methods of specifying scope are resolved in the "Scopes" section here.
Once you have validated that your use case is working inside the project, you may choose to remove access to catalogs outside of the project context. This will ensure that access to those catalogs is now managed only within the context of the project. In order to remove permissions on the catalog outside of the project, use the
olp catalog permission list and the
olp catalog permission revoke CLI commands.