Using the CLI or API, you can get a list of pipelines showing all of the pipelines that you have access to, their ID, and other relevant metadata. To do this in the UI, navigate to the Pipelines tab and find a list of pipelines in their various states, with a summary of relevant metadata for each. You can sort, filter, and search this list. An example is shown here.
To get further information on any of these Pipeline Versions, click on the pipeline name.
To upgrade a pipeline version, you replace it with a new pipeline version. The process for doing this using the CLI is described in the discussion of the Pipeline Lifecycle and applies to both batch and stream pipelines. Doing an upgrade from the platform portal is easier.
In the example shown in Figure 2, you can see four variants of the same Compute Bus Delays batch Pipeline Version. The goal here is to upgrade the Running Pipeline Version 3 to the Ready Version 4. Click on the Upgrade button for Version 4. This initiates the upgrade process that, when concluded, allows the next scheduled job to run with Pipeline Version 4.
The example in Figure 3 is a variation on the scenario in Figure 2. In this case, the goal is to upgrade the Running batch Pipeline Version 3 of Compute Bus Delays to the Scheduled Version 4. As in the previous case, clicking on the Upgrade button for Version 4 initiates the upgrade process. But the Upgrade button is not visible for V4. That is because V4 is in a Scheduled mode. You will have to use the Deactivate button to make the Upgrade function is available. Then, the Upgrade can be initiated. Another difference here is that Version 3 is running a batch job, so it will continue to run to the completion of that job. At that point, Version 3 is replaced by Version 4 and the next scheduled job runs with Pipeline Version 4. The alert message at the top of the screen confirms this action.
Hint: New Pipeline Version Required
Before you can upgrade an existing Pipeline Version, you must have a new Pipeline Version to replace it with. You can create this from the same pipeline JAR file, but you may want to create a new Template with modified parameter values.
In the next example (Figures 4 to 7), the goal is to upgrade the Running Stream Pipeline Version 3 of Compute Bus Delays to Version 4. Here are the steps to follow:
Step 1 - Pause the Running Pipeline Version by clicking on its Pause button
Although the upgrade process applies to both batch and stream pipelines, the behavior is somewhat different. When you pause a pipeline, a stream pipeline responds quickly and a batch pipeline can take some time to pause. This is because the batch pipeline only pauses when the currently running job is done. Thus, the delay to pause is primarily a function of the size and completeness of the currently running job. The stream pipeline just saves its location in the current job so it can resume at that point.
Step 2 - The Stream Processing is Paused
Step 3 - Click on the Upgrade button to start the upgrade
Step 4 - Upgrade Confirmed
When the running pipeline version was paused, the state of the current job was saved. Then, when the upgrade was complete, the job picked up where it left off. To avoid this behavior and start a new job, Cancel the job instead of pausing it and proceed as shown above.
The Admin Menu provides access to several management functions. Select a specific Pipeline Version from the versions list. Then, click on the ellipses at the far right side of that pipeline as shown below to open the menu.
View Pipeline Version Details
To view all of the information about a specific Pipeline Version, open its Admin Menu and select View Version Details. A modal information window opens that shows you all of the pipeline version configuration parameters, as well as any relevant meta information. Figure 11 shows an example of this window for a batch pipeline and Figure 12 shows an example for a stream pipeline.
Dismiss the display window by clicking on the "X" in the top right corner.
BYO data sent to HERE Workspace is private by default. All data sharing requires your explicit action, regardless of which interface you are working with.
Furthermore, PLM has both pipelines and pipeline templates. Both resources must be created either in a group or project, and cannot be in a group and a project at the same time. The following access rules apply to groups and projects:
If the pipelines and templates are created in a group, all members of the group have full access to all pipeline and pipeline template operations. Access to a pipeline automatically grants full access to all pipeline versions under that pipeline.
Members of the ‘Home’ project of a pipeline and pipeline template have full access to all pipeline and template operations, but only Project Admins can make pipeline templates available to be linked from other projects.
The only action allowed on linkable pipeline templates is ‘readResource’. Once the pipeline template has been made linkable, it can be linked, unlinked, and so on. Once linked to a project, pipelines in that project can use that pipeline template to create pipeline versions to activate and run them.
Copy a Pipeline Version
You can copy a Pipeline Version. It will be identical except that it will be assigned a new Pipeline Version ID. The procedure is described here:
Navigate to a specific Pipeline Version in the versions list.
Click on the ellipses button and select Copy as new version from the menu.
You will then see a Version Configuration Page with all of the parameters from the original Pipeline Version prepopulated. You can edit any parameter on the page, but some parameter combinations may initiate the creation of a new template.
You can create a new pipeline based on a single existing pipeline version. The procedure is described here:
Navigate to a specific Pipeline Version in the versions list.
Click on the ellipses button and select Copy as new pipeline from the menu.
You can then change the name, description, and share group before saving the new pipeline. Click Next when your changes are made.
A pipeline detail page similar to that when copying a Pipeline Version is displayed with a single version based on the original copied version. You can change any of the configuration items as needed for your new pipeline version. Click Save version when ready to commit your changes.
It may be necessary to completely remove a pipeline from the pipeline service. This is equivalent to deleting the pipeline instance and everything associated with it (pipeline JAR file, templates, Pipeline Versions). Perform this action as follows.
Note: Catalogs Not Included
This procedure does not affect any Catalogs loaded into the pipeline service.
Step 1: Go to the Pipeline List and select the Pipeline to delete
The pipeline must be in the Ready state.
Select the pipeline.
Click on the More menu.
From the More menu, click on Delete Pipeline.
Step 2: Confirm Pipeline Deletion
Warning: Irrevocable Action
This action cannot be reversed. The entire pipeline instance and everything associated with it will be removed.
Step 3: Completion status confirmation is displayed