Deploying a Pipeline via API
You can perform all of the steps presented in Deployment directly using the Pipeline API. The following series of sequence diagrams describes the low-level steps necessary to deploy and run a pipeline. Some additional operational tasks are also described.
Deploy a Pipeline
The following sequence chart shows how pipeline deployment occurs at the API level.
Unlike the CLI or the platform portal deployment process, deployment by API requires that the pipeline JAR file be uploaded to the service as a separate step. On success, the service returns a package ID that uniquely identifies that JAR file within the service. The package ID is one of the template parameters required in the next step in the process.
The following sequence diagrams show the flow of deployment processes.
Running a Pipeline
Canceling a Pipeline
Pausing a Pipeline
For batch pipelines, the currently running pipeline is not paused. Only the future scheduled ones are paused.
Resuming a Pipeline
For a stream pipeline, it restarts from the state previously saved when paused. For a batch pipeline, it activates the scheduled runs.
Upgrading a Pipeline
The sequence diagram above shows the steps to create a new Pipeline Version and then upgrade the pipeline to run the new Pipeline Version. The POST operation that actually performs the Upgrade is asynchronous, so you need to use the GET operation to return the state and status of the Pipeline.
To perform an upgrade, the Pipeline Version to be replaced must be in a Running or Scheduled state. It cannot be upgraded if in a Ready state because, in that case, you would simply use another Pipeline Version to activate.
To upgrade a Stream Pipeline Version, the framework saves a snapshot of the current state of processing and uses the saved state to run the new upgraded Pipeline Version.
To upgrade a running Batch Pipeline Version, the currently running Pipeline Version is allowed to run to completion of its job before the upgrade can begin. Future scheduled jobs use the new upgraded Pipeline Version.
A Batch Pipeline Version with the on-demand mode cannot be upgraded from or upgraded to.
Like Submitting a Pipeline, Upgrading a Pipeline may take a short time to complete.