You can perform all of the steps presented in Deployment directly using the Pipeline API. The following series of sequence diagrams describes the low-level steps necessary to deploy and run a pipeline. Some additional operational tasks are also described.
The following sequence chart shows how pipeline deployment occurs at the API level.
Unlike the CLI or the platform portal deployment process, deployment by API requires that the pipeline JAR file be uploaded to the service as a separate step. On success, the service returns a package ID that uniquely identifies that JAR file within the service. The package ID is one of the template parameters required in the next step in the process.
The following sequence diagrams show the flow of deployment processes.
For batch pipelines, the currently running pipeline is not paused. Only the future scheduled ones are paused.
For a stream pipeline, it restarts from the state previously saved when paused. For a batch pipeline, it activates the scheduled runs.
The sequence diagram above shows the steps to create a new Pipeline Version and then upgrade the pipeline to run the new Pipeline Version. The POST operation that actually performs the Upgrade is asynchronous, so you need to use the GET operation to return the state and status of the Pipeline.
To perform an upgrade, the Pipeline Version to be replaced must be in a Running or Scheduled state. It cannot be upgraded if in a Ready state because, in that case, you would simply use another Pipeline Version to activate.
To upgrade a Stream Pipeline Version, the framework saves a snapshot of the current state of processing and uses the saved state to run the new upgraded Pipeline Version.
To upgrade a running Batch Pipeline Version, the currently running Pipeline Version is allowed to run to completion of its job before the upgrade can begin. Future scheduled jobs use the new upgraded Pipeline Version.
A Batch Pipeline Version with the on-demand mode cannot be upgraded from or upgraded to.
Like Submitting a Pipeline, Upgrading a Pipeline may take a short time to complete.