How to handle failure, perform cleanups and rollback in RapidDeploy
Job Plan Canvas
The Job Plan canvas, showing standard pipelines, as well as configured failure and rollback pipelines.
1. Atomic (transactional) deployment steps
2. Server Orchestration failure branches
On the Project Orchestration tab, the Orchestration map will start with one failure branch. This is the defaut setting. All tasks with ‘failOnError‘ set to true will, by default, subscribe to this failure branch, which simply fails the job.
Any task can add a new failure branch, or subscribe to any other failure branch. The example shown has two failure branches the default fail, and failure.
You can choose an existing failure branch, or add a new one by selecting the task and choosing from the drop-down in the ‘Failure branch‘ field or clicking the ‘+‘ button. You can add any tasks to the failure branch in order to accomplish the required cleanup.
Continuing with the example. If the schema update fails, here we add a failure branch with another move task to move the files back to their original location.
3. Job Plan Failure Pipelines
A Job Plan can consist of one or more standard deployment pipelines. Here we’ll see how we can also configure failure and rollback pipelines in the Job Plan.
If a pipeline fails during a deployment of a project target, we’ve so far seen how to take corrective measures on the target server(s) themselves. However, we might under some circumstances, want to run another job on the target to revert all the changes back to a previous version, or run some specific rollback orchestration.
To achieve this, we add failure pipelines on the Job Plan Edit canvas.
On the Job Plan Edit canvas, we can add a failure pipeline, or subscribe to an existing failure pipeline for any ‘Project Job‘.
We edit the ‘Project Job‘ and add or select a failure pipeline in the ‘Failure Pipeline‘ field.
The failure pipeline can run any project target combination in just the same way as a deployment pipeline. Typically you will run the same ‘Project Job‘ but selecting a specific version to roll back to. The special ‘ROLLBACK‘ version may also be selected, which returns the target to the previous successfully deployed version for this target.
You can see failure pipelines configured in this way in the main picture at the top of this page.
4. Rollback Pipeline
So far we have only considered failure scenarios. However there is also the possibility that all our deployment tasks succeed, only for us to discover that some piece of critical functionality in the deployed application is faulty. In this case we’ll need to either fix forward, or perform a rollback.
In order to perform a rollback, we configure a rollback pipeline. On the ‘Start‘ point of the pipeline, we double click to open the ‘Pipeline Edition‘ dialog.
Set ‘Rollback Pipeline‘ to ‘Yes‘ and Apply the change. A rollback pipeline is automatically configured with all of the original pipeline steps, but having all versions set to ‘ROLLBACK‘ You can edit this pipeline as required to create your preferred rollback scenario.
In the example at the top of the page, we’ve removed most of the default job steps and just configured our rollback to promote and then deploy the ‘ROLLBACK‘ version to our production environment.