Continuous Delivery: It’s All About the Pipeline

Author: John Wetherill

The Agile Manifesto has had significant impact on the way software is built. It defines twelve fundamental principles, the first of which is “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

Continuous Delivery is explicitly mentioned in this principal, and now, 15 years after the manifesto’s formulation, the practice is hitting the mainstream.

Last month I had the pleasure of joining Continuous Delivery experts Marco Abis, founder of HighOps, and David Farley, Owner of CD Inco, and author of The Book on CD, for a fascinating webinar on the topic of Continuous Delivery, and how it relates to successful production of cloud software. This blog captures parts of the conversation.

Continuous Integration vs. Delivery vs. Deployment

We kicked off the webinar with some baseline definitions.

Continuous Integration is the process of ensuring a build is in a working state, satisfying developers that it’s ready for production. From a source code management (SCM) point of view, this generally means that all development work is constantly merged with the master branch, allowing the full unit test suite to be run against the “finished” product at every check-in.

Continuous Delivery is the “last mile” in the software lifecycle, and is the child of Continuous Integration, ensuring that a build is always in a releasable state. This is much more than integration with trunk and running of unit tests: the software is actually provisioned to a full, production-like stack, and delivered to some set of end users, whether that be QA, the product team, or a selected group of end-users. Continuous delivery goes hand in hand with a DevOps approach, where the team is responsible for all aspects of software delivery

Continuous Deployment If Continuous Delivery is the last mile, Continuous Deployment is the last inch. The entire process, from check-in to production, is fully automated, with no human intervention.

The Pipeline

The foundation of continuous delivery is the deployment pipeline, which is the path that a code change takes from check-in to production. The term pipeline as it relates to CD was actually coined by David, who says that the job of the deployment pipeline is to prove that the release candidate is not fit for production. You can never prove it’s fit – this is theoretically and practically impossible, but if you have one test or validation failing it proves it’s not fit for production.

At the head of pipeline is the development team working on software. Each commit gives birth to a release candidate which flows through the pipeline. If it makes it through to the end, it’s ready for release. Ideally, as Marco says, the release process itself should be as boring as possible: effectively a YES/NO decision based on a successful pass through the pipeline, and a button that someone can click to choose a release candidate and deploy it.

The goal here is to increase visibility of changes from check-in to production, while at the same time:

1. find and remove bottlenecks
2. shorten the feedback loop
3. automate as much as possible
4. eliminate any error-prone manual processes
5. optimize and visualize what’s going on to improve the flow

What’s in the Pipeline?

Almost anything. At the head end, as part of continuous integration side, unit tests are an expected component. But the pipeline consists of much more than unit tests, including

• regression tests
• integration tests
• UI tests
• performance tests
• latency tests
• scanning for OWASP top 25 vulnerabilities
• general security scanning, as for example offered by BrightPoint Security
• accessibility compliance to ensure sur app is usable by people with disabilities see this blog for example
• time-travel tests to manipulate time for long running scenarios
• time event tests, such as millennium rollover, daylight savings
• general event simulation
• traffic flow tests
• tests for specific hardware and OS versions
• tests for dependencies, other software versions, OS changes
• failure tests: selectively destroy bits of application as in Chaos Monkey
• memory leak detection such as provided by plumbr
• regulation and compliance tests (for example automated PCI compliance scans)
• app-specific analytics
• performance tests
• explorability tests
• usability tests

As you can see, continuous delivery isn’t just about unit tests.

Quick and Easy and Cheap First

It’s best to optimize the process as much as possible. It doesn’t make sense to invoke long-running tests, or tests that are less likely to fail, at the front of the pipeline. Developers need quick feedback as to whether the change is a fit, so that the tests most likely to fail do so quickly. Start with unit tests, potentially thousands or tens of thousands of these. After that move on to deeper and more expensive tests. Any tests requiring manual intervention should be at the end of the pipeline, as human intervention is costly.

What can’t be Automated in the Pipeline?

Full article: activestate.com/blog/2015/06/continuous-delivery-its-all-about-pipeline