The inherent linear sequencing in Waterfall creates walls and barriers to collaboration between different roles and disciplines in the organisation. One such wall, of particular importance, is the lack of ongoing collaboration and communication between business and development teams. Agile has tried to break down this barrier by bringing the two disciplines to continually work together throughout iterations. Further down the value delivery chain there is often a similar barrier between development and operations. The absence of tight collaboration between these two disciplines has been a major bottleneck, slowing down (and at times paralysing) timely delivery of software increments to customer. In the second decade of Agile, DevOps was born as a movement and a cultural shift to eliminate this very barrier.
DevOps, however, is more than just “dev” plus “ops”. It entails a shift in mind-set to embrace the culture of continuous improvement and continuous delivery of business value across management, engineering, product management, QA, test, release management, business, and basically throughout most of the organisation, just like Agile. That’s why I like to call it DevOps Transformation, to highlight the fact that DevOps is primarily about a transformation rather than tools and technology (more on DevOps pitfalls in my next posts).
Such a transformation affects everyone involved throughout the application lifecycle and is effectively an extension to the PDCA cycle to include every stage in the deployment pipeline:
Let’s take a quick look at the two core practices in DevOps transformation:
Agile manifesto stresses “working software” in two of its principles: “Deliver working software frequently, …” and “Working software is the primary measure of progress.” But how do we get to know if we are delivering working software? Continuous Delivery (CD) is the next natural step to Continuous Integration and is essentially the practice of determining whether you have a working shippable software increment by running the set of changes through a deployment pipeline (see below). This pipeline starts by continuous integration (CI) steps, Upon successful integraton it will then deploy the artefacts to a production-live environment and runs the full suite of automated acceptance tests against it. The pipeline is often triggered by any change to the code base. With this pattern, continuous delivery enables teams to ensure that every change can be released to production (hence working software) with minimal intervention at any point in time.
For continuous delivery to be effective, it heavily relies on automation throughout the pipeline in particular automated testing. It also utilises some of the Lean principles:
- Reducing waste (reducing batch sizes, limiting work in progress (WIP), reducing waiting time: Continuous delivery enables you to release smaller set of changes more frequently
- Shortening feedback loops: More frequent deployments made possible through continuous delivery will in turn enable faster feedback cycle (more on this later)
Continuous Improvement is the culture of inspecting and adapting, continual validated learning through experimentation and real data (feedback). It is built on many concepts of Lean: shortened feedback loops, Kaizen, and data-driven decision making. Although the practice of continuous improvement can and does exist in many Agile teams, coupled with continuous delivery teams will have access to an extended set of opportunities to learn better and faster through the empirical data and metrics collected throughout the deployment pipeline as well as ALM processes. Some examples of such metrics include:
- Core continuous delivery metrics: deployment frequency, deployment lead time, and mean-time to recovery.
- Production monitoring providing visibility into operation state of your production environment with metrics on traffic, application performance, availability, scaling, etc.
- Customer feedback: real time usage data collected provide feedback on how user interacts with the application as soon as a new version of the application is released.
- Automated test data collected through CI/CD pipeline, such as test coverage, failure rates, etc.
- Software quality metrics gathered via automatic code analysis done with CI
In my next blog post I’ll explore the benefits and some of the elements of DevOps transformation