6 Pitfalls to Avoid while Implementing Continuous Delivery

Continuous Delivery (CD) is not a new concept anymore. Enterprises across the globe are practicing continuous delivery in different capacities. Through continuous delivery, organizations can empower their developers to focus on writing and committing code and shipping the values into customers’ hands speedily and sustainably. 

Let me repeat it; Continuous Delivery(CD) is a way of delivering software speedily, safely, and sustainably. Read more in detail about what is Continuous Delivery. Thousands of organizations are using CD, yet the majority have not championed the art of delivering change speedily, and safely. As per the state of DevOps 2019, the survey response by more than 1000 engineers/DevOps or SREs/ITOps Managers suggests that nearly 60% of the organizations are still medium and low performers.

Medium and Low Performers as per State of DevOps 2019

In this blog, we will confide all the pitfalls to avoid while implementing continuous delivery in your organization.

We began this research with a set of questions: Why are some enterprises not reaping expected benefits out of their continuous delivery initiative? Is it a tool problem? Is the interoperability between technologies to be blamed? Is it challenging to adopt the culture? Or is it the complex monolith applications which don’t respond well to CI/CD?

After spending a month on the research, we have observed that DevOps managers’ inadequate skills and knowledge and their myopic view about CI/CD technologies are the two primary factors that contribute towards a non-effective continuous delivery process. For our research, we have borrowed heavily from OpsMx Sales intel and ascertained our hypothesis referring to the latest reports on DevOps upskilling report 2020 and State of DevOps 2019). So the following are the six pitfalls all DevOps managers and engineers should avoid to deliver software speedily, safely, and sustainably.

1. Stop interpreting CI/CD as [CI or CD], [CI & CD], [CI ->CD]

In case you are under a common impression that continuous delivery(CD) is a logical extension of continuous integration(CI), then it is not. Tracy Miranda, Executive Director of Continuous Delivery Foundation, describes CI/CD as a misleading term. Industry leaders like Jez Humble too vehemently disagree with the term. The central idea is Continuous Delivery is a set of practices that enable the business to deliver software changes quickly, frequently, and safely to users. Continuous integration is a development practice to help developers code into SCM often i.e build automation, and it is just one of the practices under the CD umbrella. Other practices are test automation, deployment automation, release process automation, deployment strategies, security and compliance, automated verification & approvals, and metrics and dashboards for constant feedback.

2. Stop Using Jenkins for Continuous Delivery

For the love of God, please stop using Jenkins for continuous delivery. Jenkins is the best continuous integration tool in the market, which provides a rich set of plugins. However, Jenkins is not meant for continuous delivery. The concept of continuous delivery is to empower developers to focus on the innovation of new products. Still, if you are using Jenkins for continuous delivery, then your developers are writing scripts for creating deployment pipelines. And “continuously” editing and “continuously” maintaining your deployment scripts are the strategies that would kill the developer’s productivity.

If you are a DevOps manager, you would know that Jenkins is not the right platform for multi-cloud deployments. For instance, your developer needs to write custom scripts and add plugins to deploy applications into AWS/Azure/GCP or bare metal. You would require more scripts to create load balancers or resize clusters for microservice-based deployments—even more scripts for deployment strategies (blue-green, canary, rolling updates, and Highlander). Read more about the problem of using Jenkins here.

To implement continuous delivery at your organization, use open-source solutions such as Spinnaker, a multi-cloud continuous delivery platform for releasing software with high speed and confidence.

Learn more on how to deploy to Kubernetes using Spinnaker

3. Stop treating security and speed of Continuous Delivery as mutual exclusive agendas

Often DevOps managers, release managers, and product managers either focus on delivery speed to adhere to business schedules at the expense of security. And ultimately, introduce more risky software into production. Contrarily, they would compromise on speed while giving more priority to security and compliance. However, this notion of treating speed and security as separate items is wrong, and you are not practicing continuous delivery.

The 2019 State of DevOps Report by Puppet, CircleCI, and Splunk ( they have surveyed 3000 IT and engineering associates), suggests that security and compliance must be treated as an integral part of the software delivery lifecycle. This will enhance collaboration between the security and software team, increase the organization’s overall security posture, and help the dev and ops team release software with confidence.

Treat security, compliance, and audit capabilities as a priority along with implementing a continuous delivery process. Below are some of the top pointers you must ensure while working with the security and compliance team:

1. Secrets Management: CD pipelines are vulnerable as they are targeted by attackers who can snoop and access credentials. There should be a mechanism to integrate CD tools with secret management systems like Hashicorp Vault to store Keys, APIs, Tokens, certificates, and secrets.
2. Identity Access Management: Appropriate mechanisms to ensure authentication and authorization of your continuous delivery process and tool are eminent. Your continuous delivery tool should follow protocols such as RBAC, Oauth 2.0.
3. Integration with 3rd party tools: Make sure your Continuous Delivery pipeline is compatible with SAST and DAST scanner. You can automatically perform code analysis and find the security vulnerabilities of your release while delivering the application. Learn more about CD security.
4. Compliance and Audit: Ensuring compliance of continuous delivery processes to various industry standards is a must. Companies often make headlines for whooping penalties due to non-adherence to industry standards such as PCI, HIPAA, GDPR, and SOX. Automated policy-enforced pipelines will act as gates and guardrails to prevent developers from inadvertently introducing software risks. Read more on Why and How to Make CICD Pipeline Compliant and Auditable.

4. Stop your team from spending hours on risk analysis and approvals of your Continuous Delivery pipelines

Companies that have implemented continuous delivery are concerned about their speed of delivery. They feel their delivery time is still not less as expected. The main reasons are their manual approval time takes at least 3 hrs per release. Besides approval time, assessing the risk of releases manually in each stage of continuous delivery doubles the delay.

Risk assessment is challenging for three reasons:

1. Risk assessment of a release in various stages are different and carried out by separate stakeholders. ( The developer will do a risk assessment of the build. Similarly, the tester would assess test scenarios, and ops will judge a newly deployment if it good to go into production or requires rollback)
2. Dev, Test, and Ops need contextual knowledge for diagnosis and triage of issues.
3. Data from various tools need to be gathered and correlated to derive insights. This is very difficult for the human eye.

Approval would take all the time because:

1. Approvers lack 360-degree visibility.
2. Lack of fine-grain policies to scale to 1000 apps and users.

Time Taken for Approvals and Risk Assessment in Continuous Delivery process

Implementing continuous verification using Autopilot, which can integrate with your existing CICD tools, will automate manual non-core activities. Automated verification tools use AI/ML algorithms to find out the risk of releases. It can take an automated decision to progress your release to your pipeline’s next stage based on the risk reports or canary analysis.

Autopilot for Automated Decisioning in Continuous Delivery Process

Software like OpsMx Autopilot helps many enterprises identify performance and quality regression in build, test, and deployment stages. It seamlessly integrates with existing APM tools like ( Dynatrace, AppDynamics, Prometheus, New Relic, etc.) and Log Analyzers (Sumo Logic, Splunk, Loggly, etc.) metrics and log data and performs automated risk analysis.

5. Stop being a digital dinosaur

According to a Forrester report titled The Future of Business is Digital, by Nigel Fenwick and Martin Gill, it is essential to regularly harness technologies for improving agility and operational efficiency. Otherwise, companies will be lost like prehistoric Dinosaurs. Implementing continuous delivery(CD) process will not be enough; one has to measure the KPIs and continually upgrade. Lack of essential metrics and dashboards hurt the CD process and enterprise. All the team members, including developers, testers, ops, senior management, and key stakeholders, must have clear visibility and a single source of truth. Different team members have other priorities and need different metrics to monitor, measure, and improve their CD initiative’s effectiveness. There must be a progressive assessment of releases in your CD dashboard, which will be useful for DevOps leaders and dev, test, ops team, SREs, and auditors. Some valuable metrics to consider for the groups are:

• • Number of deploy-ready and failed builds
  • Number of Bugs, Number of failed test scenarios
  • Number of application deployed into production per day
  • Deep insights about CD pipeline execution
  • The health of a newly deployed application
  • Lead time between code commit and production, or between code commit and feedbacks after validation
  • Meantime between build failures, bug resolution, and recovery
  • Production downtime during deployments
  • Mean Time to Resolve ( MTTR)

6. Stop tinkering with monolithic systems

A report by Atlassian suggests that more than 70% of respondents (engineers, DevOps practitioners, and CTOs) agree microservices being easy to test and deploy features. Not that continuous delivery tools don’t support monoliths, they do. However, for starters, it is advisable to use microservices for continuous delivery.

A few DevOps managers start with monolithic systems. It will be complicated because there will be thousands of interdependent applications using legacy technologies. It may be time-consuming to set up an automated deployment process for monoliths and perhaps not scalable.

If you plan to start your continuous delivery(CD) journey or have already commenced one, we would like you to be aware of the points to make your CD useful. After all, to set up continuous delivery, companies invest thousands of dollars, pour in tremendous effort in upskilling, and set high expectations. Hence avoid the pitfalls mentioned earlier and deliver software speedily, safely, and sustainably.

OpsMx is a leading provider of Continuous Delivery solutions that help Fortune 500 companies safely deliver software at scale and without human intervention. We help engineering teams take the risk and manual effort out of releasing innovations at the speed of modern business. For additional information, contact us

• Atlassian Report : https://jaxenter.com/atlassian-development-trends-microservices-155748.html
• Future of Digital by Forrester : https://www.forrester.com/report/The+Future+Of+Business+Is+Digital/-/E-RES115520#