Introduction
Argo graduated– What a lovely holiday season gift from CNCF to platform engineers and DevOps folks! The graduation means the Argo project has met the high standards of a matured project and can be used directly in production without many hiccups. So far, Argo has seen healthy growth and adoption (350+ users are using it in production) and has met high security and compliance audits. Many organizations use Argo to deploy and run their cloud applications on Kubernetes using the GitOps method.
Although Argo is a lightweight tool, enterprise adoption can face a few challenges. In my previous blog, I discussed the lifecycle management challenges of Argo. In this blog, I will discuss a list of Day-N software delivery challenges while using Argo CD- when the application team and DevSecOps team would like to verify software during the production deployment and ensure compliance while deploying applications using Argo.
Day-N Software Delivery Challenges while using Argo CD
In large production systems, DevOps and platform engineers will Argo CD to deploy cloud-native apps into production Kubernetes clusters. The below image (Fig A) represents multiple instances of Argo CD used for deploying in various applications across numerous K8s clusters.
There can be multiple developer teams for developing business solutions and deploying them into multiple clusters. Developers can use various tools and practices for their deployment. For example, app1 is getting deployed using manifest files in Argo, whereas apps 2, 3, and 4 can be deployed using HELM charts in Argo.
Ensuring risk-free delivery and adhering to organizational compliance in the GitOps method remains challenging for DevOps engineers, project managers, and platform teams.
We will discuss the challenges to controlling software delivery risks and security.
Lack of visibility of all clusters and apps in software deployment process
At the start of the GitOps initiative, Argo is used to deploying several applications. But when platform engineers scale the use case for multiple applications, visibility becomes a challenge for the project or product managers to make faster decisions. Tracking and visualizing what apps are getting deployed into which cluster daily or weekly is a clumsy process.
Argo CD only provides a unified view of deployments across some clusters or various team (or project-level) views of deployments. So, DevOps and platform engineers have to log in to each Argo instance, Git repo, and K8s cluster and manually gather data about app deployment status and each app’s health.
Lack of visibility into risk analysis and deployment verification
While deploying apps into Kubernetes using canary or blue/green rollouts, the DevOps team would like to identify the risk of newly deployed software. Software verification is essential before progressing software into production entirely. Although Argo Rollouts CRDs provide functionalities to implement deployment strategies, they don’t offer any sophisticated risk estimation techniques to verify if the new software version is good to go.
Here are the two common challenges that SREs usually face while estimating the risk of a canary:
A. Argo Rollouts provides integration with open-source Kayenta for metric analysis but does not provide integrations to perform quality regressions through log analysis.
B. Triaging issues of a new deployment are manual and time-consuming without the proper mechanism for the abnormal behavior of a canary. Monitoring tools will provide just the metrics data and logs, but event analysis and correlation have still to be done manually to calculate the actual risk of a new release.
Implementing deployment security controls can be hard
Any enterprise software delivery process would have various SDLC policies to mitigate security and compliance risks before a code is released into production. Policies are implemented to make software delivery processes compliant with industry best practices and standards such as GDPR, HIPAA or PCI, etc. DevSecOps team and security architects need help to learn yet another technology to implement security controls while deploying apps into Kubernetes continuously.
OpsMx ISD for ArgoCD: Enabling multicluster and multicloud visibility and controls
OpsMx Intelligent Software Delivery for Argo offers the power of GitOps-based deployments with ease and minimal learning curve, along with AI/ML-driven automation for approvals, verification, security checks, and compliances for comprehensive visibility, audibility, and fine-grained control on your continuous delivery (CD) process, with no vendor lock-in.
OpsMx ISD for Argo offers enterprise features to enhance visibility and controls for scalable and secure software delivery. Our integrated AI-based automation, security, visibility, and governance capabilities result in standardized and repeatable workflows and streamline the management of Argo and app deliveries.
Multicluster and Multicloud deployment visibility
ISD- Visibility and Audit module provides project managers and the DevOps team with a single pane glass for multicluster and multicloud deployments. In a single view, stakeholders can now understand who is deploying what into which clusters without the need to login into each K8s cluster. Using Argo CD APIs, ISD also provides each deployment’s health in a single dashboard.
The below image (Fig B) highlights the central dashboard for the deployments in an enterprise. The dashboard has the total number of applications developed and deployed by developers, the current status of their deployments, the health of applications, and out-of-sync status.
Enterprise-wide software deployment metrics (DORA Metrics)
OpsMx ISD provides the metrics that capture the effectiveness of the software development and delivery process. ISD provides DevOps Research Assessment (DORA) metrics (check out the image Fig C) to identify the throughput and stability of the software delivery process. Project managers will get the following metrics:
- Deployment frequency to understand how often code is getting deployed into production and determine the speed and agility of the software development team.
- The lead time of code changes from check-in to release, highlighting the fastest and slowest deployments.
- Applications or K8S clusters with more active GitOps deployments.
Deployment verification and risk analysis
ISD for Argo integrated with Argo Rollouts to ensure your deployments are safe and risk-free. Once the traffic between a new release (say Canary) and a baseline release is split using ingress, ISD will consume the metrics and logs from tools such as Prometheus and Splunk. (You can read more about deploying Canary using Argo Rollout.)
ISD will apply AI/ML to the data (refer to Fig D) and determine the confidence level to roll out the new software to 100% of the traffic. ISD for Argo provides several benefits, including:
- Accurate determination of risk (refer to Fig E) of the latest software in production using logs and metrics.
- Faster training with correlated events, errors, and warnings based on the root cause.
- More immediate rollback to older stable versions quickly in critical situations.
Security and Compliance Control in GitOps
ISD for Argo helps DevOps and the security team to create policies and implement security and compliance controls in the GitOps-style delivery environment. ISD has integrations with Open-Policy Agent (OPA), using which it can execute and validate SDLC policies. Standard compliance policies as implemented by large and matured IT organizations are:
- Configuring deployment freeze time or blackout window,
- Failing a deployment if approvals are not in place, etc.
- Pre-deployment checks like container images should be UB8 images, TLS 3.0 or Load balancer ports to be run in certain ports; images should not have CVEs, images deployed must have passed manual/automated tests, etc.
Using ISD for Argo DevSecOps and security managers will find it easy to configure deployment policies into GitOps based Argo CD.
Next Step
ISD provides a holistic approach to managing modern infrastructure by dramatically simplifying the lifecycle management of Argo located across Kubernetes clusters. ISD for Argo also offers enterprise-grade features to overcome challenges emerging from day-n of GitOps-style deployment of cloud-native applications.
Let us know in the comments if the blog was helpful. And in case of any doubts about configuring Argo CD for production, you can reach us at the Argo Center of Excellence. Or if you want to resolve any open vulnerabilities while adopting Argo, please reach out to 24*7 support for Argo.
About OpsMx
Founded with the vision of “delivering software without human intervention,” OpsMx enables customers to transform and automate their software delivery processes. OpsMx builds on open-source Spinnaker and Argo with services and software that helps DevOps teams SHIP BETTER SOFTWARE FASTER.
0 Comments