Select Page
by

Vardhan NS

|
last updated on April 25, 2023
Share

Spinnaker has multiple components ( or microservices) like Orca, Igor, Clouddriver, and so on. Each component has a specific role. While deploying or installing Spinnaker in Kubernetes, each component is installed separately. 

Enterprises often need to use custom Docker container images to specify their project-specific dependencies and incorporate additional functionality, and features that are not part of the open-source Spinnaker. But while using custom images to install Spinnaker, often a common issue faced is that the pods of that installed Spinnaker service fail to execute and run, as the “readiness probe” (Kubernetes health check probe for pods) fails. 

Here in this blog, we will discuss how OpsMx has found a way to resolve this issue and get you going with your Spinnaker deployments.

Download Advanced deployment strategies

Issue

When we use a custom image for microservice in Spinnaker, the pods of that service will not come up as the readiness probe fails since the custom image may not have the tools like netstat, wget, or curl.

Here, we have a custom docker container image for the Spinnaker’s Orca microservice for which the readiness probe fails. 

For e.g. : 

Custom image used:

				
					artifactId: docker.io/opsmxdev/orca:1.22.2-pr3989-40
kubectl describe po spin-orca-fff44f5f4-8j6zs
				
			

Output:

Readiness probe failed

Steps for resolution

The following steps will help you in resolving the issue as discussed above you can use and implement this resolution very easily. Execute the steps as given below:

1. Manually remove readiness probe from that service’s deployment as below:

				
					kubectl edit deploy spin-orca
				
			

Remove below:

				
					readinessProbe:
exec:
command:
- wget
- --no-check-certificate
- --spider
--q
- https://localhost:8083/health 
failureThreshold: 3
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
				
			

2. Verify the new pod that comes up with the custom image.

				
					kubectl describe po spin-orca-fff44f5f4-8j6ab
orca:
Container ID:   docker://
05bdc0f4122ff67ef928497cfdebc4ac78de86763fc36ba6855a64b9be0e55f2
Image: docker.io/opsmxdev/orca:1.22.2-pr3989-40
				
			

3. Now execute the custom image in the new pod.

				
					kubectl
-n spinnaker exec -it spin-orca-86df845bf6-c7sk6 -- bash
				
			

4. Verify the ‘wget’ tool used for the health check probe. 

				
					wget
Output:
wget: missing URL
				
			

5. Do the health check manually in the pod.

				
					wget
--no-check-certificate --spider -q --private-key /pkcs12/tls.key --certificate /pkcs12/tls.crt https://localhost:8083/health
				
			

6. Change the health check command accordingly in the service.yaml

				
					artifactId:
docker.io/opsmxdev/orca:1.22.2-pr3989-40
scheme: https
kubernetes:
useExecHealthCheck: true
customHealthCheckExecCommands: ["wget", "--no-check-certificate", "--spider", "-q", "--private-key", "/pkcs12/tls.key", "--certificate", "/pkcs12/tls.crt", "https://localhost:8083/health"] 
overrideBaseUrl: https://spin-orca.devmt.isym.io:8083
healthEndpoint:
				
			

7. Deploy Halyard

				
					hal deploy apply
				
			

Note: Do not use –service-names=orca, as the monitoring daemon. It is causing issues with individual updates.

Conclusion 

OpsMx offers Intelligent Software Delivery (ISD) for Spinnaker to accelerate approvals, verification, and compliance checks by analyzing data generated across Spinnaker pipelines and the software delivery chain.  ISD for Spinnaker is available as SaaS, managed service, or on-premises. 

About OpsMx

Founded with the vision of “delivering software without human intervention,” OpsMx enables customers to transform and automate their software delivery processes. OpsMx builds on open-source Spinnaker and Argo with services and software that helps DevOps teams SHIP BETTER SOFTWARE FASTER.

Vardhan NS

Vardhan is a technologist and a marketing professional, currently working as a Sr. PMM at OpsMx. His strength lies in understanding complex technologies, and explaining them in un-complicated ways. Vardhan is a passionate Product Marketer with a keen focus on Content, helping brands Position themselves uniquely with clear messaging and competitive differentiation. Outside of work, he is an athlete that is passionate about Football, Swimming and Surfing.

Link

1 Comment

  1. Syed Kalam

    Very useful

    Reply

Submit a Comment

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.