Select Page
by

Gopal Jayanthi

|
last updated on September 3, 2021
Share

Objective:

In our previous blog we have showcased how to authenticate using  LDAP. But there are customers using open source Spinnaker for production usage and want even more security. And we help them securing their application even more, with the help of mutual TLS (mTLS) as it is based on certificates.  mTLS is provides additional security because application requires private keys to be shared by two parties during communication. 

In this blog, we shall show you how to share certificates between Spinnaker and  LDAP  servers. In other words, how to install LDAP with mutual TLS. Please be noted that if we set up MTLS  for one Spinnaker instance, it will be used for that particular Spinnaker instance only. 

Prerequisites for setting up MTLS in Spinnaker

  1. Spinnaker must be installed in Kubernetes cluster. 
  2. Helm 3 Charts: By using HELM 3 charts, we are installing LDAP, and later we will share the certificates between LDAP and Spinnaker

In order to produce certs for Spinnaker and LDAP server, follow the six steps:

1.  Installing cert-manager 

Step 1.A 

Run the command to fetch cert-manager from Github of cert-manager and apply into Kubernetes cluster 

 kubectl apply –validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.16.1/cert-manager.yaml 

 For Kubernetes version less than 1.15 we need follow below steps:  

 kubectl apply –validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.16.1/cert-manager-legacy.yaml

Alternative to Step-A: If you are using HELM repo, then you follow the below steps:

Step 1.B:

Cert-manager installation in the created namespace in Kubernetes ( in our case it is cert-manager)

( point to note: In a cluster, there can be multiple namespaces. And you can have you Spinnaker installed in one namespace and can have certificated installed in same/another namespace) 

helm repo add jetstack https://charts.jetstack.io

helm repo add stable https://kubernetes-charts.storage.googleapis.com

helm repo update

kubectl create ns cert-manager

helm install certman --namespace cert-manager jetstack/cert-manager

 2. Configuring issuer:

Objective: We are using self-signed certificates to get our certs to be trusted by both the parties ( Spinnaker and LDAP)

Step 2.A:

Create a yaml file with the content and apply to the Kubernetes cluster

File name: clusterissuer.yml

         apiVersion: cert-manager.io/v1alpha2

           kind: ClusterIssuer

             metadata:

                      name: selfsigned-issuer

           spec:

                     selfSigned: {}

Step 2.B:

Applying the created yaml with the content to K8S cluster

kubectl create -f clusterissuer.yml

kubectl  get clusterissuer

Output should look something like the below:

 NAME                  READY    AGE

selfsigned-issuer    True      10s

 3. Configuring LDAP in Kubernetes Cluster:

Step 3.A:

Creating namespace by using the below command:

  kubectl create ns <<LDAP-NAMESPACE>>

Output should look something like the below:

namespace/ldap created

Step 3.B:

Installing cert in LDAP namespace

StepB1: Create cacert.yml to get the certificate generated.

 File name: cacert.yml   (edit commonName & dnsNames)

apiVersion: cert-manager.io/v1alpha2

kind: Certificate

metadata:

         name: hello-ca-tls

         namespace: <<LDAP-NAMESPACE>>

spec:

         # name of the tls secret to store  the generated certificate/key pair

         secretName: cacert

         isCA: true

         issuerRef:

                       # issuer created in step 1

                    name: selfsigned-issuer

                       kind: ClusterIssuer

         commonName: "<<LDAP-NAMESPACE>>.svc.cluster.local"

         dnsNames:

              # one or more fully-qualified domain name can be defined here

              - ldap-openldap.<<LDAP-NAMESPACE>>.svc.cluster.local

              - localhost

Applying the configured yaml file in the LDAP namespace    

    kubectl create -f cacert.yml

    kubectl -n <<LDAP-NAMESPACE>> get cert

Output should look something like the below:

NAME           READY   SECRET   AGE

hello-ca-tls   True cacert        23s

 Step B2: To know the secret created or not we can use the following code

kubectl -n <<LDAP-NAMESPACE>> get secret

Output should look something like the below:

NAME              TYPE                         DATA   AGE

cacert            kubernetes.io/tls                 50s

Step 3.C:

Setup the issuer to get our certs to be trusted one.

Step 3.C.1:

Create a yaml file to create an issuer to produce authentication verification to our generated certs in the previous stage.

 caissuer.yml

      apiVersion: cert-manager.io/v1alpha2

         kind: Issuer

         metadata:

     name: caissuer

     namespace: <<LDAP-NAMESPACE>>

         spec:

ca:

              secretName: cacert

To apply issuer use the following command

      kubectl -n <<LDAP-NAMESPACE>> create -f caissuer.yml

        kubectl -n <<LDAP-NAMESPACE>> get issuer

Output should look something like the below:

NAME       READY   AGE

caissuer   True 17s

Step 3.D:

Create LDAP mtls certificates using the below files and apply in the kubernetes namespace ( where we will install LDAP)

          ldapmutual.yml (Edit commonName and dnsNames)

         apiVersion: cert-manager.io/v1alpha2

            kind: Certificate

            Metadata:

                 name: ldapmutual

                 namespace: <<LDAP-NAMESPACE>>

            spec:

                 secretName: ldapmutual

                 duration: 2160h # 90d

                 renewBefore: 360h # 15d

                 commonName: ldap-openldap.<<LDAP-NAMESPACE>>.svc.cluster.local

                 dnsNames:

                 - ldap-openldap.<<LDAP-NAMESPACE>>.svc.cluster.local

                 - localhost

                 - '*.<<LDAP-NAMESPACE>>.svc.cluster.local'

                 usages:

                 - digital signature

                 - key encipherment

                 - server auth

                 - client auth

                 issuerRef:

        name: caissuer

        # We can reference ClusterIssuers by changing the kind here.

        # The default value is Issuer (i.e. a locally namespaced Issuer)

        kind: Issuer

Apply the below commands to create mtls certs   

   kubectl create -f ldapmutual.yml

  kubectl -n <<LDAP-NAMESPACE>> get certs

Output should look something like the below:

NAME           READY   SECRET   AGE

hello-ca-tls    True cacert   11m

ldapmutual     True ldapmutual   14s

Check if the secrets are created or not:

     kubectl -n ldap get secrets

NAME              TYPE                                          DATA   AGE

cacert                        kubernetes.io/tls                                    3      12m

default-token-d5cjz   kubernetes.io/service-account-token   3      16m

ldapmutual        kubernetes.io/tls                                     3      53s

 4. Install LDAP

In the Values.yml file we can override the default values ( like username, passwords, features) by LDAP HELM Charts

You can quickly download the file from the below link: https://github.com/helm/charts/blob/master/stable/openldap/values.yaml 

Note: Please do enable the tls-enabled as true in the values.yaml

Step 4.A:

Use the below command to install LDAP

helm -n <<LDAP-NAMESPACE>> install ldap stable/openldap -f values.yml

Step 4.A.1:

Now check if the Kubernetes pods and services are up or not

kubectl -n <<LDAP-NAMESPACE>> get pod,svc

kubectl -n <<LDAP-NAMESPACE>> get pvc    

Please note: The pod and pvc creation may take 5 or 6 minutes. 

Step 4.A.2:

You can understand the details of the pod creation use the following command ( and yes you can certainly ignore the warning messages)

kubectl -n <<LDAP-NAMESPACE>> describe pod 

Step 4A3: Connect into the LDAP pod to check LDAP connection

kubectl -n <<LDAP-NAMESPACE>> exec -it <podname> bash

ldapsearch -H ldap://localhost:389 -x -D "cn=admin,dc=example,dc=org" -w $LDAP_ADMIN_PASSWORD -b "dc=example,dc=org"  dn memberof

5. Verify connection using tls

Follow the step to check if LDAP is connected with tls or not

Step 5.A:

Fetch the secret created (ldapmutual)  in the previous steps

kubectl -n <<LDAP-NAMESPACE>> get secret ldapmutua

There will be 3 files ca.crt, tls.crt, and tls.key. Copy those certs from the secret ldapmutual into the client at /tmp directory or local machine (for me it is /decrypted-secrets/)

Note: Use “base 64 decode” for  ca.crt, tls.crt and tls.key to decode the files

       cat /tmp/ca.crt | base64 –decode >> /decrypted-secrets/ca.crt

        cat /tmp/tls.crt | base64 –decode >> /decrypted-secrets/tls.crt

         cat /tmp/tls.key | base64 –decode >> /decrypted-secrets/tls.key

         cd /decrypted-secrets

Below command will create a sharable pkcs12 type file with encrypted secrets. ( here we  have mentioned the output file name to be tls.p12)

openssl pkcs12 -export -clcerts -in tls.crt -inkey tls.key -out tls.p12 -name tls

Output will look like below ( you can use any password)

       Enter Export Password:

      Verifying – Enter Export Password:

To check if the file tls.p12 is created or not

 ls –ltra

 6. Copy the certs from the Spinnaker services from Gate microservice

Step 6.A

Copy the file  “/etc/ssl/certs/java/cacerts “ from Gate pod of Spinnaker into the local directory (/decrypted-secrets)

Note: Here the path /etc/ssl/certs/java/cacerts might be different please use the respective path by checking the path in the gate pod.

Step 6.B

Import the secrets (ca.crt) which we have got from LDAP certs

keytool -import -alias custom-ca -keystore cacerts -file ca.crt 

 Step 6.C

Now let us create tls.jks file to place in the Spinnaker services

keytool -importkeystore -srckeystore tls.p12 -srcstoretype pkcs12  -destkeystore tls.jks -deststoretype pkcs12

  Step 6.D

Till now we have created two certs: one from Gate pod in Spinnaker and another from the LDAP. By using those two we are creating secrets. 

Below is the command to create a secret from the two certs

kubectl -n <<namespace-spinnaker>> create secret generic ldaptls --from-file tls.jks --from-file cacerts

 Step 6.E

Connecting to the halyard pod to place our encrypted secret created in the above command.

kubectl exec -it <<halayard-pod>> bash -n <<spinnaker-namespace>>

In the halyard pod, we are modifying changes to direct the Spinnaker service to use our certs.

File path: ~/.hal/default/service-settings/gate.yml 

healthEndpoint: /health

kubernetes:

     useExecHealthCheck: false

  volumes:

      - id: ldaptls

        mountPath: /etc/ssl/certs/java

    type: secret

    readOnly: true

env:

 JAVA_OPTS: "-Djdk.tls.client.protocols=TLSv1.2 -Djavax.net.debug=all -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStorePassword=changeit  -Djavax.net.ssl.keyStoreType=jks -Djavax.net.ssl.keyStore=/etc/ssl/certs/java/tls.jks -Djavax.net.ssl.keyStorePassword=changeit"

And similarly   ~/.hal/default/service-settings/fiat.yml

 kubernetes:

volumes:

     - id: ldaptls

  mountPath: /etc/ssl/certs/java

  type: secret

  readOnly: true

env:

 JAVA_OPTS: "-Djdk.tls.client.protocols=TLSv1.2  -Djavax.net.debug=all -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStorePassword=changeit  -Djavax.net.ssl.keyStoreType=jks -Djavax.net.ssl.keyStore=/etc/ssl/certs/java/tls.jks -Djavax.net.ssl.keyStorePassword=changeit"

Step 6.F: 

So if LDAP is connected with mtls or not , we have to do the following changes.

Go to: ~/.hal/config

  • Change the ldap url the following: 

ldaps://<ldap svc name>.<ldap namespace>.svc.cluster.local:636/dc=example,dc=org

 Go to:  ~/.hal/default/profiles/fiat-local.yml

auth:

     groupMembership:

service: ldap

ldap:

   url: ldaps://<ldap svc name>.<<LDAP-NAMESPACE>>.svc.cluster.local:636

   managerDn: cn=admin,dc=example,dc=org

   managerPassword: <<LDAP_ADMIN_PASSWORD>> (PRESENT IN VALUES.YAML)

   groupSearchBase: ou=groups,dc=example,dc=org

   groupSearchFilter: member={0}

   groupRoleAttributes: cn

   userDnPattern: cn={0},dc=example,dc=org

And apply your changes

 hal deploy apply

Now try to access the Spinnaker from UI

 Conclusion:

By following the above  mentioned steps, we can transfer data or information securely between LDAP and Spinnaker using mutual TLS  certificates,  thereby eliminating any security vulnerability risks.    

References

https://www.openldap.org/doc/admin22/tls.html

https://github.com/helm/charts/tree/master/stable/openldap

https://cert-manager.io/docs/installation/kubernetes/

https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.2/manage_applications/create_issuer.html


 

Gopal Jayanthi

Gopal Jayanthi has 15+ years of experience in the software field in development, configuration management, build/release, and DevOps areas. Worked at Cisco, AT&T (SBC), IBM in USA and Accenture, Bank of America, and Tech Mahindra in India. Expertise in Kubernetes, Docker, Jenkins, SDLC management, version control, change management, release management.

Link

0 Comments

Submit a Comment

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.