TAP Multi Cluster (1.3) : Part 3 – Deploy Application

Eknath - 23 December 202224 December 2022

Reading Time: 3 mins

Contents

Scan Policy

The ScanPolicy defines a set of rules to evaluate for a particular scan to consider the artifacts (image or source code) either compliant or not. When a ImageScan or SourceScan is created to run a scan, those reference a policy whose name must match the following sample scan-policy:

Note: In this policy, I have added few CVE’s to ignore list for smoother application deployment. If you are running a different application other than the one given in this post, then please check and change the CVE’s accordingly.

Scan policy

---
apiVersion: scanning.apps.tanzu.vmware.com/v1beta1
kind: ScanPolicy
metadata:
  name: scan-policy
  labels:
    'app.kubernetes.io/part-of': 'enable-in-gui'
#    'app.kubernetes.io/part-of': 'component-a'
#   'app.kubernetes.io/part-of': 'hello-world'
#    'app.kubernetes.io/part-of': 'tanzu-java-web-app'
#    'app.kubernetes.io/part-of': 'partnertapdemo'
spec:
  regoFile: |
    package main

    # Accepted Values: "Critical", "High", "Medium", "Low", "Negligible", "UnknownSeverity"
    notAllowedSeverities := ["UnknownSeverity"]
    ignoreCves := ["CVE-2016-1000027","CVE-2021-26291","GHSA-3mc7-4q67-w48m"]

    contains(array, elem) = true {
      array[_] = elem
    } else = false { true }

    isSafe(match) {
      severities := { e | e := match.ratings.rating.severity } | { e | e := match.ratings.rating[_].severity }
      some i
      fails := contains(notAllowedSeverities, severities[i])
      not fails
    }

    isSafe(match) {
      ignore := contains(ignoreCves, match.id)
      ignore
    }

    deny[msg] {
      comps := { e | e := input.bom.components.component } | { e | e := input.bom.components.component[_] }
      some i
      comp := comps[i]
      vulns := { e | e := comp.vulnerabilities.vulnerability } | { e | e := comp.vulnerabilities.vulnerability[_] }
      some j
      vuln := vulns[j]
      ratings := { e | e := vuln.ratings.rating.severity } | { e | e := vuln.ratings.rating[_].severity }
      not isSafe(vuln)
      msg = sprintf("CVE %s %s %s", [comp.name, vuln.id, ratings])
    }

Tekton/Pipeline

A pipeline is a series of tasks to run against the source code that has been found by earlier resources in the Supply Chain.

Tekton Pipeline

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: developer-defined-tekton-pipeline
  labels:
    apps.tanzu.vmware.com/pipeline: test      # (!) required
spec:
  params:
    - name: source-url                        # (!) required
    - name: source-revision                   # (!) required
  tasks:
    - name: test
      params:
        - name: source-url
          value: $(params.source-url)
        - name: source-revision
          value: $(params.source-revision)
      taskSpec:
        params:
          - name: source-url
          - name: source-revision
        steps:
          - name: test
            image: gradle
            script: |-
              cd `mktemp -d`
              wget -qO- $(params.source-url) | tar xvz -m
              ./mvnw test

## Change the context to Build cluster to apply the scan policy and Tekton Pipeline

$ kubectl config use-context tap-build-cluster
Switched to context "tap-build-cluster".

## Apply scan policy

kubectl apply -f scanpolicy.yaml -n tap-install
scanpolicy.scanning.apps.tanzu.vmware.com/scan-policy created

## Apply Tekton Pipeline in build cluster: 

kubectl apply -f tekton-pipeline.yaml -n tap-install
pipeline.tekton.dev/developer-defined-tekton-pipeline created

Set up developer namespaces

Note: Ensure to setup developer namespaces to use installed packages – i.e., by running below kubectl apply command in 3 clusters: build, run and iterate

cat <<EOF | kubectl -n tap-install apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: tap-registry
  annotations:
    secretgen.carvel.dev/image-pull-secret: ""
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: e30K
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
secrets:
  - name: registry-credentials
imagePullSecrets:
  - name: registry-credentials
  - name: tap-registry
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: default-permit-deliverable
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deliverable
subjects:
  - kind: ServiceAccount
    name: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: default-permit-workload
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: workload
subjects:
  - kind: ServiceAccount
    name: default
EOF

Setup Metadata Store

Supply Chain Security Tools – Store saves software bills of materials (SBoMs) to a database and allows you to query for image, source code, package, and vulnerability relationships. It integrates with Supply Chain Security Tools – Scan to automatically store the resulting source code and image vulnerability reports. It accepts CycloneDX input and outputs in both human-readable and machine-readable formats, including JSON, text, and CycloneDX.

Metadata Store

## Change the context to View Cluster

$ kubectl config use-context tap-view-cluster
Switched to context "tap-view-cluster".

## The following command retrieves the access token from the default metadata-store-read-write-client service account and stores it in METADATA_STORE_ACCESS_TOKEN:

export METADATA_STORE_ACCESS_TOKEN=$(kubectl get secrets metadata-store-read-write-client -n metadata-store -o jsonpath="{.data.token}" | base64 -d)

## Collect the ENVOY_IP: 

kubectl get svc -n tanzu-system-ingress , point the external ip with hostname metadata-store.captainvirtualization.co.in

## To get the certificate, run:

kubectl get secret ingress-cert -n metadata-store -o json | jq -r '.data."ca.crt"' | base64 -d > insight-ca.crt

create a record set in DNS pointing to ip ( ENVOY IP collected in previous step ) to host name metadata-store.captainvirtualization.co.in, as shown in below example:

##  Set the target by running:

$ tanzu insight config set-target https://metadata-store.captainvirtualization.co.in --ca-cert insight-ca.crt
ℹ  Config File "/home/azureuser/.config/tanzu/insight/config.yaml" Already Exists
ℹ  Using config file: /home/azureuser/.config/tanzu/insight/config.yaml
ℹ  Setting trustedcacert in config
ℹ  Setting endpoint in config to: https://metadata-store.captainvirtualization.co.in
✔  Success: Set Metadata Store endpoint

Start the workload

Build Profile Cluster

## Switch the context to build cluster

$ kubectl config use-context tap-build-cluster
Switched to context "tap-build-cluster".

## Use the Tanzu CLI to start the workload:

$ tanzu apps workload create tanzu-java-web-app  --git-repo https://github.com/Eknathreddy09/tanzu-java-web-app --git-branch main --type web --label apps.tanzu.vmware.com/has-tests=true --label app.kubernetes.io/part-of=tanzu-java-web-app  --type web -n tap-install --yes
Create workload:
      1 + |---
      2 + |apiVersion: carto.run/v1alpha1
      3 + |kind: Workload
      4 + |metadata:
      5 + |  labels:
      6 + |    app.kubernetes.io/part-of: tanzu-java-web-app
      7 + |    apps.tanzu.vmware.com/has-tests: "true"
      8 + |    apps.tanzu.vmware.com/workload-type: web
      9 + |  name: tanzu-java-web-app
     10 + |  namespace: tap-install
     11 + |spec:
     12 + |  source:
     13 + |    git:
     14 + |      ref:
     15 + |        branch: main
     16 + |      url: https://github.com/Eknathreddy09/tanzu-java-web-app

Created workload "tanzu-java-web-app"

To see logs:   "tanzu apps workload tail tanzu-java-web-app --namespace tap-install"
To get status: "tanzu apps workload get tanzu-java-web-app --namespace tap-install"

## To monitor the progress of this process, run:

$ tanzu apps workload tail tanzu-java-web-app --namespace tap-install

## To get the workload info: 

$ tanzu apps workload get tanzu-java-web-app --namespace tap-install
📡 Overview
   name:   tanzu-java-web-app
   type:   web

💾 Source
   type:     git
   url:      https://github.com/Eknathreddy09/tanzu-java-web-app
   branch:   main

📦 Supply Chain
   name:   source-test-scan-to-url

   RESOURCE           READY   HEALTHY   TIME    OUTPUT
   source-provider    True    True      6m4s    GitRepository/tanzu-java-web-app
   source-tester      True    True      5m38s   Runnable/tanzu-java-web-app
   source-scanner     True    True      5m11s   SourceScan/tanzu-java-web-app
   image-provider     True    True      3m31s   Image/tanzu-java-web-app
   image-scanner      True    True      2m25s   ImageScan/tanzu-java-web-app
   config-provider    True    True      2m17s   PodIntent/tanzu-java-web-app
   app-config         True    True      2m17s   ConfigMap/tanzu-java-web-app
   service-bindings   True    True      2m17s   ConfigMap/tanzu-java-web-app-with-claims
   api-descriptors    True    True      2m17s   ConfigMap/tanzu-java-web-app-with-api-descriptors
   config-writer      True    True      2m5s    Runnable/tanzu-java-web-app-config-writer
   deliverable        True    True      6m7s    ConfigMap/tanzu-java-web-app

🚚 Delivery

   Delivery resources not found.

💬 Messages
   No messages found.

🛶 Pods
   NAME                                         READY   STATUS      RESTARTS   AGE
   scan-tanzu-java-web-app-pzwr6-24blz          0/1     Completed   0          3m31s
   scan-tanzu-java-web-app-svfgv-s45wd          0/1     Completed   0          5m37s
   tanzu-java-web-app-build-1-build-pod         0/1     Completed   0          5m8s
   tanzu-java-web-app-config-writer-gbqzg-pod   0/1     Completed   0          2m16s
   tanzu-java-web-app-sfrdj-test-pod            0/1     Completed   0          6m2s

To see logs: "tanzu apps workload tail tanzu-java-web-app --namespace tap-install"

## Apply Annotation

$ tanzu apps workload apply tanzu-java-web-app --annotation autoscaling.knative.dev/minScale=1 -n tap-install -y

## Generate the deliverable.yaml file once the workload status turns READY state after waiting for few mins once the workload create command is executed successfully. 

## Verify that your supply chain has produced the necessary ConfigMap containing Deliverable content produced by the Workload:

$ kubectl get configmap tanzu-java-web-app --namespace tap-install -o go-template='{{.data.deliverable}}'
apiVersion: carto.run/v1alpha1
kind: Deliverable
metadata:
  name: tanzu-java-web-app
  labels:
    app.kubernetes.io/part-of: tanzu-java-web-app
    apps.tanzu.vmware.com/has-tests: "true"
    apps.tanzu.vmware.com/workload-type: web
    app.kubernetes.io/component: deliverable
    app.tanzu.vmware.com/deliverable-type: web
spec:
  source:
    image: captainrepo.azurecr.io/supply-chain/tanzu-java-web-app-tap-install-bundle:cd1e7143-a878-40b0-a6a7-db7107d35a4d

## Store the Deliverable content, which you can take to the Run profile clusters from the ConfigMap by running:

$ kubectl get configmap tanzu-java-web-app -n tap-install -o go-template='{{.data.deliverable}}' > deliverable.yaml

RUN Profile Cluster

## Switch the context to RUN cluster

$ kubectl config use-context tap-run-cluster
Switched to context "tap-run-cluster".

## Take this Deliverable file to the Run profile clusters by running:

$ kubectl apply -f deliverable.yaml --namespace tap-install


## Verify that this Deliverable is started and Ready by running:

$ kubectl get deliverables --namespace tap-install
NAME                 SOURCE                                                                                                           DELIVERY         READY   REASON   AGE
tanzu-java-web-app   captainrepo.azurecr.io/supply-chain/tanzu-java-web-app-tap-install-bundle:cd1e7143-a878-40b0-a6a7-db7107d35a4d   delivery-basic   True    Ready    24s

## To test the application, query the URL for the application. Look for the httpProxy by running:

$ kubectl get httpproxy --namespace tap-install
NAME                                                              FQDN                                                         TLS SECRET   STATUS   STATUS DESCRIPTION
tanzu-java-web-app-contour-25f5ce7861bbc5d885a1a4865b2ac9e3tanz   tanzu-java-web-app.tap-install.captainvirtualization.co.in                valid    Valid HTTPProxy
tanzu-java-web-app-contour-ac7a26d7751472c885a6709b26f92948tanz   tanzu-java-web-app.tap-install.svc.cluster.local                          valid    Valid HTTPProxy
tanzu-java-web-app-contour-tanzu-java-web-app.tap-install         tanzu-java-web-app.tap-install                                            valid    Valid HTTPProxy
tanzu-java-web-app-contour-tanzu-java-web-app.tap-install.svc     tanzu-java-web-app.tap-install.svc                                        valid    Valid HTTPProxy

## Collect the Envoy External IP 
$ kubectl get svc -n tanzu-system-ingress

create a record set in DNS pointing to ip ( ENVOY IP collected in previous step ) to host name from httpproxy output (ref below screenshot).

Access the url in browser to check the application (http)