TAP Multi Cluster (1.3) : Part 1 – Prepare The Setup

Eknath - 22 December 202224 December 2022

Reading Time: 5 mins

Contents

Overview

VMware Tanzu Application Platform (TAP) is an application development platform with a rich set of developer tools. It offers developers a paved path to production to build and deploy software quickly and securely on any compliant public cloud or on-premises Kubernetes cluster. TAP can be installed in various topologies to reflect your existing landscape and it is highly recommended by VMware to use multicluster topology for production use.

In this series of post, I will be covering the steps to install TAP (multi cluster) on 4 different kubernetes clusters each adopting one of following multicluster-aligned profiles:

Iterate: Intended for inner-loop iterative application development.
Build: Transforms source revisions to workload revisions; specifically, hosting workloads and supply chains.
Run: Transforms workload revisions to running pods; specifically, hosting deliveries and deliverables.
View: For applications related to centralized developer experiences; specifically, Tanzu Application Platform GUI and metadata store.

Prerequisites

A Tanzu Network account to download Tanzu Application Platform packages.
A container image registry, such as Harbor or Docker Hub with at least 10 GB of available storage for application images, base images, and runtime dependencies. When available, VMware recommends using a paid registry account to avoid potential rate-limiting associated with some free registry offerings.
Registry credentials with push and write access made available to Tanzu Application Platform to store images.
Network access to https://registry.tanzu.vmware.com
Network access to your chosen container image registry.

There are some optional prerequisites as well, refer to official doc for more details.

Note: For this demo, I have used ubuntu machine as jumpbox to install TAP.

Create Kubernetes Cluster

Steps to create AKS cluster is documented here (includes kubectl, az cli), but to install TAP with multi cluster topology, you need each kubernetes cluster with min 8 CPUs, 8 GB RAM. So ensure to deploy AKS cluster accordingly.
For this post, I have created 4 AKS clusters and named it as
- - tap-build-cluster
  - tap-run-cluster
  - tap-view-cluster
  - tap-iterate-cluster

Once the cluster is created and status turns green, click on the cluster > connect > “Run the following commands”

# Set the account
az account set --subscription <Subscription id>

# Get the credentials of BUILD cluster, below is the example for ref where capv-tapmc-rg  is resource group name and tap-build-cluster is
 the name of AKS cluster.

az aks get-credentials --resource-group capv-tapmc-rg --name tap-build-cluster

# Get the credentials of RUN cluster, below is the example for ref where capv-tapmc-rg is resource group name and tap-run-cluster is
 the name of AKS cluster.

az aks get-credentials --resource-group capv-tapmc-rg --name tap-run-cluster

# Get the credentials of VIEW cluster, below is the example for ref where capv-tapmc-rg is resource group name and tap-view-cluster is
 the name of AKS cluster.

az aks get-credentials --resource-group capv-tapmc-rg --name tap-view-cluster

# Get the credentials of ITERATE cluster, below is the example for ref where capv-tapmc-rg is resource group name and tap-iterate-cluster is
 the name of AKS cluster.

az aks get-credentials --resource-group capv-tapmc-rg --name tap-iterate-cluster

List the contexts

$ kubectl config get-contexts
CURRENT   NAME                  CLUSTER               AUTHINFO                                        NAMESPACE
          tap-build-cluster     tap-build-cluster     clusterUser_capv-tapmc-rg_tap-build-cluster
          tap-iterate-cluster   tap-iterate-cluster   clusterUser_capv-tapmc-rg_tap-iterate-cluster
          tap-run-cluster       tap-run-cluster       clusterUser_capv-tapmc-rg_tap-run-cluster
*         tap-view-cluster      tap-view-cluster      clusterUser_capv-tapmc-rg_tap-view-cluster

Install Tanzu CLI, plugins

Install Docker

Refer to Docker page to get the detailed steps of Installing Docker.

Install Tanzu CLI

Sign to Tanzu Network and accept below EULAs:

Cluster Essentials for VMware Tanzu
Tanzu Application Platform
Tanzu Build Service and its associated components:

Sign in to Tanzu Network and download tanzu-framework-bundle-linux (for Linux) into local machine and copy to destination jumpbox using scp or download directly using pivnet cli.

## Download the TAP plugin bundle using below PIVNET CLI: 

pivnet download-product-files --product-slug='tanzu-application-platform' --release-version='1.3.4' --product-file-id=1352407

# Create a directory named tanzu:

mkdir $HOME/tanzu

# unpack the TAR file into the tanzu directory:

tar -xvf tanzu-framework-linux-amd64-v0.25.0.4.tar -C $HOME/tanzu

# Set env var TANZU_CLI_NO_INIT to true to assure the local downloaded versions of the CLI core and plug-ins are installed:

export TANZU_CLI_NO_INIT=true

# Install the CLI core by running:

cd $HOME/tanzu

export VERSION=v0.25.0

sudo install cli/core/$VERSION/tanzu-core-linux_amd64 /usr/local/bin/tanzu

# Confirm the installation by running:

$ tanzu version
version: v0.25.0


## Install plug-ins from the $HOME/tanzu directory 

$ tanzu plugin install --local cli all
Installing plugin 'accelerator:v1.3.1'
Installing plugin 'apps:v0.9.0'
Installing plugin 'insight:v1.3.4'
Installing plugin 'package:v0.25.0'
Installing plugin 'secret:v0.25.0'
Installing plugin 'services:v0.4.0'
✔  successfully installed 'all' plugin

## Verify that you installed the plug-ins by running:

$ tanzu plugin list
  NAME                DESCRIPTION                                                                       SCOPE       DISCOVERY  VERSION  STATUS
  login               Login to the platform                                                             Standalone  default    v0.25.0  not installed
  management-cluster  Kubernetes management-cluster operations                                          Standalone  default    v0.25.0  not installed
  package             Tanzu package management                                                          Standalone  default    v0.25.0  installed
  pinniped-auth       Pinniped authentication operations (usually not directly invoked)                 Standalone  default    v0.25.0  not installed
  secret              Tanzu secret management                                                           Standalone  default    v0.25.0  installed
  telemetry           Configure cluster-wide telemetry settings                                         Standalone  default    v0.25.0  not installed
  accelerator         Manage accelerators in a Kubernetes cluster                                       Standalone             v1.3.1   installed
  apps                Applications on Kubernetes                                                        Standalone             v0.9.0   installed
  insight             post & query image, package, source, and vulnerability data                       Standalone             v1.3.4   installed
  services            Explore Service Instance Classes, discover claimable Service Instances and        Standalone             v0.4.0   installed
                      manage Resource Claims

Install cluster essentials:

Sign in to Tanzu Network , Download tanzu-cluster-essentials-darwin-amd64-1.3.0.tgz (for OS X) or tanzu-cluster-essentials-linux-amd64-1.3.0.tgz (for Linux) into local machine and copy to destination jumpbox using scp or download directly using pivnet cli and install tanzu cluster essentials in all 4 kubernetes clusters as mentioned in below steps:

## OS X
pivnet download-product-files --product-slug='tanzu-cluster-essentials' --release-version='1.3.0' --product-file-id=1330472

## LINUX: 
pivnet download-product-files --product-slug='tanzu-cluster-essentials' --release-version='1.3.0' --product-file-id=1330470

## Create a directory named tanzu-cluster-essentials

mkdir $HOME/tanzu-cluster-essentials

## Unpack the TAR file into the tanzu-cluster-essentials directory:

tar -xvf tanzu-cluster-essentials-linux-amd64-1.3.0.tgz -C $HOME/tanzu-cluster-essentials

BUILD CLUSTER

## Check the contexts and ensure the current context is set to Build Cluster: 

kubectl config get-contexts
CURRENT   NAME                  CLUSTER               AUTHINFO                                        NAMESPACE
*         tap-build-cluster     tap-build-cluster     clusterUser_capv-tapmc-rg_tap-build-cluster
          tap-iterate-cluster   tap-iterate-cluster   clusterUser_capv-tapmc-rg_tap-iterate-cluster
          tap-run-cluster       tap-run-cluster       clusterUser_capv-tapmc-rg_tap-run-cluster
          tap-view-cluster      tap-view-cluster      clusterUser_capv-tapmc-rg_tap-view-cluster

## Configure and run install.sh, which will install kapp-controller and secretgen-controller on your cluster:
export INSTALL_BUNDLE=registry.tanzu.vmware.com/tanzu-cluster-essentials/cluster-essentials-bundle@sha256:54bf611711923dccd7c7f10603c846782b90644d48f1cb570b43a082d18e23b9
export INSTALL_REGISTRY_HOSTNAME=registry.tanzu.vmware.com

## TANZU-NET-USER and TANZU-NET-PASSWORD are the Tanzu network credentials. 

export INSTALL_REGISTRY_USERNAME=TANZU-NET-USER
export INSTALL_REGISTRY_PASSWORD=TANZU-NET-PASSWORD
cd $HOME/tanzu-cluster-essentials
./install.sh --yes

RUN CLUSTER

## Change the context to RUN cluster

$ kubectl config use-context tap-run-cluster
Switched to context "tap-run-cluster".

## Check the contexts and ensure the current context is set to RUN Cluster: 

$ kubectl config get-contexts
CURRENT   NAME                  CLUSTER               AUTHINFO                                        NAMESPACE
          tap-build-cluster     tap-build-cluster     clusterUser_capv-tapmc-rg_tap-build-cluster
          tap-iterate-cluster   tap-iterate-cluster   clusterUser_capv-tapmc-rg_tap-iterate-cluster
*         tap-run-cluster       tap-run-cluster       clusterUser_capv-tapmc-rg_tap-run-cluster
          tap-view-cluster      tap-view-cluster      clusterUser_capv-tapmc-rg_tap-view-cluster

## Configure and run install.sh, which will install kapp-controller and secretgen-controller on your cluster:

./install.sh --yes

VIEW CLUSTER

## Change the context to VIEW cluster

$ kubectl config use-context tap-view-cluster
Switched to context "tap-view-cluster".

## Check the contexts and ensure the current context is set to VIEW Cluster: 

$ kubectl config get-contexts
CURRENT   NAME                  CLUSTER               AUTHINFO                                        NAMESPACE
          tap-build-cluster     tap-build-cluster     clusterUser_capv-tapmc-rg_tap-build-cluster
          tap-iterate-cluster   tap-iterate-cluster   clusterUser_capv-tapmc-rg_tap-iterate-cluster
          tap-run-cluster       tap-run-cluster       clusterUser_capv-tapmc-rg_tap-run-cluster
*         tap-view-cluster      tap-view-cluster      clusterUser_capv-tapmc-rg_tap-view-cluster

## Configure and run install.sh, which will install kapp-controller and secretgen-controller on your cluster:

./install.sh --yes

Iterate Cluster

## Change the context to Iterate cluster

$ kubectl config use-context tap-iterate-cluster
Switched to context "tap-iterate-cluster".

## Check the contexts and ensure the current context is set to Iterate Cluster: 

$ kubectl config get-contexts
CURRENT   NAME                  CLUSTER               AUTHINFO                                        NAMESPACE
          tap-build-cluster     tap-build-cluster     clusterUser_capv-tapmc-rg_tap-build-cluster
*         tap-iterate-cluster   tap-iterate-cluster   clusterUser_capv-tapmc-rg_tap-iterate-cluster
          tap-run-cluster       tap-run-cluster       clusterUser_capv-tapmc-rg_tap-run-cluster
          tap-view-cluster      tap-view-cluster      clusterUser_capv-tapmc-rg_tap-view-cluster

## Configure and run install.sh, which will install kapp-controller and secretgen-controller on your cluster:

./install.sh --yes

Install imgpkg, kapp CLIs

## Install kapp CLI onto your $PATH:
sudo cp $HOME/tanzu-cluster-essentials/kapp /usr/local/bin/kapp

## Install imgpkg CLI onto your $PATH:

sudo cp $HOME/tanzu-cluster-essentials/imgpkg /usr/local/bin/imgpkg

## Install JQ

sudo apt  install jq

Create ACR (Optional)

In this demo, I will be using ACR (Azure Container Registries) to store the images. You can deploy the application with images located in any registry of your choice.

Login to Azure portal > Container registries > Create container registry
In Resource Group, create new as shown below and give it a name

Give a registry name and select the Location from drop down.

Review + Create
Create

Once registry is successfully created, navigate to container registries > click on created registry > Settings > Access keys > Enable Admin user to get the password and collect below to authenticate using docker login, save it for future ref which will be used in TAP values file in later sections:

- - Login server
  - Username
  - password

Docker login

## Docker login to repo, In this case the registry name I have used is captainrepo. you should replace it with the name that you created. 

$ docker login captainrepo.azurecr.io
Username: captainrepo
Password:
WARNING! Your password will be stored unencrypted in /home/azureuser/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

Relocate images to a registry

## Log in to the VMware Tanzu Network registry with your VMware Tanzu Network credentials by running:

docker login registry.tanzu.vmware.com

## Set up environment variables for installation use by running:

export INSTALL_REGISTRY_USERNAME=captainrepo
export INSTALL_REGISTRY_PASSWORD=<Registry password>
export INSTALL_REGISTRY_HOSTNAME=captainrepo.azurecr.io
export TAP_VERSION=1.3.4
export INSTALL_REPO=tap1.3

## Image copy takes about 15-20 mins to complete, proceed further once this process is completed successfully. 

imgpkg copy -b registry.tanzu.vmware.com/tanzu-application-platform/tap-packages:${TAP_VERSION} --to-repo ${INSTALL_REGISTRY_HOSTNAME}/${INSTALL_REPO}/tap-packages

Add the Tanzu Application Platform package repository:

Execute the below commands on all the 4 kubernetes clusters by changing the contexts i.e., Build, Run, View and Iterate clusters.

## Create a namespace called tap-install for deploying any component packages

kubectl create ns tap-install

## Create a registry secret by running:

tanzu secret registry add tap-registry \
  --username ${INSTALL_REGISTRY_USERNAME} --password ${INSTALL_REGISTRY_PASSWORD} \
  --server ${INSTALL_REGISTRY_HOSTNAME} \
  --export-to-all-namespaces --yes --namespace tap-install

## Create secret registry-credentials

kubectl create secret docker-registry registry-credentials --docker-server=captainrepo.azurecr.io --docker-username=captainrepo --docker-password=<registry password> -n tap-install

## Add the Tanzu Application Platform package repository to the cluster by running:

tanzu package repository add tanzu-tap-repository \
  --url ${INSTALL_REGISTRY_HOSTNAME}/${INSTALL_REPO}/tap-packages:$TAP_VERSION \
  --namespace tap-install

## Get the status of the Tanzu Application Platform package repository, and ensure the status updates to Reconcile succeeded by running:

tanzu package repository get tanzu-tap-repository --namespace tap-install

NAME:          tanzu-tap-repository
VERSION:       20254
REPOSITORY:    captainrepo.azurecr.io/tap1.3/tap-packages
TAG:           1.3.4
STATUS:        Reconcile succeeded
REASON:

## List the available packages by running:

$ tanzu package available list --namespace tap-install

## List version information for the TAP package by running:

$ tanzu package available list tap.tanzu.vmware.com --namespace tap-install

  NAME                  VERSION  RELEASED-AT
  tap.tanzu.vmware.com  1.3.4    2022-12-20 00:00:00 +0000 UTC