Tanzu Kubernetes Grid on VMware Cloud on AWS

A few days ago I had the opportunity to prepare a Tanzu Kubernetes Grid environment on VMware Cloud on AWS for a customer demo, and I thought it would have made sense to make a blog post on this.

TKG deployment is pretty straightforward, but to give deployment time a further boost I decided to leverage the Demo Appliance for Tanzu Kubernetes Grid. The Demo Appliance for TKG has been released a few days ago as a VMware Fling.

Kudos to William Lam for the release of this Fling, a really powerful way to speed up the installation of TKG and go straight to showcase its power and flexibility.

For the scope of the demo I had to deliver, I decided to follow William’s demo script, provided with the Fling’s instructions, as I believe it really fits for my demo purpose.

Tanzu Kubernetes Grid Overview

VMware Tanzu Kubernetes Grid is an enterprise-ready Kubernetes runtime that packages open source technologies and automation tooling to help you get up and running quickly with a scalable, multi-cluster Kubernetes environment.

In VMware Cloud on AWS, the choice has been made to support the “Tanzu Kubernetes Grid Plus” offering. The “Plus” is an addon to a traditional TGK installation which means the customer will get support for several additional open-source products, together with support from a specialized team of Customer Reliability Engineers (CRE) dedicated to support TKG+ customers. In this knowledge base article you can find a detailed comparison between TKG and TKG+.

For further details on the solution, you can browse to the official Tanzu Kubernetes Grid page, and read the VMware Tanzu Kubernetes Grid Plus on VMware Cloud on AWS solution brief.

To setup TKG, we are leveraging a bootstrap environment that can live on your laptop, in a VM, in a Server etc. and it’s based on kind (Kubernetes in Docker). Basically, with kind we are leveraging Kubernetes to deploy our TGK Management Cluster. Using the Demo Appliance for Tanzu Kubernetes Grid, we have all the components required to setup the bootstrap environment ready to be used, and we can simply focus on the TGK deployment itself. I really hope to see the Demo Appliance being productized and as such becoming supported to deploy production environments.

Once the TKG Management Cluster is in place, we will leverage Cluster API through the TKG Command Line Interface to manage the creation and lifecycle of all our TKG Workload Cluster where our applications will live. I believe the following picture could be well descriptive of the concept: we leverage Kubernetes to create a Management Cluster, and we’ll manage all our Workload Clusters from the Management Cluster. (Thanks again to William for the nice diagram).

TKG on VMC conceptual architecture
TKG on VMC conceptual architecture

For the scope of this article, I’ve already satisfied the little amount of networking and security prerequisites you need to implement in your VMC SDDC before deploying TGK.

Now, let’s go straight to the steps needed to implement TKG on VMC and see it in action: from 0 to Kubernetes on VMC in less than 30 minutes!

Setup Content Library to sync all TKG Virtual Appliance templates

Open the SDDC’s vCenter and from the “Home” screen, select “Content Libraries”.

vSphere Client - Content Library
vSphere Client – Content Library

Click on the “+” button to start the “New Content Library” wizard. In the “Name and location” window, we can enter the name of the new Content Library and click “NEXT”.

New Content Library
New Content Library

In the “Configure content library” window, check the “Subscribed content library” radio button, input the following subscription URL: “https://download3.vmware.com/software/vmw-tools/tkg-demo-appliance/cl/lib.json“, check the option to download content immediately, then click “NEXT”.

Subscribe to Content URL
Subscribe to Content URL

Accept the authenticity of the subscription Host clicking “YES”.

Trust the subscription Host
Trust the subscription Host

In the “Add storage” window, select the “WorkloadDatastore”, then click “NEXT”.

Content Library - Storage
Content Library – Storage

In the “Ready to complete” window, review the settings then click “FINISH”.

New Content Library - review settings
New Content Library – review settings

The new Content Library is now created, select it and move to the “OVF & OVA Templates” tab, where we can wait for our templates to be downloaded.

TGK Demo Content Library
TGK Demo Content Library

Once the download is completed, right-click on the “TKG-Demo-Appliance_1.0.0.ova” and select “New VM from This Template”. This will start the new VM creation wizard.

Deploy the Demo Appliance for Tanzu Kubernetes Grid

New VM from template
New VM from template

Select a name for the new Virtual Machine, select a location (VM Folder) then click “NEXT”.

New VM - Name and Folder
New VM – Name and Folder

Select the destination Resource Pool (by default, Compute-Resource-Pool in VMC) and click “NEXT”.

Select Compute Resource
Select Compute Resource

Review the details and, if everything is fine, click on “NEXT”.

New VM - Review Details
New VM – Review Details

Select “WorkloadDatastore” as the target storage, optionally choose a specific Storage Policy to apply to the new VM, then click “NEXT”.

New VM - Select Storage
New VM – Select Storage

Select the destination network for the new VM, then click “NEXT”.

New VM - Select Network
New VM – Select Network

In the “Customize template” window, configure the new VM IP address, Gateway, DNS, NTP, password, optional proxy Server, then click “NEXT”.

New VM - Customize template
New VM – Customize template

Review all the settings, then click “FINISH” to start the TKG Demo Appliance VM creation.

New VM - Ready to complete
New VM – Ready to complete

Before moving to the Management Cluster setup, we need to have in place two Virtual Machine Templates that will be used to deploy the environment. Back to the same Content Library we created before, where we have our TKG Demo Appliance template, we must deploy two new VMs by using the templates “photon-3-capv-haproxy-v0.6.3_vmware.1” and “photon-3-v1.17.3_vmware.2” and, once the two VMs are created, convert them to vSphere Templates, as we’ll need these in the next steps. Once done, we are ready to proceed.

TKG Management Cluster Setup

We can locate our newly created Demo Appliance for TKG in the vCenter Inventory, together with all the others VMs and Templates. We need to take note of the IP address or DNS name assigned to the TKG Demo Appliance as we’ll access it with SSH and we’ll configure everything we need by using both the tkg and kubectl CLI.

Demo Appliance for TKG in vCenter
Demo Appliance for TKG in vCenter

Now that we have the IP address or the DNS name of the Appliance, let’s connect to it via SSH. Once connected, we can start the setup of our TKG Management Cluster typing the command “tkg init –ui” followed by Enter. This will start the TKG Management Cluster setup. As the Demo Appliance doesn’t have a UI, we’ll need to open a second session to it, setting up SSH port redirection to being able to use our local browser to access the TKG Installer Web UI.

SSH connection to Demo Appliance for TKG
SSH connection to Demo Appliance for TKG

Once port redirection is in place, we can access the TKG Installer wizard by navigating to “http://localhost:8080” on our local machine. Here, we can select the “DEPLOY ON VSPHERE” option.

TKG Installer – Deploy on vSphere

The first step is to input the vCenter Server IP or DNS name, and credentials, then click “CONNECT”.

TKG Installer - Connect to vCenter
TKG Installer – Connect to vCenter

We can safely ignore the notification about vSphere 7.0.0 and click “PROCEED”. The reason we are getting this information message is that VMware Cloud on AWS, following a different release cycle than the commercial edition of vSphere, already have vSphere 7 deployed in its SDDCs.

TKG Installer - vSphere 7 notification
TKG Installer – vSphere 7 notification

The second step is to select the vSphere Datacenter inventory item where we want to deploy our TKG Management Cluster. I’m not providing a real SSH Public Key in my example, but we should provide a “real” key if we want to easily connect to any Kubernetes node once deployed. Click “NEXT”.

TKG Installer - Iaas Provider
TKG Installer – Iaas Provider

In the “Control Plane Settings”, we can choose between several instance types and sizes based on our requirements. For the goal of this demo, it’s safe to pick the “Development” (single control plane node) flavour and the “medium” instance type. Input a name for the Management Cluster and select the API Server Load Balancer template. This will be the template we’ve created before starting from the “photon-3-capv-haproxy-v0.6.3_vmware.1” image. Click “NEXT” when done.

TKG Installer - Control Plane Settings
TKG Installer – Control Plane Settings

In the next step we must select the Resource Pool, VM Folder and Datastore where the Management Cluster will be created. Then click “NEXT”.

TKG Installer - Resources
TKG Installer – Resources

Select the SDDC network that will host our TKG VMs, leave the default CIDR selected for the Cluster Service CIDR and the Cluster Pod CIDR if you don’t have any specific requirement to change the default values. Click “NEXT” when done.

TKG Installer - Kubernetes Network
TKG Installer – Kubernetes Network

Select the vSphere Template configured with the required Kubernetes version. This is the template we’ve created before starting from the “photon-3-v1.17.3_vmware.2” image we’ve imported in our Content Library. Click “NEXT” when done.

TKG Installer - OS Image
TKG Installer – OS Image

With all the required fields compiled (green check mark), we can click on “REVIEW CONFIGURATION”.

TKG Installer - Review Configuration
TKG Installer – Review Configuration

We can then click on “DEPLOY MANAGEMENT CLUSTER” to start the deployment of our TKG Management Cluster. This will approximately take 6 to 8 minutes.

TKG Installer - Start Deployment
TKG Installer – Start Deployment

After all the configuration steps are completed, we will get an “Installation complete” confirmation message. Our TKG Management Cluster is now ready to be accessed. It’s safe to close the browser window and move back to the Demo Appliance for TGK SSH session we have previously opened.

TKG - Installation Complete
TKG – Installation Complete

Looking at the vCenter Inventory, we can easily see the Management Cluster VMs we’ve just deployed.

vCenter Inventory – TKG Management Cluster

Back in the SSH session, we automatically found ourselves in the context of the TKG Management Cluster. Here, leveraging the TKG Command Line Interface, we can create and manage the lifecycle of our Workload Clusters. Let’s create our first Workload Cluster using the command “tgk create cluster –plan=<dev_or_prod> <cluster-name>“. In my example I’m using the “dev” template and a cluster name of “it-tkg-wlc-01“. You can see in the low right end of the screenshot the VMs being deployed with the chosen name.

TKG - Create Workload Cluster
TKG – Create Workload Cluster

Once our Kubernetes Workload Cluster is created, we automatically find ourselves in the context of the new Cluster and we can immediately start deploying our applications.

TKG - Workload Cluster Created
TKG – Workload Cluster Created

I would like to leverage the YELB application for this Kubernetes demo, to deploy the application I’m starting by cloning the YELB git repository to my local machine.
Then, from the directory where we cloned the git repository (in my case “~/demo/yelb“), I create a new namespace with the command “kubectl create namespace yelb” followed by the resource creation with the command “kubectl apply -f yelb.yaml“.

YELB application resources deployed
YELB application resources deployed

As we have chosen to deploy a Kubernetes Cluster with only a single Control Plane and a single Node, it’s easy to get the IP address needed to access our application as the Control Plane doesn’t host running pods. With the command “kubectl get nodes -o wide” we can obtain the external IP address of our single node, which for sure is hosting the running yelb pods.

Get node external IP address
Get node external IP address

Once we have the external IP address of the node hosting the application, we can point our browser to that IP on port 30001, and here we can see our application is working and ready for the demo.

The YELB application
The YELB application

This concludes this post, we’ve seen how we can quickly deploy Tanzu Kubernetes Grid on top of our VMware Cloud on AWS Infrastructure.
This enables us to have a very powerful solution where Virtual Machines, Containers (orchestrated by Kubernetes) and native AWS services can coexist and be very well integrated to design your application modernization strategy.

In one of the next blog posts, I’ll show you how we can leverage such an integrated solution to quickly lift and shift an application, and subsequently modernize it.

Stay tuned! #ESVR