Fluent Bit is a super fast, lightweight, and highly scalable logging and metrics processor and forwarder that allows you to collect data and logs from different sources, unify them, and send them to multiple destinations. Tanzu Kubernetes Grid includes signed binaries for Fluent Bit, that you can deploy on management clusters and on Tanzu Kubernetes clusters to provide a log-forwarding service. This post explains how to install the fluent-bit package onto a Tanzu Kubernetes (workload) cluster using TMC catalog and forward the logs to Azure blob storage.
- Bootstrap machine with the following installed: Tanzu CLI, kubectl installed as mentioned here
- Tanzu Kubernetes Grid management cluster and workload cluster running on vSphere, Amazon EC2, or Azure, with the package repository installed. For this demo, I have deployed TKG on Azure.
- Storage Account and a blob container: You can use an existing one if already exists. if not, below are the steps to create a storage account:
Navigate to Azure portal > Storage Accounts > Create
- Resource group: Create new or use existing one. In this case, I have used an existing resource group (images-rg)
- Storage account name: provide a name, here I have given capvfluentd
- Review + Create > Create
- In Azure portal > Storage Accounts > click on newly created storage account (capvfluentd) > Create Container by clicking on + Container
- Name it and Create. In this case, I named it as capv-logs
- Once successfully created, you can find it under the containers list
Install the package
In TMC Console: Catalog > select the workload cluster (capv-workload in my case) from drop down and click on fluent-bit
- Click on Install Package which can be found on top right side of the page
- Name the Installed package as : capv-workload-fluentd and click NEXT
- Package install resources: Leave to Default and click NEXT
- Configure values: Provide below values
- Install Package
In TMC Console: Clusters > capv-workload > Add-ons > Installed > check if the package ( fluent-bit.tanzu.vmware.com ) is succeeded and healthy
Note: If you see below errors in fluent bit logs, then follow the below steps. If not, then Navigate to Azure portal > Storage Accounts > click on container: capv-logs > kubernetes
- Replace the image name with fluent/fluent-bit:latest
- Navigate to Azure portal > Storage Accounts > click on container: capv-logs > Kubernetes
This confirms that fluent bit is forwarding the logs to Azure blob storage successfully.