AKS Pricing: Unlock Visibility With Kubecost

Introduction

The Microsoft Azure Kubernetes Service (AKS) is a popular fully managed Kubernetes service that offers embedded continuous integration and delivery, as well as enterprise-grade security and governance. When running Azure Kubernetes Service (AKS), it can be hard to gain visibility for cost management in environments with multiple teams, projects, or even departments. As with any complex infrastructure, AKS requires proper governance and financial transparency for successful organizational adoption.

Kubecost provides teams with visibility into Kubernetes spend, and is a widely recommended solution for engineers and finance teams facing this problem. Kubecost supports a wide range of self-managed and hosted Kubernetes environments, including AKS, which we’ll cover in this article. With Kubecost, you gain full transparency into your Kubernetes usage and cost within minutes of installation. Officially launched in 2019 and built on open source, Kubecost monitors over one billion dollars in Kubernetes spend, and enables startups and global enterprises alike to understand their spend and identify cost savings ranging from 30% to over 50%.

Note: This documentation page for AKS provides helpful context for using Kubecost to implement a cost governance strategy.

Challenges in Cost Allocation

Kubernetes clusters are often shared across teams, microservices, projects, applications, departments, and cost centers to save on infrastructure and overhead. The most popular methods for organizing these shared resources is by simply using labels or creating namespaces. A Kubernetes namespace is a logical separation inside a Kubernetes cluster which could be assigned to a particular team, application, or even a business unit.

Most organizations map a namespace to a specific workload type or purpose. For example, a cluster might have one namespace for monitoring and another namespace for logging used by the DevOps teams who maintain the cluster. Whereas, a customer-facing application hosted in the same cluster might have a namespace for its front-end components and a separate namespace for its back-end services.

Creating these logical divisions inside a cluster is convenient, but adds complexity when trying to accurately measure resource cost and usage based on billing data.

Sharing a Cluster

There are three approaches for sharing Azure Kubernetes Service (AKS) clusters across cost centers:

1. Dedicated AKS cluster: Each team or project gets its own dedicated AKS cluster resource to easily track its billing. This approach does not benefit from resource sharing.
2. Dedicated AKS node: Each team or project gets its pods assigned to a specific node in a shared AKS cluster resource using labels and nodeSelector, a configuration constraint. Each team can then isolate its spending according to the cost of its assigned nodes. Resource sharing is not efficient in this model.
3. Dedicated AKS Namespace: Each team or project gets its own dedicated namespace, along with more flexibility. This approach can be challenging for cost allocation—this is where Kubecost comes in.

Measurement Example

Suppose you have chosen to support multiple projects and teams on your AKS cluster using a few dozen namespaces. In this example, each team has its own namespace and autonomous administrative control over provisioning pods and containers. After a couple of months, the cluster costs are higher than expected and you want to identify the cause. It’s a simple question, but complicated to answer.

To start, you allocate the following by namespace:

1. Actual usage of CPU, GPU, memory, network, and persistent storage.
2. Excessive requests for CPU and memory leaving resources unused.
3. Usage of shared resources, such as databases or blob storage.
4. Proportionate use of cloud instance reservations and support services.

Using just network usage as example, you then:

1. Measure the egress traffic generated by containers belonging to each namespace across regions, availability zones, and out to the internet.
2. Determine data transfer rates that may be negotiated with your cloud provider.

Only then can you measure the network usage cost associated with each namespace.

This process is daunting, especially when you consider it requires:

• Telemetry for collecting relevant data in real-time
• Storage of time-series data over time
• Collection of detailed billing logs
• Usage and cost analysis
• Consolidation into a report or dashboard

Mounting Risks

Ignoring this challenge can lead to longterm frustration, as you struggle to effectively allocate costs and pinpoint sources for cost spikes. Misconfiguration-related cost increases are typically a minor operational issue if detected quickly. However, if unnoticed for weeks or months, even a small misconfiguration can lead to significant budget overages. While Kubernetes offers benefits like economies of scale and auto-scaling, it needs effective governance and clear financial management for ongoing and successful use.

Navigating this Challenge

A high-level view of your resource cost and usage, and the ability to drill down into details, is a universal ideal. In the case of AKS, there are two types of summarized views:

1. Azure resources directly or indirectly related to AKS
2. Azure services unrelated to AKS (e.g. other Azure Virtual Machines)

Having both in a single tool mitigates switching between consoles and reconciling complex cloud bills.

The costs directly associated with AKS include the nodes that form the cluster, and the network and persistent storage volumes supporting the nodes. Costs can be broken down into charges associated with:

• CPU
• Memory
• GPU
• Network
• Storage Volumes

There are also external costs associated with clusters or namespaces. These may be dedicated to each namespace or resource, or shared across multiple namespaces. For example, a proxy server may be accessed by all containers regardless of the namespace, and therefore its cost is shared across all namespaces.

Below are types of costs needed for an accurate view of spending by namespace:

Cost Type	Kubernetes Cost Description	Examples
In-Cluster	The cost of the nodes that make up the cluster	CPU, memory, storage, network, and GPU
External	The cost of Azure services dedicated to an individual namespace	Azure SQL, Azure Blob, Azure Synapse Analytics
Shared	The cost of services shared by all namespaces	kube-system namespace, common labor costs

The Kubecost Solution

Kubecost is a popular solution because of its ease of implementation, functionality, and support resources. As pioneers in Kubernetes FinOps, Kubecost helped define many of the modern concepts behind financial reporting on shared Kubernetes resources. Kubecost is installed within the Kubernetes cluster and does not transmit any data outside of your cluster, eliminating the risk of sensitive data leaving your environment.

Azure Integration

Kubecost collects data from the Azure Billing Rate Card API to provide accurate cost information from your billing records. Our Azure integration collects your exact pricing data to account for any special pricing. Additionally, our integration is able to reconcile exact costs of your in-cluster usage and display them alongside out-of-cluster costs for a holistic view of your cloud spend.

Comprehensive Visibility

Once installed, you can access the overview, or drill into cluster details such as total cost, cluster health, efficiency, estimated savings, cost allocation, and more.

Navigating Kubecost

The left side navigation menu presents a number of reports and dashboards available.

These tools include:

• Allocations: Costs by Kubernetes meta-data and Azure tags
• Assets: All Azure spending beyond AKS
• Cloud Costs: Costs for all assets in your connected cloud accounts
• Clusters: All monitored clusters captured in your cloud bill
• Reports: Group, filter, sort detailed cost information and store for later
• Savings: Recommendations for cost savings based on preset best practices
• Alerts: Configured cost alerts based on cluster or namespace budgets
• Health: Cluster health score based on preset best practices

We won’t cover all tools in this article, but will highlight primary ones related to AKS.

Allocations

Costs associated with a Kubernetes cluster fall into the following categories: In-Cluster, External, and Shared. Below illustrates grouping by namespace, including the option to allocate the cost of a shared cluster namespace across other namespaces:

The Allocations UI displays cost categories below:

To access this tool, you must export your Azure cost report into an Azure storage by following these instructions (select the daily month-to-date option for the export and wait a few hours to populate). Next, create a JSON file named azure-storage-config.json with your secret information, as formatted below,** and provide Kubecost secure access via the Azure storage API.

{
	"azureStorageAccount": "<STORAGE_ACCOUNT_NAME>",
	"azureStorageAccessKey": "<STORE_ACCESS_KEY>",
	"azureStorageContainer": <REPORT_CONTAINER_NAME>
}

Next, create the secret using this command:

kubectl create secret generic <SECRET_NAME> --from-file=azure-storage-config.json -n kubecost

Once the secret is created, set .Values.kubecostProductConfigs.azureStorageSecretName to <SECRET_NAME> and upgrade Kubecost via Helm (detailed instructions).

Note: Later in the article, we’ll provide instructions for using helm to install Kubecost.

Cost Breakdown

In the Kubecost UI, you can select just about any line item and drill down for more details. For example, while in the Allocations view, select a specific value for a breakdown of the charges:

Implementing Tags

Now that we’ve covered cost categories and shared Azure costs in Kubecost, we’ll cover best practices for controlling allocations. Kubecost uses tags and labels to allocate Azure spending outside of the Kubernetes cluster to specific Kubernetes components, such as pods and namespaces.

For example, you could take the costs related to a SQL instance and associate them with a specific pod. Suppose you have a namespace called “app-backend” containing microservices with their own dedicated Azure SQL database. This database is provisioned under the Managed Instance model, and you could attribute all its costs to this namespace in the form of external costs. In this scenario, you apply a tag to the SQL database instance with the tag key “kubernetes_namespace” and the tag-value “app-backend.” That’s it – the next time you visit Kubecost and use the Allocations view, the costs of the SQL instance will be shown in the appropriate row, under the column titled “external costs’’.

The table below shows the various tags available in Kubecost:

Kubernetes Component	Azure Key	Azure Value
Cluster	kubernetes_cluster	<cluster-name>
Namespace	kubernetes_namespace	<namespace-name>
Deployment	kubernetes_deployment	<deployment-name>
Label	kubernetes_label_NAME	<label-value>
DaemonSet	kubernetes_daemonset	<daemonset-name>
Pod	kubernetes_pod	<pod-name>
Container	kubernetes_container	<container-name>

You can learn more about tagging using the Azure portal on this page.

Cost Allocation Configuration

Kubecost makes it easy to configure the allocation of shared costs to Kubernetes components.

Shared resources may be in the form of:

1. Fixed dollar value: For example, $1,000 may be shared across all namespaces
2. Shared namespace: For example, the cost of the default “kube-system” namespace may be spread out. The configuration field accepts a list of common-separated namespaces.
3. Share by label: For example, the cost of pods with a label of app=splunk may be distributed. Note: In the screenshot below, the tag key named “app” is repeated and separated by a comma to capture both the “datadog” value for the key named “app” and the value “splunk” for the same key named “app.”

The fixed dollar value is a simple solution for allocating costs of any type. You can attribute the proportional maintenance labor cost of the cluster to each namespace, ensuring your financial report reflects a fully-loaded cost.

The shared namespace feature allows you to take costs associated with multiple namespaces, and distribute them among all other namespaces. A namespace dedicated to hosting security software intended to protect the entire cluster is a great candidate for this feature. Another is kube-system, which is created by the Kubernetes system to host common utilities such as kube-dns and kube-proxy.

Lastly, labels allow you to select just about any Kubernetes component and distribute costs proportionally among them. You may have pods running in a user acceptance testing (UAT) environment. With labels, you can take all your UAT costs and spread them across your production namespaces, so testing costs are included in your application costs.

Shared costs are allocated proportionally to the in-cluster resource cost percentage the namespaces consume (CPU, memory, network, storage, and GPU), so a namespace using more cluster resources gets a higher allocation of shared expenses. However, you can also instruct Kubecost to distribute costs equally across all namespaces for a uniform distribution.

Assets

Assets provides the ability to view all your Azure spending, even if unrelated to AKS. This view supports a full set of controls to filter, group, and sort your Azure bill.

You can also drill into specific billing record details. The following screenshot drills into networking charges, and includes itemized charge:

Installing Kubecost

Get up and running in 5 minutes or less following these steps:

1. Install Kubecost using a Helm chart.
2. Create an Azure role definition for Kubecost in your Azure account.
3. Enter your account information on the Kubecost Settings page.

Once installed in an Azure account, create an Azure role definition to provide access privileges to Kubecost in the form of JSON files. This is required for accessing accurate billing data via the Billing Rate Card API. For more detailed instructions, visit our Docs Site or Microsoft Azure’s instruction page.

Lastly, enter your Azure Subscription ID, Storage Account, Storage Access Key, and Storage Container in the Kubecost’s Settings UI.

Conclusion

Unraveling AKS pricing can be a challenge, but don’t let it be a blocker. Gain visibility of shared AKS resources with Kubecost for deeper cost analysis, and unlock insights for cost savings. Kubecost allows you to monitor your cluster’s health and efficiency, view cost allocations, and send customized alerts. To learn about more Kubecost’s growing list of features, check out our Docs Site.

About us

Kubecost can run across all major cloud providers, including Azure Kubernetes Service (AKS), Google Kubernetes Engine (GKE), Amazon Elastic Kubernetes Service (EKS), and on-prem. With capabilities like multi-cluster visibility and custom reporting, we are tailored to meet the diverse needs of complex enterprises, making us an effective tool for effective Kubernetes cost management and optimization. To learn more, reach out via email (team@kubecost), Slack, or visit our website.