Learn how to integrate Kubecost with Amazon Managed Service for Prometheus to monitor your Kubernetes cost at scale effectively
When you install Amazon EKS cost monitoring with Kubecost on your Amazon EKS cluster, the open-source project Prometheus is installed by default (bundled Prometheus). Kubecost leverages Prometheus as a time-series database and uses the metrics inside Prometheus to perform cost allocation calculations and generate cost optimization insights. Prometheus server is a single binary, making it easy to implement and working perfectly for cloud-native environments. However, it also introduces some trade-offs: as your Amazon EKS cluster grows and scales out, it could exceed the scraping capabilities of a single Prometheus server instance, and scaling the Prometheus instance is complicated. Therefore, many customers tell us they are looking for a solution to remove the operational pain of maintaining and scaling the Prometheus instance. As part of the partnership with AWS, Kubecost delivers the subsequent integration with Amazon Managed Service for Prometheus (AMP) to help customers effectively monitor their Kubernetes without worrying about scaling the Prometheus instance. Amazon Managed Service for Prometheus is a serverless, Prometheus-compatible monitoring service for container metrics that makes it easier to monitor container environments at scale securely.
How this new integration helps Amazon EKS customers
Auto scaling: By integrating with AMP, Kubecost queries the metrics from the remote AMP instance managed by AWS. The AMP service automatically scales operational metrics’ ingestion, storage, and querying as workloads grow or shrink.
Passwordless communication: When Kubecost queries the metrics from AMP or the Prometheus instance remote-write the metrics to AMP, the request is signed by using AWS Signature Version 4 (SigV4) signing process which allows the HTTP request authenticated with AMP instance securely. In addition, by using IAM roles for Service Account (IRSA), Kubecost and Prometheus use IAM temporary role credentials, which are generated and managed by AWS IAM services. It helps to reduce the attack surface and the risk of exposing your AWS access/secret keys.
Best in class customer service: this integration will receive both AWS support (depending on the customer’s AWS support contract) and the Kubecost support team.
Simple installation: Customers can integrate Kubecost with AMP with only a few installation commands.
Let’s take a look at customer experience in the next section.
To integrate Kubecost with AMP, customers need to follow these steps:
- Create new AMP workspace
- Install Amazon EKS cost monitoring with Kubecost
- Set up IAM role for Kubecost Service Accounts
- Integrate Kubecost with AMP
For full installation instructions, customers can check out the documentation at: Amazon Managed Service for Prometheus
In the next section, we will review the architecture of this integration.
Kubecost AMP architecture overview
The architecture of this integration is similar to Amazon EKS cost monitoring with Kubecost, which is described in the previous blog post. with some enhancements as follows:
- In this integration, an additional AWS SigV4 container is added to the cost-analyzer pod, acting as a proxy to help query metrics from AMP using the AWS sigV4 signing process. It enables passwordless authentication to reduce the risk of exposing your AWS credentials.
- Bundled Prometheus server has remote_write mode enabled to send the metrics and cost allocation data to AMP using AWS sigV4 signing process. Now, all metrics and data are stored and managed in AMP so Kubecost can query the metrics directly from AMP instead of the bundled Prometheus. It helps customers not worry about maintaining and scaling the local Prometheus instance when their Amazon EKS cluster grows.
The architecture of this integration is illustrated in the following diagram:
To learn more about the installation process and this collaboration, customers can refer to these additional resources:
- AWS blog post
- Amazon EKS documentation
- AWS container from the couch recorded live session
- Kubecost documentation
- Kubecost and AWS collaborate to deliver cost monitoring for EKS customers
Customers can also reach out to the Kubecost team for more information here