If kube-state-metrics is running but not showing useful data, RBAC is often the silent reason.
Most setups fail here, not because the tool is complex, but because permissions are either too restrictive or too open. Both cause problems. Either metrics do not show up, or your cluster becomes unnecessarily exposed.
This guide covers kube-state-metrics RBAC with a clear focus on setup and real-world usage, helping you understand how everything fits together.
What RBAC Means in Kubernetes
RBAC stands for Role-Based Access Control. In simple terms, it decides:
- what kube-state-metrics can read
- which resources it can access
- how wide its visibility is inside your cluster
Kube-state-metrics does not create or modify anything. It only reads cluster state and converts it into metrics. But even read-only access must be explicitly allowed.
Without RBAC:
- it cannot see resources
- it cannot expose meaningful metrics
- Prometheus ends up scraping empty or incomplete data
So RBAC is not optional. It is foundational.
Why Kube-State-Metrics Needs RBAC at All
Kube-state-metrics pulls information from the Kubernetes API.
That includes:
- pods
- deployments
- nodes
- services
- replica sets
- namespaces
Each of these resources requires permission to be accessed, even for reading. Think of it this way:
kube-state-metrics is like a reporter. RBAC is its press pass.
Without the pass, it sees nothing.
Kube State Metrics RBAC: Core Components Explained
To understand the setup, you need to know four key pieces.
1. Service Account
This is the identity used by kube-state-metrics inside the cluster. It tells Kubernetes:
“this pod is requesting access”
Without a service account, permissions cannot be applied cleanly.
2. Role vs ClusterRole
- Role → applies to a single namespace
- ClusterRole → applies across the entire cluster
Kube-state-metrics usually need cluster-wide visibility, so you use a ClusterRole.
3. RoleBinding vs ClusterRoleBinding
These connect permissions to the service account.
- RoleBinding → namespace scope
- ClusterRoleBinding → cluster-wide scope
For kube-state-metrics, you typically use:
ClusterRoleBinding
4. Rules (The Actual Permissions)
This is where access is defined.
Example:
- apiGroups
- resources
- verbs (like get, list, watch)
These rules control exactly what kube-state-metrics can read.
How Kube-State-Metrics RBAC Works (Simple Flow)
Here is what actually happens behind the scenes, step by step, in a way that is easy to follow.
Step 1: Pod Starts with a Service Account
Kube-state-metrics runs inside your cluster as a pod, and every pod needs an identity. This identity comes from a service account, which tells Kubernetes who the pod is. Without this, the pod would not be able to request anything from the API.
Step 2: RBAC Assigns Permissions
Once the identity is in place, RBAC decides what that identity can do. A ClusterRole defines which resources can be accessed, usually with read-only permissions. Then a ClusterRoleBinding connects that role to the service account, making those permissions active.
Step 3: It Queries the Kubernetes API
With permissions set, kube-state-metrics starts talking to the Kubernetes API. It requests details about resources like pods, nodes, deployments, and more. This is simply a read process. It is observing the current state, not making changes.
Step 4: Kubernetes Validates Access
Every request made to the API is checked by RBAC before it goes through. Kubernetes looks at the request and confirms whether it is allowed, whether it is read-only, and whether it matches the defined scope. If something does not match the rules, the request is denied right there.
Step 5: Metrics Are Generated
Once access is approved, kube-state-metrics takes that data and turns it into metrics. These metrics are structured in a way that Prometheus can easily scrape and store. If some data is missing here, it usually means it was never allowed through RBAC in the first place.
Kube-State-Metrics RBAC Setup (Practical Example)
Here is a simplified, clean RBAC configuration:
apiVersion: v1
kind: ServiceAccount
metadata:
name: kube-state-metrics
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kube-state-metrics
rules:
- apiGroups: [""]
resources:
- pods
- nodes
- services
- namespaces
verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
resources:
- deployments
- replicasets
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kube-state-metrics
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kube-state-metrics
subjects:
- kind: ServiceAccount
name: kube-state-metrics
namespace: kube-system
What This Setup Does
- Creates a service account
- Grants read-only access to key resources
- Applies permissions cluster-wide
- Keeps everything minimal and controlled
This is the foundation. You expand only when needed.
How Much Access Should You Give?
This is where most setups go wrong.
Too Little Access
Results:
- missing metrics
- incomplete dashboards
- misleading alerts
Example:
If you do not allow access to deployments, replica metrics will be missing.
Too Much Access
Results:
- unnecessary exposure
- harder audits
- increased risk surface
Even read-only access should be intentional.
The Right Approach
Start with:
- core resources
- read-only permissions
- minimal scope
Then expand based on real needs.
Common Mistakes in Kube State Metrics RBAC
1. Using Default or Over-Permissive Roles
Many setups copy full-access roles without thinking. This works but it is not safe or clean. It gives access to resources you may never use, increasing unnecessary exposure.
2. Missing Required Resources
If you forget to include specific resources:
- metrics silently disappear
- dashboards break without obvious errors
This is frustrating because nothing “fails”, it just does not work fully.
3. Confusing Namespace vs Cluster Scope
Using Role instead of ClusterRole limits visibility. Kube-state-metrics then only sees one namespace, which defeats its purpose in most setups.
4. Forgetting to Bind the Role
You can define the perfect role, but without binding:
- it does nothing
- permissions are never applied
This is one of the most common setup errors.
5. Ignoring RBAC When Troubleshooting
When metrics are missing, people check:
- Prometheus
- dashboards
- queries
But not RBAC. In reality, it is often the root cause.
How RBAC Impacts Your Monitoring Quality
RBAC directly affects:
Data Accuracy
If permissions are incomplete, your metrics are incomplete.
Alert Reliability
Missing data leads to:
- false negatives
- blind spots
Dashboard Clarity
If resources are not accessible:
- graphs look empty
- trends become misleading
System Trust
If your monitoring is not reliable, decisions based on it are flawed. RBAC plays a bigger role than most teams realize.
When You Should Customize RBAC Further
Sometimes the default RBAC setup works fine. But as your cluster grows or becomes more complex, you may need to adjust it to keep things efficient and secure.
You Run Large Clusters
Larger clusters mean more resources to track, which increases API requests. You may need to expand or fine-tune permissions to avoid missing data or slow responses.
You Use Custom Resources (CRDs)
Custom resources are not included by default. You must explicitly allow access to them, otherwise they will not show up in your metrics.
You Want Fine-Grained Control
RBAC lets you limit what kube-state-metrics can see. You can restrict access to certain resources or namespaces to improve security and control.
You Optimize for Performance
Allowing only necessary resources reduces extra API calls. This keeps scraping faster and avoids putting unnecessary load on the cluster.
Conclusion
RBAC is not isolated. It is directly tied to how kube-state-metrics functions. The full flow (from API access to Prometheus scraping) becomes easier to follow when you understand the system’s architecture, where RBAC fits in as a control layer.
FAQ Section
1. Does kube-state-metrics need write permissions?
No. It only needs read access. Permissions like get, list, and watch are enough.
2. Can kube-state-metrics work without RBAC?
No. Without RBAC, it cannot access the Kubernetes API, so it cannot generate metrics.
3. Should I use Role or ClusterRole?
In most cases, ClusterRole is required because kube-state-metrics needs cluster-wide visibility.
4. Why are some metrics missing even though it is running?
Most likely due to missing RBAC permissions for specific resources.
5. Is it safe to give full read access?
It works, but it is not ideal. Always aim for minimal, necessary permissions.