Kube-State-Metrics RBAC Setup Explained Clearly

If kube-state-metrics is running but not showing useful data, RBAC is often the silent reason.

Most setups fail here, not because the tool is complex, but because permissions are either too restrictive or too open. Both cause problems. Either metrics do not show up, or your cluster becomes unnecessarily exposed.

This guide covers kube-state-metrics RBAC with a clear focus on setup and real-world usage, helping you understand how everything fits together.

What RBAC Means in Kubernetes

RBAC stands for Role-Based Access Control. In simple terms, it decides:

  • what kube-state-metrics can read
  • which resources it can access
  • how wide its visibility is inside your cluster

Kube-state-metrics does not create or modify anything. It only reads cluster state and converts it into metrics. But even read-only access must be explicitly allowed.

Without RBAC:

  • it cannot see resources
  • it cannot expose meaningful metrics
  • Prometheus ends up scraping empty or incomplete data

So RBAC is not optional. It is foundational.

Why Kube-State-Metrics Needs RBAC at All

Kube-state-metrics pulls information from the Kubernetes API.

That includes:

  • pods
  • deployments
  • nodes
  • services
  • replica sets
  • namespaces

Each of these resources requires permission to be accessed, even for reading. Think of it this way: 

kube-state-metrics is like a reporter. RBAC is its press pass.

Without the pass, it sees nothing.

Kube State Metrics RBAC: Core Components Explained

To understand the setup, you need to know four key pieces.

1. Service Account

This is the identity used by kube-state-metrics inside the cluster. It tells Kubernetes:

“this pod is requesting access”

Without a service account, permissions cannot be applied cleanly.

2. Role vs ClusterRole

  • Role → applies to a single namespace
  • ClusterRole → applies across the entire cluster

Kube-state-metrics usually need cluster-wide visibility, so you use a ClusterRole.

3. RoleBinding vs ClusterRoleBinding

These connect permissions to the service account.

  • RoleBinding → namespace scope
  • ClusterRoleBinding → cluster-wide scope

For kube-state-metrics, you typically use:

ClusterRoleBinding

4. Rules (The Actual Permissions)

This is where access is defined.

Example:

  • apiGroups
  • resources
  • verbs (like get, list, watch)

These rules control exactly what kube-state-metrics can read.

How Kube-State-Metrics RBAC Works (Simple Flow)

Here is what actually happens behind the scenes, step by step, in a way that is easy to follow.

Step 1: Pod Starts with a Service Account

Kube-state-metrics runs inside your cluster as a pod, and every pod needs an identity. This identity comes from a service account, which tells Kubernetes who the pod is. Without this, the pod would not be able to request anything from the API.

Step 2: RBAC Assigns Permissions

Once the identity is in place, RBAC decides what that identity can do. A ClusterRole defines which resources can be accessed, usually with read-only permissions. Then a ClusterRoleBinding connects that role to the service account, making those permissions active.

Step 3: It Queries the Kubernetes API

With permissions set, kube-state-metrics starts talking to the Kubernetes API. It requests details about resources like pods, nodes, deployments, and more. This is simply a read process. It is observing the current state, not making changes.

Step 4: Kubernetes Validates Access

Every request made to the API is checked by RBAC before it goes through. Kubernetes looks at the request and confirms whether it is allowed, whether it is read-only, and whether it matches the defined scope. If something does not match the rules, the request is denied right there.

Step 5: Metrics Are Generated

Once access is approved, kube-state-metrics takes that data and turns it into metrics. These metrics are structured in a way that Prometheus can easily scrape and store. If some data is missing here, it usually means it was never allowed through RBAC in the first place.

Kube-State-Metrics RBAC Setup (Practical Example)

Here is a simplified, clean RBAC configuration:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-state-metrics
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-state-metrics
rules:
  - apiGroups: [""]
    resources:
      - pods
      - nodes
      - services
      - namespaces
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources:
      - deployments
      - replicasets
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
  - kind: ServiceAccount
    name: kube-state-metrics
    namespace: kube-system

What This Setup Does

  • Creates a service account
  • Grants read-only access to key resources
  • Applies permissions cluster-wide
  • Keeps everything minimal and controlled

This is the foundation. You expand only when needed.

How Much Access Should You Give?

This is where most setups go wrong.

Too Little Access

Results:

  • missing metrics
  • incomplete dashboards
  • misleading alerts

Example:

If you do not allow access to deployments, replica metrics will be missing.

Too Much Access

Results:

  • unnecessary exposure
  • harder audits
  • increased risk surface

Even read-only access should be intentional.

The Right Approach

Start with:

  • core resources
  • read-only permissions
  • minimal scope

Then expand based on real needs.

Common Mistakes in Kube State Metrics RBAC

1. Using Default or Over-Permissive Roles

Many setups copy full-access roles without thinking. This works but it is not safe or clean. It gives access to resources you may never use, increasing unnecessary exposure.

2. Missing Required Resources

If you forget to include specific resources:

  • metrics silently disappear
  • dashboards break without obvious errors

This is frustrating because nothing “fails”, it just does not work fully.

3. Confusing Namespace vs Cluster Scope

Using Role instead of ClusterRole limits visibility. Kube-state-metrics then only sees one namespace, which defeats its purpose in most setups.

4. Forgetting to Bind the Role

You can define the perfect role, but without binding:

  • it does nothing
  • permissions are never applied

This is one of the most common setup errors.

5. Ignoring RBAC When Troubleshooting

When metrics are missing, people check:

  • Prometheus
  • dashboards
  • queries

But not RBAC. In reality, it is often the root cause.

How RBAC Impacts Your Monitoring Quality

RBAC directly affects:

Data Accuracy

If permissions are incomplete, your metrics are incomplete.

Alert Reliability

Missing data leads to:

  • false negatives
  • blind spots

Dashboard Clarity

If resources are not accessible:

  • graphs look empty
  • trends become misleading

System Trust

If your monitoring is not reliable, decisions based on it are flawed. RBAC plays a bigger role than most teams realize.

When You Should Customize RBAC Further

Sometimes the default RBAC setup works fine. But as your cluster grows or becomes more complex, you may need to adjust it to keep things efficient and secure.

You Run Large Clusters

Larger clusters mean more resources to track, which increases API requests. You may need to expand or fine-tune permissions to avoid missing data or slow responses.

You Use Custom Resources (CRDs)

Custom resources are not included by default. You must explicitly allow access to them, otherwise they will not show up in your metrics.

You Want Fine-Grained Control

RBAC lets you limit what kube-state-metrics can see. You can restrict access to certain resources or namespaces to improve security and control.

You Optimize for Performance

Allowing only necessary resources reduces extra API calls. This keeps scraping faster and avoids putting unnecessary load on the cluster.

Conclusion

RBAC is not isolated. It is directly tied to how kube-state-metrics functions. The full flow (from API access to Prometheus scraping) becomes easier to follow when you understand the system’s architecture, where RBAC fits in as a control layer.

FAQ Section

1. Does kube-state-metrics need write permissions?

No. It only needs read access. Permissions like get, list, and watch are enough.

2. Can kube-state-metrics work without RBAC?

No. Without RBAC, it cannot access the Kubernetes API, so it cannot generate metrics.

3. Should I use Role or ClusterRole?

In most cases, ClusterRole is required because kube-state-metrics needs cluster-wide visibility.

4. Why are some metrics missing even though it is running?

Most likely due to missing RBAC permissions for specific resources.

5. Is it safe to give full read access?

It works, but it is not ideal. Always aim for minimal, necessary permissions.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top