Skip to main content
Version: 1.0.0

Kubernetes Workload Permissions Guide


This guide discusses how Kubernetes' RBAC (Role-Based Access Control) interacts with cloud-specific IAM roles, focusing on the Zeet platform. Understanding these interactions is vital for ensuring security, optimizing access, and enhancing deployment processes.

1. Kubernetes RBAC System

Kubernetes uses RBAC to manage permissions for its resources. This system offers detailed control, setting specific permissions for different roles within a cluster.

1.1. Default RBAC Permissions

By default, Kubernetes has built-in roles with set permissions. For example, the cluster-admin role has full administrative access. When developers access the cluster using kubectl or lens, they're using the cluster-admin role. Zeet also uses this role to manage and orchestrate container deployments.

1.2. Container Permissions in Kubernetes

Typically, containers in Kubernetes don't have permissions. This means they can't access the Kubernetes API or change resources directly.

1.3. Customizing RBAC Permissions

To adjust RBAC permissions:

  1. Define Roles: Start by creating a Role (for specific namespace access) or ClusterRole (for cluster-wide access). These roles set permissions for different resources.
  2. Bind Roles: After creating roles, link them to users or service accounts using RoleBinding (for Role) or ClusterRoleBinding (for ClusterRole).
  3. Limit Access: Only grant required permissions. Overly broad permissions can introduce security risks.

2. Container Networking Permissions

Kubernetes uses network policies to control how pods communicate within the cluster and with other network points.

2.1. Default Networking Behavior

Unless a network policy is applied, all pods can freely communicate within the cluster, which helps in smooth internal operations.

2.2. Customizing Network Permissions

To set a pod's networking permissions:

  1. Create Network Policies: Make a NetworkPolicy that lists the pod selectors and their allowed traffic rules.
  2. Apply Policy: Implement the policy in the needed namespaces or throughout the cluster.
  3. Monitor & Adjust: Regularly check that the policies work as expected and make changes if needed.

3. Node Permissions & Cloud IAM

When Kubernetes nodes need to access cloud resources, cloud IAM roles dictate this access.

3.1. Default Node IAM Permissions

By default, nodes can read compute, network, and storage configurations. They don't have permissions to modify these configurations.

3.2. Interaction with Container Permissions

Containers typically inherit the IAM permissions of nodes. Adjusting this can enhance security.

3.3. Limiting Container Access to Node Permissions

To restrict containers from inheriting node IAM permissions:

  1. Limit at Metadata Service Level: For example, use AWS IMDSv2 Configuration.
  2. Control at Networking Level: Define a NetworkPolicy that restricts containers from accessing the metadata endpoint.

4. Assigning Custom IAM Permissions to Containers

When containers need specific cloud API access, grant custom IAM permissions through either direct methods or environment variable injection.

  1. Direct Method: This method provides containers with auto-refreshed, short-lived tokens without needing explicit credentials.

    1. For AWS: See AWS EKS IAM Role with Zeet.
    2. For GCP: Check out the GCP GKE Workload Identity guide.
    3. For Azure: Refer to Azure Managed Identity.
  2. Environment Variables: Obtain cloud credentials and set them as environment variables in your Zeet projects. Ensure you rotate these credentials regularly for security.