Skip to main content
Version: 1.0.0

Kubernetes Workload Permissions Guide

Overview​

This guide discusses how Kubernetes' RBAC (Role-Based Access Control) interacts with cloud-specific IAM roles, focusing on the Zeet platform. Understanding these interactions is vital for ensuring security, optimizing access, and enhancing deployment processes.

1. Kubernetes RBAC System​

Kubernetes uses RBAC to manage permissions for its resources. This system offers detailed control, setting specific permissions for different roles within a cluster.

1.1. Default RBAC Permissions​

By default, Kubernetes has built-in roles with set permissions. For example, the cluster-admin role has full administrative access. When developers access the cluster using kubectl or lens, they're using the cluster-admin role. Zeet also uses this role to manage and orchestrate container deployments.

1.2. Container Permissions in Kubernetes​

Typically, containers in Kubernetes don't have permissions. This means they can't access the Kubernetes API or change resources directly.

1.3. Customizing RBAC Permissions​

To adjust RBAC permissions:

  1. Define Roles: Start by creating a Role (for specific namespace access) or ClusterRole (for cluster-wide access). These roles set permissions for different resources.
  2. Bind Roles: After creating roles, link them to users or service accounts using RoleBinding (for Role) or ClusterRoleBinding (for ClusterRole).
  3. Limit Access: Only grant required permissions. Overly broad permissions can introduce security risks.

2. Container Networking Permissions​

Kubernetes uses network policies to control how pods communicate within the cluster and with other network points.

2.1. Default Networking Behavior​

Unless a network policy is applied, all pods can freely communicate within the cluster, which helps in smooth internal operations.

2.2. Customizing Network Permissions​

To set a pod's networking permissions:

  1. Create Network Policies: Make a NetworkPolicy that lists the pod selectors and their allowed traffic rules.
  2. Apply Policy: Implement the policy in the needed namespaces or throughout the cluster.
  3. Monitor & Adjust: Regularly check that the policies work as expected and make changes if needed.

3. Node Permissions & Cloud IAM​

When Kubernetes nodes need to access cloud resources, cloud IAM roles dictate this access.

3.1. Default Node IAM Permissions​

By default, nodes can read compute, network, and storage configurations. They don't have permissions to modify these configurations.

3.2. Interaction with Container Permissions​

Containers typically inherit the IAM permissions of nodes. Adjusting this can enhance security.

3.3. Limiting Container Access to Node Permissions​

To restrict containers from inheriting node IAM permissions:

  1. Limit at Metadata Service Level: For example, use AWS IMDSv2 Configuration.
  2. Control at Networking Level: Define a NetworkPolicy that restricts containers from accessing the metadata endpoint.

4. Assigning Custom IAM Permissions to Containers​

When containers need specific cloud API access, grant custom IAM permissions through either direct methods or environment variable injection.

  1. Direct Method: This method provides containers with auto-refreshed, short-lived tokens without needing explicit credentials.

    1. For AWS: See AWS EKS IAM Role with Zeet.
    2. For GCP: Check out the GCP GKE Workload Identity guide.
    3. For Azure: Refer to Azure Managed Identity.

  2. Environment Variables: Obtain cloud credentials and set them as environment variables in your Zeet projects. Ensure you rotate these credentials regularly for security.