Aws-iam-authenticator: QUESTION: How to create the k8s users and groups to map to?

[davidham]

I'm new to kubernetes. I want to use my IAM roles to authenticate my users to the cluster. I see how to do this in the aws-iam-authenticator config using mapRoles like so:

mapRoles:
- roleARN: <my role ARN>
  username: some_k8s_user
  groups:
  - groups1
  - group2

For now I want users for:

• cluster admin
• deploy user (write access to a staging namespace)
• dev user (read access to staging, write access to dev namespace)

I also know how to create the k8s user, but I don't understand the groups. Why do I need to specify groups if I'm already specifying a user? Also, how do I know what groups there are and which groups I want my users in?

Sorry if this is the wrong forum for this, and thanks for any advice you can offer.

[mattlandis]

In kubernetes you don't need to create the user/groups. They are determined by an authenticator and just used by kubernetes, so any user/groups returned by your mappings will be 'created' and kubernetes will use those values. To give permissions you can use rbac to create a role and then bind that role to a user or groups. https://kubernetes.io/docs/reference/access-authn-authz/rbac/

The reason you would want to give a user groups is multiple users can be in a group and permissions given to that group instead of redefining permissions for individual users. You also get some audit-ability in having a person as a user belonging to a group that has staging permissions instead of multiple people sharing a single deploy user.

To do what you want you can:

1. Create the mappings for IAM -> username/groups
2. Create roles for staging-writer, staging-reader, dev-writer (examples)
3. Bind the user or groups to the roles (examples)

So authenticator says who you are and RBAC (an authorizer) says what you are allowed to do.

If you have any other questions let me know.

[davidham]

Thanks for the reply. So I have two AWS roles called k8s-admin and k8s-user that I have in mapRoles like this:

mapRoles:
- roleARN: arn:aws:iam::000000000000:role/k8s-admin
   username: k8s-admin
   groups:
   - system:masters
- roleARN: arn:aws:iam::556767347111:role/k8s-user
   username: k8s-user
   groups:
   - system:masters

I have a ClusterRoleBinding for k8s-admin like this:

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: k8s-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: User
  name: k8s-admin

and this works fine, I can log in as this user with AWS creds. However, my other AWS role, k8s-user, I want to have more limited permissions. I defined two Roles and two RoleBindings, like this:

# This file defines:
# - staging-read role, allowing read access to the staging namespace
# - dev-all-access role, allowing all access to dev namespace
# - RoleBindings connecting this role to the k8s-user

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: staging-read
  namespace: staging
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["*"]
  verbs: ["get", "list", "watch"]
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dev-all-access
  namespace: dev
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["get", "list", "watch", "create", "update", "delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: k8s-user-staging-read-binding
  namespace: staging
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: staging-read
subjects:
- kind: User
  name: k8s-user
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: k8s-user-dev-all-access-binding
  namespace: dev
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: dev-all-access
subjects:
- kind: User
  name: k8s-user

And I can't even authenticate with it. when I run kubectl version I get this:

error: You must be logged in to the server (the server has asked for the client to provide credentials)

Am I writing my Roles/bindings wrong?

Do I need to restart the aws-iam-authenticator daemonset? If so, how?

And I'd like to set up my own groups, like mycompany:engineering. If I do that, do I define them in my RoleBinding and then map the IAM role to it in the aws-iam-authenticator config?

Thanks for your help, I'm struggling with this today.

[davidham]

I got it to work! For anyone else finding this issue, here's what I did. In my configmap for aws-iam-authenticator, I set up a role like this:

mapRoles:
- roleARN: arn:aws:iam::000000000000:role/k8s-admin
   username: k8s-admin
   groups:
   - system:masters # this one worked fine
- roleARN: arn:aws:iam::000000000000:role/k8s-user
   username: k8s-user
   groups:
   - mycompany:engineering # this was my new one that I had trouble with

Then, in my RoleBinding, I did this:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: k8s-user-dev-all-access-binding
  namespace: dev
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: dev-all-access
subjects:
- kind: Group
  name: mycompany:engineering # This is the link between k8s and the IAM role I mapped in the mapRoles bit

The part that made it work was that I did in fact have to rollout the aws-iam-authenticator DaemonSet. I did this by changing the resource requests slightly (20Mi to 21Mi).

[venkyvb]

@davidham => Just to understand, is the mycompany:engineering is an IAM group in AWS? Where exactly is this defined?

[davidham]

No, it's not an IAM group, it's a Kubernetes group. So the first snippet maps the k8s-user role to the mycompany:engineering group, and the second, the RoleBinding, maps the group to the Kubernetes dev-all-access role.

[kenske]

@davidham thanks, this worked for me. What I still don't understand is why you need to specify a username in the configmap? Seems like the username is irrelevant since it's going to get mapped to the group.

[davidham]

Not sure if it's a required field or not. But you're right in that the main mapping seems to be between the IAM role and the k8s group.

[mr-karan]

@davidham Thanks for this config, it helped me fix a very silly mistake of mine after hours of debugging :D
Was using kind:ServiceAccount in the rolebinding, instead of kind:Group which I guess for AWS IAM Authenticator is necessary to map the IAM user to a "group" not a service account. Sigh. Heavily undocumented stuff though.

[jansony1]

No, it's not an IAM group, it's a Kubernetes group. So the first snippet maps the k8s-user role to the mycompany:engineering group, and the second, the RoleBinding, maps the group to the Kubernetes dev-all-access role.

Hi David, where you create this mycompany:engineering group?

[davidham]

@jansony1 that was the weird part, you don't actually create it anywhere, or I didn't anyway. You just specify it in these two k8s objects. You mapRole to the group name in one place, and then you use the same group name in the RoleBinding under subjects.

[haani-niyaz]

There are 2 parts to this; Authentication and Authorization. The authentication part is managed by AWS and the authorization part is via an authorization plugin in k8s. When authenticated, the username and group is passed to k8s which is checked against RBAC.

The group is passed in the API request (as per aws-auth ConfigMap) and on successful authentication, it is validated against the RoleBinding.

[SudoBrendan]

Just want to leave a note for lost souls - I discovered through the aws-iam-authenticator logs that if your Roles include paths (and those would be visible in the ARN displayed in console - e.g. arn:aws:iam::12345:role/my/path/foo), you do NOT include the path in the configmap mappings. For example, in our aws-iam-authenticator configmap, the roleMapping had - roleARN: arn:aws:iam::12345:role/foo instead of - roleARN: arn:aws:iam::12345:role/my/path/foo