Nomad: [question] Vault Integration with Token Role

[stevenscg]

Based on a conversation between mafonso and myself on gitter chat starting ~2017-02-14 09:20, I wonder if the website is correct or complete around the vault integration with a token role.

I have struggled to use an instance token (generated via the vault aws-ec2 backend) to create a token for nomad per the docs if disallowed_policies is set to nomad-server.

I think part of the issue is that the aws-ec2 vault backend does not allow a periodic token like the approle does or the token backend does. If vault were to allow this, I think nomad could directly use the instance token without the additional token-create step.

If that is not possible or not correct, I think it might be beneficial to document how this flow should work.

In the interim, I am testing a nomad-cluster token role without any allowed or disallowed policies, but don't want to run with this long-term.

Website / Doc reference: https://www.nomadproject.io/docs/vault-integration/index.html

It looks like several docs changes were recently rolled out per https://github.com/hashicorp/nomad/pull/2226.

Config

These are yaml, but reflect what is currently in the docs and what I was using when encountering the issues:

- path: auth/aws-ec2/role/nomad-server
  data:
    role: nomad-server
    bound_iam_instance_profile_arn: arn:aws:iam::XXXXXX
    policies: "default,nomad-server"

- path: auth/token/roles/nomad-cluster
  data:
    name: nomad-cluster
    explicit_max_ttl: 0
    orphan: false
    period: 72h
    renewable: true
    disallowed_policies: nomad-server

Steps

1) An instance token is created by the vault aws-ec2 backend using the auth/aws-ec2/role/nomad-server role listed above.

2) On startup, a nomad wrapper script uses the instance token to call auth/token/create/nomad-cluster and vault returns errors:

{
  "errors": [
    "token policy \"nomad-server\" is disallowed by this role"
  ]
}

By recreating the nomad-cluster token role without any disallowed policies, I am able to create a valid token and give that token to Nomad.

Nomad version

Nomad v0.5.5-dev (a16709ef4360ec4e453ec4560fe1bbebf3cb3be5)

Vault v0.6.5
Consul v0.7.4

[dadgar]

@stevenscg The newest version of Vault supports periodic tokens from ec2 auth backend: https://www.vaultproject.io/docs/auth/aws-ec2.html#auth-aws-ec2-role-role

So the issue is step 2. What you should do is just give Nomad the token generated in step 1.

[stevenscg]

@dadgar That will be perfect! TBH, I think I've had that page up in a tab for a few days, so probably missed the refresh or cache expiration.

[dadgar]

@stevenscg No worries! Let me know if you run into any hiccups, if not would you mind closing this issue when you get it working!

[stevenscg]

Yeah, I will close it here shortly I think. Real-time devops, love it!

[stevenscg]

Confirmed: Using a periodic instance token (generated via aws-ec2 backend on vault v0.6.5) with nomad works as expected and as documented.

Config as tested:

- path: auth/aws-ec2/role/nomad-server
  data:
    role: nomad-server
    bound_iam_instance_profile_arn: arn:aws:iam::XXXXXX
    period: 72h
    policies: "default,nomad-server"

- path: auth/token/roles/nomad-cluster
  data:
    name: nomad-cluster
    explicit_max_ttl: 0
    orphan: false
    period: 48h
    renewable: true
    disallowed_policies: nomad-server