Swarmkit: Provide initial set of node labels during swarm join

[alexmavr]

As of now, setting a node's labels is a task that can be performed by a manager through docker node update after a node has already joined the cluster. However, the following scenarios would benefit from the ability to specify an initial set of labels through something similar to docker swarm join --label key=value:

• A global or replicated service with a constraint of node.label.key!=value might deploy a task in the new node before the operator labels the node with key=value
• The formation of a multi-node cluster where a labelling scheme is known a priori requires an open connection to a manager node just for labelling, assuming secrets or autoaccept are used.
• A global service with a node.role==manager constraint which creates containers that bind-mount the docker socket could utilize an initial set of node labels for node-related metadata.

[Richard-Mathie]

This would be so useful for provisioning nodes using a aws cloud formation template as the node can advertise its configuration and capabilities. (pub/private network, storage backend ect...)...

[aluzzardi]

@alexmavr @Richard-Mathie You can do that by setting engine labels.

1) start docker engine with labels, e.g. dockerd --label foo=bar
2) use constraints with engine.labels.key==value (note I'm using engine rather than node)

[aluzzardi]

/cc @diogomonica

[diogomonica]

@aluzzardi would love to hear alternative suggestions on how to drive this. We need manager set labels and engine set labels. One possibility is to merge them (if the node label doesn't exist, try to match the engine next). That would achieve our goal, but might be confusing (least-surprise)

[blieberman]

+1

[borfast]

I know it has been over a year since this has been discussed but I'm curious as to the state of this request because I am in need of the same. Is it still being considered or has it been dropped?

[stevvooe]

@borfast Did you try the suggestion in https://github.com/docker/swarmkit/issues/1160#issuecomment-237727853? The main issue with this proposal is that "node labels" are set by the operator, not the node and allowing a node to do that would be against the security model.

Engine labels are exactly for this task.

If engine labels don't work, we'd be interested in hearing an alternative proposal.

[borfast]

@stevvooe, I did not try that. It would require that I modify the default Docker installation on each node and that feels quite hacky and cumbersome. Besides, I don't want to set a system-wide label on Docker because I may need to run other things there which may clash with that label.

I have automated infrastructure creation and configuration with Terraform and Ansible for a non-trivial range of services that require various node types (more memory, more CPU, etc). I need to be able to set the node labels before deploying the services to them so I can define deployment constraints (some services need more RAM, others more CPU).

Right now I resorted to:

1. giving each node a name by setting its hostname;
2. on a manager node run docker node ls --filter "name=<node_name>" --format="{{ '{{' }} .ID {{ '}}' }}", which gives me the node ID;
3. finally, using that ID I can set the node labels using docker node update.

But this is obviously hacky as hell and I don't feel comfortable with it at all. Hacking the default docker installation doesn't sound much better, though, and it just doesn't feel right. If nothing else because I will be setting a label for the whole Docker engine and thus affect every single container on that machine.

[pepijnblom]

Is there no solution to this yet? We are running into the same problem with autoscaling instance groups on Google Cloud.