Nomad: Support max per key constraint

[autocracy]

The simple version of this is that each rack with in our datacenter will have its own separate meta.rack attribute. By example, we might have 10 task groups to be run, but with a maximum of 3 scheduled on any given rack.

[dadgar]

Hey,

This is a great idea. Global constraints like this is something we want in the future but are not actively working on in the short term.

[timperrett]

@dadgar any chance to bump this up? Spreading things evenly among racks, coupled with nomad's aggressive preference to bin-pack, makes this a tough spot for services requiring redundancy.

[dvusboy]

So, neither distinct_host nor distinct_property would work? The latter one can be used to cover anything from different racks to different availability zones, depending on how you set up your agents.

[jippi]

I would love to have distinct_property on aws.az with limit of 3

to never run more than3 allocs on the job per az - or even better for my usecase

split_property on aws.az where nomad will do best effort to split allocs between the AZ's in my cluster like an AWS ASG does

[autocracy]

At the time this was filed, distinct_property didn't exist. If changed to max_property with a check like https://github.com/hashicorp/nomad/pull/2418/files#diff-b141e83fa6e3c9f720ea4148a0134a4eR285 where the number is the value rather than 0, we might have it.

[jippi]

That would be so badass !

[timperrett]

Agreed this would be splendid. Use case is similar to what @jippi highlights with split_property: to say I want X container count, but i want them split as fairly as possible over a set of AZs (or racks).

@autocracy my reading of your comment is that it would solve @jippi's first case, but not the case I just stated (which seems like a case most people running services would want). Is that accurate?

[dvusboy]

@jippi When you say distinct_property on aws.az with limit of 3, you're not saying count = 3, right? You may have count = 12, but you want the tasks to be distributed to no more than 3 distinct aws.az, right? Bin-packing those 4 instances within a single AZ is OK?

[jippi]

@dvusboy yes :)

[dadgar]

@timperrett Sure. Will try to get this in 0.6.1

[timperrett]

@dadgar you sir, are a gent! that would be awesome!!!

[dvusboy]

And it's not just Alex, all the Nomad devs have been extremely responsive and helpful.