Lxd: Add a project limits consumption API and CLI

[stgraber]

Projects have a set of limits that can be applied to them, but users can't easily tell how far along they are in consuming those resources.

We should add a basic API under /1.0/projects/NAME to query the current usage for a project and add a lxc project info CLI option to display it.

[komish]

Am I understanding that the information for the limits already exists, and this feature would simply allow querying and presenting that data? I could work on this, just need to get more familiar with things.

[stgraber]

That's right. The limits are enforced right now so there is existing logic to query the limit and its current usage.

[stgraber]

Assigned it to you.

[stgraber]

The limits currently available are:

• limits.containers
• limits.cpu
• limits.disk
• limits.memory
• limits.networks
• limits.processes
• limits.virtual-machines

The API endpoint to return the limits for a project should:

• Load all instances in the project
• Iterate through them and perform a count based on their type (to get the current usage of limits.containers and limits.virtual-machines)
• Add up all their limits.processes, limits.cpu and limits.memory (to fill in the usage of the same name)

For networks, just load the list of networks in the project and that's going to be your count.

The tricky one of those is limits.diskwhich is the sum of:

• Size of all images in the project
• size of all root disks of instances in the project
• size of all volumes in the project

The existing query logic isn't exactly ideal for this, but we do have existing logic to do everything I described in this comment.
So best may be to just add an API endpoint and just have a function which pulls all the information needed above.

As for the API, GET /1.0/projects/NAME/state would be consistent with what we've done in other places.

I'd expect roughly this set of commits:

• api: projects_usage extension (shared/version/api.go, doc/api-extensions.md)
• doc/rest-api: Add project state endpoint (doc/rest-api.md)
• shared/api: Add ProjectState (shared/api/project.go)
• client: Add GetProjectState (client/interfaces.go client/lxd_projects.go)
• lxc/project: Add info subcommand (lxc/project.go)
• i18n: Update translations templates (po/ as result of make i18n)
• lxd/projects: Add project usage API (lxd/*)
• tests: Add project usage test (test/suites/projects.sh)

[komish]

Thanks for the info. Very much appreciated. It'll likely take me some time and I'm sure to have questions. For starters:

I read this in the documentation:

Note that to be able to set one of the limits.* config keys, all instances in the project must have that same config key defined, either directly or via a profile.

Instances created before a project has a limit defined in config don't seem to have this restriction. That is - they continue to run without the config key. API requests to create instances _were_ met with an error if they didn't contain the config key, so I would guess that the check to make sure an instance in a project is within the upper bound only really takes place at creation time. Does that sound correct?

It makes sense (stopping a workload because it doesn't have a config key setting a limit after the fact could be dangerous, etc). This could impact the accuracy of what's represented by the project limits consumption API, and I want to make sure there's nothing special I need to take into consideration as a result.

[stgraber]

Yeah, that's correct. It's a pretty unusual situation as we normally expect the admin to set the limit prior to anything getting created, but it's certainly a possible situation.

Similarly, it's possible for an admin to lower the project limits after things got created, effectively giving you a situation where the reported usage is higher than the limit. In such a situation, nothing new can get created until the user has done some cleanup.

[komish]

Sorry for the delay in any kind of update, I've been a bit short on time. I know this is listed as a Hacktoberfest issue but if it's alright with the project, I'll likely find more time to make progress on this after the event ends.

[stgraber]

Yep, that's fine with us :)