Flux-core: job-manager: allow specific job id's to be listed

[garlick]

The job-manager.list RPC currently lists all jobs, or optionally a maximum number of the highest priority jobs. It accepts a list of attributes to determine what specifically is returned for each job.

It would be handy to allow a json array of jobid's to be optionally passed in, so that the entries for one or more specific jobs can be fetched.

The flux_job_list() API function prototype could be changed to

flux_future_t *flux_job_list (flux_t *h, int max_entries,
                              const char *attribute_list, const char *jobid_list);

The flux job list command usage could then be changed to

Usage: flux-job list [OPTIONS] [ID, ID, ...]
List active jobs
  -c, --count=N          Limit output to N jobs
  -h, --help             Display this message.
  -s, --suppress-header  Suppress printing of header line

[grondo]

It would also be nice if we could pass an alternate sort order for the max_entries jobs returned. Currently you have to list all jobs to get the most recently submitted job.

I'm trying to figure out how the flux_job_list() function could be extended in the future without continuously adding new args. I wonder if flux_job_list() should stay simple and we could introduce a flux_job_query() later (if needed, maybe job listing will be fast enough to do filtering on the client side)

[garlick]

Agreed. "list jobs for userid" is another variant that is likely needed.

Two other points worth bringing up are

• job manager doesn't read jobspec or R so it can't efficiently answer queries based in info in those (even command/job name is unavailable)
• (old idea) dumping job data out to a file such as a sqlite db or even an external nosql database might be useful as the basis for more sophisticated queries than the job manager can provide

Maybe plumbing tool provides job manager only data, porcelain adds in elements from jobspec and R, and enhanced query?

[grondo]

Maybe plumbing tool provides job manager only data, porcelain adds in elements from jobspec and R, and enhanced query?

Yeah, that is where I was going as well. We'll likely want to offload much of this from the job-manager for the "million jobs" case anyway, or the job-manager will spend all its time constructing JSON arrays...

[garlick]

Hmm, maybe a default cap on the number of jobs that can be returned would be useful here, in case people start using flux_job_list() (or flux job list) as the basis for custom job listing scripts? Wouldn't want to be building million entry json arrays unnecessarily?

[grondo]

I keep coming to the same conclusion when I think about how this job listing interface will have to work in the future: we're going to need a different/another interface.

The job list rpc should be able to filter on any job data kept internally by the job manager. We probably have to accept we can't know now what all that data will be for future versions. We can't just keep adding function arguments to flux_job_list() each time there are new parameters passed to the underlying rpc.

We could try to make a more extensible form of flux_job_list(), which might be a bit painful in C (you'd probably start by making a job_query_t object on which you could set flags, options, and jobids, then pass it to a flux_future_t flux_job_query (job_query_t *q, ..).

Alternately, we could keep the flux_job_list() C api function simple (and accept it would not be a commonly used interface), and build a python interface to job listing that takes a dictionary of parameters for filtering, which is much more naturally extensible without breaking everyone, and would be used to build the porcelain job listing command anyway.

[grondo]

maybe a default cap on the number of jobs that can be returned would be useful here

This is not a bad idea. If the rpc also supported a 'start after jobid' and the response added a flag if the listing is truncated, then the job.list rpc could be called multiple times to fetch the entire list without requiring huge json arrays to be constructed.

[grondo]

Now that job-info also supports a list interface, the ideas here should be applied to job-info.list instead/also.

[grondo]

Seems like we've covered the issues described here in the job-info.* service(s). Reopen please if something was missed.