Caseflow: Be more selective about which appeals to update representation for nightly

[lowellrex]

9190 added a job that runs nightly, hits BGS to see if appellant information for the case has changed, and updates TrackVeteranTasks that give visibility into appeals in Caseflow for VSOs if it has. Currently, that job makes requests for every AMA appeal that is active at the Board. This number is small at the moment (Appeal.active.count.keys.count => 1413 at the moment) and so the job runs relatively quickly (18 minutes last night or ~1 second per 100 appeals). As the number of appeals this job affects grows (by including legacy appeals or by the number of AMA appeals growing), the current approach will no longer work.

This ticket exists to explore other ways of updating representative information that we get from BGS and cache in Caseflow. Some ideas include:

• Running the job more frequently (hourly?) on a smaller subset of active appeals (1000?)
• Making a distinction between dormant appeals and more active ones (appeals with tasks that have changed in the past week vs those that haven't, for instance) and updating those different sets of appeals on different schedules (weekly vs. nightly?)

Acceptance criteria

• [ ] Update UpdateAppellantRepresentationJob to handle a dramatic increase in number of appeals processed.

[mdbenjam]

I like both of these ideas. I think for now a first pass can be bullet 1. But in the future I think doing something with bullet 2 to reduce the number of calls we need to make would be 👌 . I can take over bullet 1 for now since we need to expand this job for hearing schedule.

[mdbenjam]

Been thinking more about this. Here is my plan.

1) Add a new table, RecordUpdatedByJob, with a polymorphic association record_type, record_id, and with the asyncable columns submitted_at, attempted_at, processed_at, error.
2) Add a has_one relation between both legacy appeal and appeal to this new RecordUpdatedByJob table.
3) In the UpdateAppellantRepresentationJob set a limit of 2000 cases to be worked every time it is run. Allocate these cases between the legacy and ama appeals. i.e. 2000 * legacy / ( legacy + ama ) and then order the appeals and legacy appeals by the date of processed_at. Appeals that haven't been processed or were processed a while ago get priority, up to the limit calculated above.
4) Update the job to run hourly instead of daily.

[lowellrex]

I really like that. I think it would have made sense for UpdateAppellantRepresentationJob to have followed the pattern established by SyncReviewsJob all along.

Would it makes sense to include the name of the job in the RecordUpdatedByJob table so that multiple different asyncable jobs can run for the same record (say one that queries BGS for appellant information on an hourly basis and another that BGS for address information on a monthly basis)?

[mdbenjam]

Yup I like that. Going to start implementing this.

[pshahVA]

@lowellrex , do you have any context here? It looks like it was about to be implemented, then removed from scope.

[lowellrex]

I think we're good to close this. Doing so now.

[pshahVA]

@lowellrex How did we end up solving this problem anyway?

[lowellrex]

Mark implemented what he described in this comment with this code change and an associated change in the appeals-deployment to make it run every hour.