Caseflow: Case was deleted in VACOLS, but Caseflow tasks still exist

[alisan16]

VACOLS allows deletion of appeals but it's limited to certain individuals like the Branch Chiefs.
Also the ROs can delete an appeal if it's still in the NOD stage.

When a case is deleted in VACOLS, the associated colocated Caseflow tasks are not being deleted and still appearing in users' queue, but the user gets an error message when trying to access it. So far, we have manually been looking at the audit trail, confirming the case was intentionally deleted, then manually cleaning up the Caseflow task. We need a solution that addresses the root cause so this process does not have to be manual.

Discussion here: https://dsva.slack.com/archives/CHX8FMP28/p1574364262362100

[pshahVA]

@D-L-Ware do you know the history behind deleting cases from VACOLS. It seems that simply deleting them causes undesirable effects in Caseflow. I know that you had mentioned that we probably cannot tell the Board that a blanket statement "do not delete from VACOLS" would not go over well. Can we think through some alternate strategies where we turn off that feature in VACOLS, but then enable it in Caseflow, so we can clean up issues seen above during the delete case.

[D-L-Ware]

I know there has been some discussion, see slack referenced above, of preventing Board user from deleting records but there are many reasons records should be deleted. Below are some examples.

•The RO processes a NOD twice (easy, this is what most people think of)
•The RO processes the vet’s appeal under both a C number and a SSN. Happens more with VHA, but still happens in elsewhere. The information is duplicative, but VACOLS can’t cross check and the redundant appeal is likely to be removed and the remainder needing the file number corrected. This is usually found late by counsel.
•The Board’s intake folks activate a remand incorrectly. VACOLS spawns a new record for each post-remand/CAVC remand, when intake activates incorrectly, the action of restoring the first record, the second record needs to go away.
•Lit Support/Co-located erroneously create a spare record. An unnecessary split, CUE, vacate, or de novo record would need to be cleaned up.

What seems to be getting lost here is that these records do need to go away. I know Caseflow will probably never delete erroneously created records instead just close them down but that is not really an option for VACOLS. I see two paths forward.

1. We continue to manually correct this on the backend in Caseflow with a process modification. Given that deletion in VACOLS is limited to very few users we ask them to notify us immediately when they delete so we can start our manual process instead of waiting for a ticket to come in because someone has a Caseflow error. This should be easy as the person deleting the record is likely doing a lot of clean up in VACOLS already dropping us an email would just be one additional step.

2. Caseflow develops some sort of functionality that closes the associate Caseflow record and tasks when a VACOLS record is deleted. Note, if we choose this it is likely a heavy enough lift that it would need to be put in the backlog and we would not be able to work on it until it was prioritized by the Board.

[jimruggiero]

Should this be moved to Define pipeline for further discussion and requirements definition, or moved to Ready for Dev for Echo estimation?

[pkarman]

If we take @D-L-Ware options as exhaustive, then (1) doesn't require any technical changes to Caseflow, only a process change for BVA to implement. If we think (2) is the better option then it should go to Echo.

This ticket, on its own, isn't really actionable, so there's nothing really to estimate. If you move it to ready for dev, I expect that Echo will want to create multiple tickets, including one to represent a tech spec describing an architectural solution for keeping track of deleted items across both systems (devilishly tricky since once things are deleted, they aren't there to keep track of...).

[pshahVA]

On a similar flavor of the same problem, we've seem hearings deleted in VACOLS and then recreated, but they lose the association for this record.

[hschallhorn]

Sentry alerts we've been getting on this:
Here
Here

[hschallhorn]

On a similar flavor of the same problem, we've seem hearings deleted in VACOLS and then recreated, but they lose the association for this record.

Opened #13208 to handle alerts related to this

[hschallhorn]

Deleted vacols records causing more issues: https://dsva.slack.com/archives/CJL810329/p1581433711251600

[pkarman]

@D-L-Ware wrote:

I know Caseflow will probably never delete erroneously created records instead just close them down but that is not really an option for VACOLS.

Is there any reason not to delete a LegacyAppeal record that no longer points to a VACOLS case? If the VACOLS case is removed, the BVA is essentially saying they don't want/need to track the history of that particular case record anymore. Could Caseflow just do the same?

NOTE that I am not referring here to hearings or hearing association records. Those are a separate issue. I am talking specifically about VACOLS::Case records that do not exist anymore (if they ever did).

[D-L-Ware]

No reason from a business perspective. As @pkarman said, the business has made the decision to permanently delete them from the legacy system of record so Caseflow could mimic that. My statement above really just came from an assumption about Caseflow whether to delete or not in Caseflow come down to whatever the team thinks is best and is comfortable with.

[nanotone]

Example of nil case record causing a call to /appeals/VACOLS_ID/hearings to 404: https://sentry.ds.va.gov/department-of-veterans-affairs/caseflow/issues/8559

Slack investigation: https://dsva.slack.com/archives/C4JECDLSE/p1583245052073200

[pkarman]

As of this morning there are zero open LegacyAppeal records with null pointers. Our daily check will report any new ones.

[pkarman]

And today there are dozens more. We have some process creating them.

[pkarman]

Appears to be some aggressive clean up happening in VACOLS. Created PR to run a nightly clean up.

[pkarman]

this appears to be fixed via our nightly sync.