Caseflow: Certification: Allow fuzzy date matches when verifying documents

[askldjd]

This is an action item from the Honolulu RO interview.

Currently, the Caseflow Certification logic requires the received_at date in VBMS to match with the dates (bfdnod, bfdsoc, bfd19) in VACOLS case record. However, according to Honolulu RO, the received_at date is always different from the actual letter date by 1 business day. This is likely because of the Hawaiian timezone and the way VBMS is batching their documents upload.

After a brief discussion with the team, I think the consensus is to allow fuzzy date matching to account for this error. I know we have discussed this back in November during the Oakland RO visit, but we didn't make any decision because we thought it was a rare scenario.

[amprokop]

Cool. Would this consist of allowing Form9/NOD/SOC/etc dates to match in VACOLS/VBMS if they're only one day apart?

[askldjd]

One _business_ day. According to Honolulu RO, if the letter date is Friday, it could show up as received_at on Monday in VBMS.

Update after clarification from RO
One _business_ day. According to Honolulu RO, if the letter date is Monday, it could show up as received_at on Friday in VBMS.

[amprokop]

Hm, can we drill into this — do we think this is an issue where timezones are causing date mismatches (e.g., a document uploaded at 530am ET on 12/10/2000 would have been uploaded at 1130pm HST on 12/09/2000)— or is this something else?

It sounds like based on your description that this might be a process problem, where the date in VBMS and the date in VACOLS mean different things—one is, the date of the document's creation, the other is the date of the document's being uploaded to VBMS?

Just want to be clear on what the problem is before we go about trying to solve it.

@askldjd @shellicious

[askldjd]

I think it might be a combination of problems based on the RO's description. However, to fully understand the issue, we might need to screen-share with the user.

From what I gathered, letter receive date != letter upload date. Here's my speculation.

After the letter is received, it appears that there is a batching job that uploads them to VBMS. This job appears to be scheduled during business hours in EST. When Honolulu RO receives a letter from the veteran during Honolulu business hours, the letter doesn't get uploaded until the next business day because it is 6 hours behind EST. Since we do our matching using letter upload date, it will always be mismatched for Honolulu RO.

[cmgiven]

We match on the :received_at date from VBMS, not the upload date. So I'm not sure that that's the issue.

But for some additional context, the scripts we use to detect errors in the seam use a fuzzy match of three days. That choice precedes me, so I'm not sure how it was arrived at.

[askldjd]

Do we know how received_at is set in VBMS? The RO said that the date written on the letter itself differs from the date in VBMS (which I assume received_at).

[cmgiven]

@abbyraskin @shellicious ^

[askldjd]

So I pinged Honolulu RO for clarification on this matter.

The receipt_date in VBMS is actually the date the letter was prepared and uploaded to VBMS. However, the date _printed_ on the letter itself is actually the next business day. The date on the letter is what VACOLS stores in its database.

This is why Caseflow will see a date mismatch between VBMS and VACOLS.

[cmgiven]

What is the logic behind the date printed on the letter? And is there any reason why the VBMS date doesn't use that date?

Keeping in mind that we are able to certify most appeals, trying to figure out whether this issue is specific to Honolulu and how they do things.

[askldjd]

From Honolulu RO

it's to afford the veteran a full year for his/her appeal period.   the theory is that the mail man already picked up the correspondence so if i say generate a letter now, and the mailman already picked up the outgoing mail, the Veteran will lose one day b/c the mail is not being picked up until tomorrow.

[cmgiven]

Gotcha. So this is just outbound (SOC, SSOC).

And why can't they use the same date in VBMS?

[cmgiven]

Oh, right, because it's in the future.

[askldjd]

More Update:

The system they use to generate letters is called ADL. ADL defaults the letter to the next business day. The RO is not sure if the date configuration can be changed.

In the older system that generate letters for older cases, they use a system called MAP-D. This system allows the date to be adjusted.

From RO

it could be that the system knows we are 6 hrs behind the East Coast and just defaults us to the next business day because the mail was already picked up on the East Coast. I'm not too familiar with the reasoning for this.

[amprokop]

It sounds like we should count VACOLS dates as matching VBMS dates if the VBMS received_at date is equal to the VACOLS date OR is 1, 2, or 3 days before the VACOLS date. Does that sound right to you, @askldjd @shellicious @cmgiven?

Next, should this fuzzy date matching apply to both documents sent by the VA (SOC, SSOC) and documents sent by the Veteran ( NOD, Form 9)? Sounds like this problem should only affect SOC/SSOCs as I understand it, but we're not 100% sure. Thoughts?

[amprokop]

Also, is this big enough of a policy change to loop in AMO? cc @laurjpeterson

[amprokop]

(ccing you because maybe that's something you would be interested in taking on as you build your bridge to AMO)

[cmgiven]

Yes, we should loop in AMO, specifically Amy Chelgren. Shelly has worked with her on the NY pilot stuff. Let's put together a nice email explaining what we propose to do. Don't expect much pushback.

That will also be a good way of synthesizing the logic behind this approach for the team.

Does someone want to take that on?

[amprokop]

@laurjpeterson, any interest? Shelly or I can do it as well, but this might be a good thing to start to get involvement in.

[laurjpeterson]

@amprokop yep I can take stab at writing this out - thanks for looping me in! Will get this to you guys on Monday so we can send early next week

[laurjpeterson]

@cmgiven @amprokop @shellicious @askldjd I took a stab at an email draft to Amy. Would love your to review it given your closeness to this issue (and because this will be my first interaction with her). It also sounds to me like this only affects SOCs and SSOCs, so I didn't include it in this draft. Let me know if you think Amy is the right person to answer the question on if it affects NODs and Form 9s.

Hi Amy,

My name is Lauren Peterson and I am the new Product Manager on the DSVA team - I look forward to working together! A few weeks ago, we talked with the Honolulu RO about their utilization of Caseflow Certification. They've identified a complexity due to their time zone - that the VBMS and VACOLS dates for SOCs and SSOCs do not match because of the time difference from Hawaii time and East Coast time.

To mitigate this, we propose a fuzzy date feature that counts VACOLS dates as equal to VBMS dates if the VBMS received at date is the same as the VACOLS date, or if it is i 1, 2, or 3 days before the VACOLS date.

We understand that the date in VACOLS is the date printed on the SOC or SSOC letter itself, which adds one business day to make sure the veteran has a full year from the date of the letter given the time difference from where the letter is sent from (East Coast) to where it is delivered (Hawaii). So, our proposal would account for that one business day difference, when it occurs on a weekday or over the weekend.

We think this is a better solution than the current state, in which the receipt_date in VBMS is the date the letter was prepared and uploaded to VBMS.

Please let me know what you think of this change, and if there are any AMO policy implications. We're eager to implement a solution that contributes to increased adoption of Caseflow by the more ROs across the country.

Thanks for your help,
Lauren Peterson

[cmgiven]

• Identify us as the Appeals Modernization team.
• We should be specific about the issue originating with how ADL sets dates.
• We need to match up to four days, in order to account for three-day weekends.
• I would articulate this as "we're considering" not "we propose." The latter suggests we're looking for sign-off. The former suggests we're looking to identify issues if they see them, and will move forward otherwise.
• Note that we've already established that having the VACOLS and VBMS dates differ by a few days does not have an impact on the Board side of things.

[msknee]

Thanks for putting that together, @laurjpeterson! No feedback from me, just +1ing @cmgiven!

While discussing this in storytime just now with @kierachell and @amprokop:

• We're considering expanding the documents covered by the fuzzy date match to NOD, Form 9, SOC and SSOC. This is because we have not heard of date differences of more than a few days causing trouble at BVA for Form 9s, NODs, SOCs, or SSOCs. This is also because we heard from a Regional Office during a Caseflow Hearings Status usability session, that it can take up to 24 hours for a document to be uploaded to VBMS from the mail portal. This leads us to think that there may be delays for documents received by the Board.

Any opposed to the above @cmgiven @nicholasholtz?

[cmgiven]

The difference being that this also covers NODs and Form 9s? I don't have concerns. You probably have the best perspective from discussing with Ivy.

[nicholasholtz]

@shellicious - I don't have concerns based on that. This makes the docs more likely to be present pre-Cert - that is only a positive thing.

[laurjpeterson]

@cmgiven @shellicious thanks for the feedback! we have time to discuss this solution in further detail tomorrow. will send on to amy with our final thoughts and the above edits

[msknee]

Design and development work on this ticket is being discussed here: https://github.com/department-of-veterans-affairs/caseflow/issues/1889

[laurjpeterson]

@cmgiven @askldjd @shanear @amprokop @shellicious

Based on the above, we know for SOCs and SSOCs, the VBMS date could be 0-4 days before the VACOLS date. And Shelly suggested that the Form 9 and NOD VBMS date could be 24 hours after the VACOLS date based on upload.

Can we confirm which we propose to implement?

(1) fuzzy date match for SOCs, SSOCs should be with VBMS date 0-4 days before VACOLS date
AND fuzzy date match for Form 9s and NODs with VBMS date 0-4 days after VACOLS date

OR

(2) fuzzy date match for all docs (SSOCs, SSOCs, Form 9s, and NODs) should be with VBMS date 0-4 days before and after VACOLS date

[cmgiven]

What's the story on why we need to match when VBMS > VACOLS? As far as I know, there's no reason VBMS dates can't be set before the upload date.

[laurjpeterson]

@cmgiven the 'after' VACOLS state was introduced with @shellicious's post above - so the story is mainly for NODs and Form 9s. my option 2 above includes before and after for all forms for simplicity

• We're considering expanding the documents covered by the fuzzy date match to NOD, Form 9, SOC and SSOC. This is because we have not heard of date differences of more than a few days causing trouble at BVA for Form 9s, NODs, SOCs, or SSOCs. This is also because we heard from a Regional Office during a Caseflow Hearings Status usability session, that it can take up to 24 hours for a document to be uploaded to VBMS from the mail portal. This leads us to think that there may be delays for documents received by the Board.

[cmgiven]

If there's nothing preventing them from setting the correct date in VBMS, I'd suggest that we prefer fixing the VBMS date. How long it takes to upload is immaterial, the issue is that the document has the wrong received at date.

[laurjpeterson]

@shellicious what do you think about not implementing the 'after' VACOLS date solution for Form 9s and NODs

So we propose to implement only:
(1) fuzzy date match for SOCs, SSOCs should be with VBMS date 0-4 days before VACOLS date

cc: @amprokop

[amprokop]

@shellicious felt we should expand it to being before and after. I'll ping her to add her justification here, then we can discuss.

[msknee]

@cmgiven @amprokop @laurjpeterson It's currently not possible to move the VBMS date forward in time due to VBMS limitations. But I don't think that moves the needle here...thinking out loud...this affects situations in which the VBMS date is several days before the VACOLS date.

There is the situation in which the VBMS document takes 24 hours to "show up in VBMS". We don't know if this means the received date in VBMS is actually 24 hours after the VACOLS date. Is there a way we can find out if this results in VBMS dates being 1 day after VACOLS dates, or to assess how often VBMS dates are 1-2 days after VACOLS dates? Right now we're assuming lag in uploading doesn't affect the received date.

I don't think it hurts to include a 1-2 day after VACOLS date solution and a 0-4 days before solution for Form 9s and NODs. What is the downside of dates for Form 9s and NODs being several days off from each other?

[nicholasholtz]

This might be understood, but the Date of Receipt on VBMS docs can move forward - just not beyond the date of upload. Generally speaking, the Date of Receipt should almost never be the date of scan/upload, so there is usually some leeway to move them forward.

[cmgiven]

Even if it does affect the received date, this is our opportunity to correct that date, yes? The goal I've seen to implement fuzzy date matching is because VBMS prevents entering the correct date, due to the fact that it can't be after the upload date. I don't think the goal is to allow ROs to avoid correcting VBMS when it can be corrected.

[laurjpeterson]

Since we should get this communication to Amy sooner rather than later, let's table the discussion of fixing dates for Form 9 and NODs until we have some more usability research on that issue (for both VBMS dates before and after VACOLS dates), and can gain wider buy-in across this group.

In my email, I'll highlight the proposal (that we're "considering") for SOCs and SSOCs: fuzzy date match for SOCs and SSOCs should be with VBMS date 0-4 days before VACOLS date

In re-reading this thread, I don't see a firm instance where the Form 9 and NOD VBMS dates will be before the VACOLS date, just a possibility that their VBMS dates could be after the VACOLS dates, so I've removed that for now as well.

I will mention that to Amy that we're considering extending the fuzzy date match to Form 9s and NODs pending further user research.

How does this sound? Will email Amy by end of day today.

cc: @cmgiven @shanear @nicholasholtz @amprokop @shellicious

[laurjpeterson]

We're a go!

Response from Amy:

On first read (and a quick one at that), I personally love the “fuzzy” date match idea. The most common complaint I’ve heard from field employees is the exact date match requirement for the Caseflow certification that then forces the employee to either change the data in VACOLS or do the more cumbersome process of changing the document properties in VBMS. Having the “fuzzy” date feature would help immensely!

I’m copying the AMO Program box as well as my AMO Operations colleagues for situational awareness. Both AMO Program and AMO Operations appreciate the opportunity to review the idea and consider any implications for policies, procedures, and daily operations.

Relevant excerpt from my email for transparency:

A few weeks ago, we talked with the Honolulu RO about their utilization of Caseflow Certification. They've identified a complexity that the VBMS and VACOLS document dates do not match due to the time difference from Hawaii time and East Coast time. This issue stems from the way ADL sets dates - the printed date on SOCs and SSOCs adds one business day to account for the time difference. VACOLS stores this printed date, while the reciept_date in VBMS is the date the letter was prepared and uploaded to VBMS.

To mitigate this, we're considering a fuzzy date feature that counts VACOLS dates for SOCs and SSOCs as equal to VBMS dates if the VBMS receipt_date is the same as the VACOLS date, or if it is 1, 2, 3 or 4 days before the VACOLS date. This accounts for that one business day difference, when it occurs on a weekday, over the weekend, and over a holiday weekend.

We've already established that having the VACOLS and VBMS dates differ by a few days does not have an impact on the Board side of things. We're also considering expanding a fuzzy date feature to NODs and Form 9s pending further user research - will keep you posted here.

[amprokop]

Awesome, thanks Lauren!

On Wed, May 10, 2017 at 4:59 PM, Lauren Peterson [e-mail]
wrote:

We're a go!

Response from Amy:

On first read (and a quick one at that), I personally love the “fuzzy”
date match idea. The most common complaint I’ve heard from field employees
is the exact date match requirement for the Caseflow certification that
then forces the employee to either change the data in VACOLS or do the more
cumbersome process of changing the document properties in VBMS. Having the
“fuzzy” date feature would help immensely!

I’m copying the AMO Program box as well as my AMO Operations colleagues
for situational awareness. Both AMO Program and AMO Operations appreciate
the opportunity to review the idea and consider any implications for
policies, procedures, and daily operations.

Relevant excerpt from my email for transparency:

A few weeks ago, we talked with the Honolulu RO about their utilization of
Caseflow Certification. They've identified a complexity that the VBMS and
VACOLS document dates do not match due to the time difference from Hawaii
time and East Coast time. This issue stems from the way ADL sets dates -
the printed date on SOCs and SSOCs adds one business day to account for the
time difference. VACOLS stores this printed date, while the reciept_date in
VBMS is the date the letter was prepared and uploaded to VBMS.

To mitigate this, we're considering a fuzzy date feature that counts
VACOLS dates for SOCs and SSOCs as equal to VBMS dates if the VBMS
receipt_date is the same as the VACOLS date, or if it is 1, 2, 3 or 4 days
before the VACOLS date. This accounts for that one business day difference,
when it occurs on a weekday, over the weekend, and over a holiday weekend.

We've already established that having the VACOLS and VBMS dates differ by
a few days does not have an impact on the Board side of things. We're also
considering expanding a fuzzy date feature to NODs and Form 9s pending
further user research - will keep you posted here.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/department-of-veterans-affairs/caseflow/issues/1732#issuecomment-300610794,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADm0y9Hb6F40qsh-LcV0YxmwHxB4lCvxks5r4iUzgaJpZM4NH8fg
.

[amprokop]

Already validated. Closing.