Caseflow: Add stats page stats for certification v2
As stewards of Certification, we'd like to create a temporary V2 stats page so we can easily track our new changes.
Initial stats for the stats page:
- V2 certifications started
- V2 certifications completed
- V2 certifications completed in the same period
- V2 time to certify (will most likely go up)
- V2 median time to certify
This will be a temporary page that we'll remove once V2 fully rolls out. Make a TODO issue to remove it as well.
amprokop
All 11 comments
I like the idea of separating V1 and V2 stats while in transition. I think that goes away once we've fully converted, though.
What do you hope to learn from the number with fuzzy date matches?
cmgiven
on 5 Jun 2017
Yep, definitely goes away once we're fully converted.
amprokop
on 5 Jun 2017
With the fuzzy date match # — I'd like to get a general sense of how big of a problem it was for ROs.
amprokop
on 5 Jun 2017
I like seeing fuzzy date match # as well - can we separate these out by RO to see if it affects all time zones?
laurjpeterson
on 6 Jun 2017
I think we're reaching the point where we should address this through a separate report, like the utilization report. There's value in keeping these dashboards simple.
cmgiven
on 6 Jun 2017
Hmm. Is the utilization report updated in real-time? How often are these things generated? And who are the most common users of the stats page? I've only looked at it 3 or 4 times in 6 or 7 months myself, so I don't really know.
amprokop
on 6 Jun 2017
Currently it's just internal to the team. Goal is to be able to quickly diagnose yes, the tool continues to be used, and there aren't any weird issues that have suddenly sprung up (i.e. we're suddenly missing a lot more documents). The reports are periodic, updated as needed, and intended to dive deeper in a specific area of inquiry. The utilization report is updated monthly (or as I get pinged to please refresh it).
cmgiven
on 6 Jun 2017
The Given comments above seem like an argument for adding these live stats. @shellicious and @laurjpeterson especially might want to be able to monitor certification v2 usage when we start rolling out to whole groups of users that we aren't personally talking to. It's not a large engineering task to do so — we basically just have to copy over the metrics that already exist in certification_stats.rb and add an additional condition to the queries there.
However, keeping the stats page simple is a good goal too — so let's just create a separate, temporary V2 stats page instead of grafting on to the old one.
amprokop
on 12 Jun 2017
The difference between v1 and v2 fields once certification is loaded comes from fetch_power_of_attorney! method which includes bgs and vacols rep name, type and address fields
However, there are cases when bgs record may be empty. This would bring v2 in the same position as v1 and no way to tell the difference.
If we exclude certification started stat (including the very first step), there are some other fields (i.e. poaMatches that gets set at the second step) that v2 has but v1 does not.
So, it's either going without certifications started stat or introducing a new field that will distinction v1 and v2
@amprokop
nemodjuric
on 16 Jun 2017
nemodjuric
on 21 Jun 2017
PASSED
ghost
on 23 Jun 2017
Related issues
nikitarockz
·
3Comments
lowellrex
·
5Comments
lomky
·
3Comments
lomky
·
5Comments
lomky
·
3Comments
Most helpful comment
The Given comments above seem like an argument for adding these live stats. @shellicious and @laurjpeterson especially might want to be able to monitor certification v2 usage when we start rolling out to whole groups of users that we aren't personally talking to. It's not a large engineering task to do so — we basically just have to copy over the metrics that already exist in
certification_stats.rband add an additional condition to the queries there.However, keeping the stats page simple is a good goal too — so let's just create a separate, temporary V2 stats page instead of grafting on to the old one.