Openrefine: Editing records and advancing forward page by page- how the list refreshes and displays the records

[tfmorris]

_Original author: [email protected] (May 05, 2012 17:33:44)_

An annoying behavior when editing lots of records, page by ascending page;

1. Load the attached .tsv file into refine.
2. For this demonstration, make sure your page view is 10 records per page(default view).
3. facet > Facet > Text facet.
4. On the Facet, click 'include' for the 'facetCriteria(30)' item.
5. Click 'next' to go to second page.
6. At this point, record # 11 should show 'start of new page' record, and record # 20 should show 'end of page' record, as is seen in the 'notes' column.
7. in the column names 'facet', remove the 'facetCriteria' text from records 12, 14, 16 and 18.
8. Click the 'next' link to go to page 3 which should display records 21 to 30.
9. You will now notice on this page, there is no record that says 'start of new page', and the reason for this is because it is now on the previous page, a result of those 4 records(12, 14, 16, and 18) being edited, and thus no longer being bound to the respective Facet. And herein is the problem.

When editing hundreds of pages of data, this is a real pain, because instead of the user being able to simply click 'next' and have all un-revised/un-edited records be displayed, the user instead must first click the 'back' link to commit the changes, and then click the 'next' link so that those 4 records that took the place of the above modified 4 records, can be reviewed and edited if need-be prior to moving on to the new page. If any of those 4 records are edited, then the problem occurs again, and is further compounded, which is to say, the user has to click the 'back' link again for a measly <4 records.

The correct behavior should allow the user to click the 'next' link without having to worry about un-revised records ending up on the previous page. This should be accomplished by taking the last edited record on the page(e.g.- # 18), and using the next record up(e.g.- # 19) as the record that will display FIRST on the refreshed 'next' page, accounting for any/all records that dropped out of the Facet as a result of the edits.

So, in the above example, when the 4 records(12, 14, 16, and 18) are edited wherein their edits result in their exclusion from/within the Facet, the list, upon clicking the 'next' link, subsequently consolidates downward, wherein # 13 goes down to the # 12 row, # 15 goes down to the # 14 row, # 17 goes down to the # 16 row, and # 19 goes down to the # 18 row. This means that records # 21, 22, 23, and 24 that are on the next/un-seen and un-revised page, end up consolidating down accordingly, ending up on the second page(records 21-30) when the user has clicked through to the 3rd page, hence, the user misses those records as a result. Ideally, Refine should not allow any records that are on the subsequent page to end up on the previous page... the adjustment should be made based on the next page, not based on consolidating downward imo. Or, at minimum, the user should be able to toggle the option, or have a 'next-fixed' link to choose from.

Thanks,

Eric Jarvies

_Original issue: http://code.google.com/p/google-refine/issues/detail?id=570_

[ChristopherSouth]

I'm seeing a similar refresh issue when doing reconciliation -- if i choose the "double checkmark" icon - or match to all identical cells, it refreshes the entire page and goes back to the top instead of where you last were. I suspect that could be intended behavior to ensure one doesn't have items to revalidate -- but it's a pain when you're on the bottom third of a large list and it keeps taking you back to the top....

[ettorerizza]

The "refresh-and-goes-back-to-the-first-page-after-a-match-all-identical-cells" issue is a pain in the ass. It should be considered as a high priority issue for reconciliation.

[thadguidry]

@wetneb Can you perhaps provide a quick fix for this issue over the next 2 weeks ?

[wetneb]

@ettorerizza after thinking about @thadguidry's suggestions for reconciliation UI I would actually be in favour of a completely different screen setup for reconciliation.
The task of going through a list of potential matches and choosing the right is extremely different from what OR is designed for (apply operations on many rows and drill down with facets).

During a manual reconciliation session:

• the facets do not change and we do not look at them -> they should not be visible
• we look at one row/record at a time -> the other rows should not be visible
• we want to see as much information as possible about the potential matches -> their previews should be visible without any click

So, I would be interested in adding a button "Start reconciliation session", or something like that, which would take you to a completely different screen, tailored for this task. You would do your matches there, and when you get bored you click "Back" and you are back to the normal OR screen.

[wetneb]

@thadguidry sure I will look into that

[thadguidry]

@wetneb Thanks, this is Google Sponsored actually via the Wikidata Reconciliation subproject we have for Phase 1, still ongoing (still money left), so track your time.

And yes, a different screen is ideal if not to much trouble. That's what the Freebase Judgement screen did hosted by Freebase Apps back in the day. Its too bad we didn't have the foresight to save that App before it got destroyed by Google, such as what Spencer did with his apps

[ettorerizza]

@wetneb Yeah, you put the right words on something that I felt without formulating it ! When it comes to examine records one by one manually, we move from a logic of columns (a logic of database) to a logic of rows (a logic of spreadsheet) for which the interface is not appropriate.

[ostephens]

Just to check - although you aren't changing the facets while you reconcile, you may want to only deal with a filtered set of rows as you go into the reconcile process - would what you are suggesting only deal with the filtered rows (I guess yes, but just to be clear)

[ettorerizza]

we look at one row/record at the time -> the other rows should not be visible

Be careful that there are some cases where you have to look at another column to choose the best candidate. For example, to check if a matched place is in the right country, or if it is the right kind of place (some place names refer to both a municipality and a province).

Another case where it is necessary to look at another column is when the matching is done on a copy of the original column (in order to then visually check the quality of the matching.).

[wetneb]

@ettorerizza sure, looking at other columns is super important - I only proposed to hide the other rows.

[ettorerizza]

Ouch, sorry, I had misread (with the copy-paste of the sentence in front of the eyes, moreover). :/

[wetneb]

A lot of different issues are mentioned here:

• #33;
• Having a different UI for reviewing reconciliation results;
• Paging through records giving unexpected results, which is the original issue.

[VojtechDostal]

I wanted to report this bug to find out that it has already been reported a long time ago. This is really a pain for manual reconciliation because you always get returned to the beginning after each and every reconciled line. (the bug also seems identical to #33 )

[wetneb]

@VojtechDostal have you tried the workaround with the facet mentioned in #33 ?

[VojtechDostal]

@VojtechDostal have you tried the workaround with the facet mentioned in #33 ?

Yep, I do that. But sometimes I don't want to match all lines, just some of them, so I inevitably reach line 50 sooner or later. To prevent this I would have to manually mark all skipped lines as "do not match".