Openrefine: "Add columns from reconciled column copy" causes HTTP 403 error

[thadguidry]

Describe the bug
When using Add columns from reconciled column copy and then clicking on a Wikidata property in the listing, it shows "Error" in the preview white area. Unsure if this is transient issue or not with Wikidata recon service.

To Reproduce
Steps to reproduce the behavior:

1. Make copy of "Scientific Name with Author" column
2. Reconcile column "copy" with taxon Q16521 and include column "Symbol" as property "USDA PLANTS ID P1772"
3. After reconciling, Click on "copy" column and Add column from reconciled values...
4. Click on "taxon common name".
5. See error

Current Results

07:00:35.183 [                   refine] POST /command/core/preview-extend-data (32661ms)
07:00:35.361 [                  command] Exception caught (178ms)
java.io.IOException: Server returned HTTP response code: 403 for URL: https://tools.wmflabs.org/openrefine-wikidata/en/api
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
        at com.google.refine.model.recon.ReconciledDataExtensionJob.performQuery(ReconciledDataExtensionJob.java:157)
        at com.google.refine.model.recon.ReconciledDataExtensionJob.extend(ReconciledDataExtensionJob.java:103)
        at com.google.refine.commands.recon.PreviewExtendDataCommand.doPost(PreviewExtendDataCommand.java:124)
        at com.google.refine.RefineServlet.service(RefineServlet.java:178)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
        at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81)
        at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:132)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Expected behavior
Expected to have Preview without error.

Screenshots

Desktop (please complete the following information):

• OS: Windows 10 64bit
• Browser Version: Firefox latest
• JRE or JDK Version: 1.8.0_151

OpenRefine (please complete the following information):

• Version: OpenRefine 3.0 Trunk

Datasets

"Symbol","Synonym Symbol","Scientific Name with Author","Common Name","Family"
"ABAB","","Abutilon abutiloides (Jacq.) Garcke ex Hochr.","shrubby Indian mallow","Malvaceae"
"ABAB","ABAM5","Abutilon americanum (L.) Sweet","",""
"ABAB","ABJA","Abutilon jacquinii G. Don","",""
"ABAB","ABLI","Abutilon lignosum (Cav.) G. Don","",""
"ABAB70","","Abietinella abietina (Hedw.) Fleisch.","abietinella moss","Thuidiaceae"
"ABAB70","HYAB","Hypnum abietinum Hedw.","",""
"ABAB70","THAB70","Thuidium abietinum (Hedw.) Schimp.","",""
"ABAL","","Abronia alpina Brandegee","Ramshaw Meadows sand verbena","Nyctaginaceae"
"ABAL3","","Abies alba Mill.","silver fir","Pinaceae"
"ABAM","","Abies amabilis (Douglas ex Loudon) Douglas ex Forbes","Pacific silver fir","Pinaceae"

Additional context
I tried other Wikidata properties to add and they also resulted in a HTTP 403 error in console.

[wetneb]

The Wikidata reconciliation service is quite unstable at the moment. That is generally due to the instability of the underlying APIs used by the wrapper - but if this bug persists over time, it might be something else.

In any case, we should definitely make sure OR reacts correctly to any HTTP error it gets from services.

[ettorerizza]

Looks like the same error as this one in the mailing list.

[glenviewjeff]

https://github.com/OpenRefine/OpenRefine/issues/2034

[ostephens]

As per comment on #2304, looks like the problem is with properties which have a monolingual text datatype - the "taxon common name" property is monolingual text datatype

[ostephens]

Confirmed. Requesting properties where a language can be specified for the value leads to a 403 error

Suspect this is an issue that needs raising against https://github.com/wetneb/openrefine-wikibase but pinging @wetneb to check

[wetneb]

@ostephens many thanks for the debugging, I will get this fixed

[wetneb]

Issue moved to https://github.com/wetneb/openrefine-wikibase/issues/59.