Openrefine: Silent errors when using JSON importer

Created on 12 Mar 2020  路  3Comments  路  Source: OpenRefine/OpenRefine

Describe the bug
Errors in parsing JSON files with Json Importer are not shown to user, but instead are silent and shown only in console.

To Reproduce
Steps to reproduce the behavior:

  1. Use Clipboard with the following:
{
"stats":[
"goal":"1",
"time":".4"
],
"stats":[
"goal":"1",
"time":".3"
],
"stats":[
"goal":"1",
"time":".2"
]
}
  1. Click Next
  2. Choose JSON importer
  3. The preview of some elements are shown (although not completely because of a JSON structure problem in the provided test Clipboard, but the user doesn't know this quite yet)
  4. Click checkbox "Load at most"
  5. See error in console, and JSON elements disappear from preview (Silent Error)

Current Results

09:10:45.530 [       XmlImportUtilities] Exception from XML parse (124ms)
com.google.refine.importers.tree.TreeReaderException: com.fasterxml.jackson.core.JsonParseException: Unexpected character (':' (code 58)): was expecting comma to separate Array entries
 at [Source: (com.google.refine.importers.ImporterUtilities$2); line: 3, column: 8]
        at com.google.refine.importers.JsonImporter$JSONTreeReader.next(JsonImporter.java:361)
        at com.google.refine.importers.tree.XmlImportUtilities.processSubRecord(XmlImportUtilities.java:493)
        at com.google.refine.importers.tree.XmlImportUtilities.processSubRecord(XmlImportUtilities.java:495)
        at com.google.refine.importers.tree.XmlImportUtilities.processRecord(XmlImportUtilities.java:361)
        at com.google.refine.importers.tree.XmlImportUtilities.findRecord(XmlImportUtilities.java:323)
        at com.google.refine.importers.tree.XmlImportUtilities.importTreeData(XmlImportUtilities.java:263)
        at com.google.refine.importers.tree.TreeImportingParserBase.parseOneFile(TreeImportingParserBase.java:223)
        at com.google.refine.importers.JsonImporter.parseOneFile(JsonImporter.java:206)
        at com.google.refine.importers.tree.TreeImportingParserBase.parseOneFile(TreeImportingParserBase.java:120)
        at com.google.refine.importers.tree.TreeImportingParserBase.parse(TreeImportingParserBase.java:86)
        at com.google.refine.importing.ImportingUtilities.previewParse(ImportingUtilities.java:961)
        at com.google.refine.importing.DefaultImportingController.doUpdateFormatAndOptions(DefaultImportingController.java:174)
        at com.google.refine.importing.DefaultImportingController.doPost(DefaultImportingController.java:93)
        at com.google.refine.commands.importing.ImportingControllerCommand.doPost(ImportingControllerCommand.java:68)
        at com.google.refine.RefineServlet.service(RefineServlet.java:189)

Expected behavior
We should show a popup dialog to the user with any Parse exception message from Jackson.
As well as another message line in the same dialog that says "Please correct the format of your input file, or choose another importer type such as Line-based".
During OpenRefine Preview is the right time to let the user know if they have malformed structure that the chosen importer does not understand, and we should never be silent about any errors that a Parser detects but instead always inform the user so they can make sensible choices to fix their input data or choose another importer type that might help them analyze further.

Desktop (please complete the following information):

  • OS: Windows 10
  • Browser Version: Firefox
  • JRE or JDK Version: JDK8

OpenRefine (please complete the following information):

  • Version: 3.3

Additional context
This is also related to #1164 where a TAB char causes a silent exception on import.

JSON bug error handling

Most helpful comment

We are pretty bad at using status codes in OpenRefine at the moment: there are plenty of errors for which we return status 200. I'd say use your judgment as to what these status codes should be :) (It's great that you are considering them)

All 3 comments

Related issue #1715

Hi, I am now currently working to address this issue.
Should I worry about the response's status code when there is a JsonParseException (Say 400 bad request)?

We are pretty bad at using status codes in OpenRefine at the moment: there are plenty of errors for which we return status 200. I'd say use your judgment as to what these status codes should be :) (It's great that you are considering them)

Was this page helpful?
0 / 5 - 0 ratings

Related issues

kushthedude picture kushthedude  路  3Comments

thadguidry picture thadguidry  路  4Comments

wetneb picture wetneb  路  3Comments

wetneb picture wetneb  路  3Comments

stellasia picture stellasia  路  4Comments