Nighttab: Import data files doesn't work with special characters

[siomekziomek]

Hi there,
Let me congratulate you for making this add-on. I fell in love in it with the first sight.

Unfortunately "Import data" option doesn't work properly on my Firefox 70.0.1. It works fine when you export small file with a few tiles. But when I exported my set of 32 tiles, every time I tried to load it, import option says:

"Not a JSON file. Make sure the selected file came from nightTab."

Even though it definitely is. I didn't change the files name as well. I checked this issue number of times. Even if I export new file, then change accent, and then try to import the file I exported moment ago, the result is the same: "Not a JSON file...".

Keep up the good work :)

[zombieFox]

Thanks! I'll investigate this. Can you describe your setup please? Do you have any background images applied?

Would it be possible to share the JSON file you have? This will greatly help investigate this issue.

[zombieFox]

Found the problem, there seems to be a special character in one of your bookmark "name" fields. The bookmark "Ustaw Lig" has an invisible special character at the end. Which is causing the isJsonString check to return false. This function uses a native Javascript function:JSON.parse(). Which is throwing an error because, I guess, that special invisible character isn't allowed in JSON format.

This could have been caused by copying and pasting some text when creating the bookmark. I would suggest removing that bookmark and adding it again by manually typing in the name.

I'll see if I can't sanitise the bookmark input values on the creation step in an update.

[siomekziomek]

This special character is "ę" from polish alphabet. I didn't paste it, it's simply alt+e for me ;) It was typed manually.

Yeah, apparently exported files have problem with them. "ę" and "ą" give the same error, while others like "ł" are changed into different characters after importing (WeszBo).

I will restrain from using them for the time being. Thanks for your help and can't wait for 4.0 :)

Cheers

[zombieFox]

That is good to know! Seems I need to investigate how JSON.parse() deals with intentional special characters in strings. 😺

[maxnatt]

The issue can be reproduced with letters from Russian alphabet in bookmark names' as well.

[SteadyArcticFox]

Chinese characters also have this issue.Only English is well ?

[zombieFox]

Further investigation, still no joy finding a way to prevent some characters like "ę" causing JSON.parse to return false.

A possible solution found here:
https://stackoverflow.com/questions/20856197/remove-non-ascii-character-in-string
Is to use this to remove special characters from the JSON Blob:
str.replace(/[^\x00-\x7F]/g, "")

This would allow the JSON to be reimported but without offending characters.

[zombieFox]

Going with this solution:
https://github.com/zombieFox/nightTab/commit/dc59045cc3d3a75ff5dbbf6e5ce1f309ed83f8e5

[zombieFox]

A fix is now live for this issue on version v4.44.0. I've been testing with characters listed in this issue as well as others I could find on this table. It's working for me.

It would be good to get feedback on this from @siomekziomek and others so I can close this issue. 👍

[siomekziomek]

Yup, both export and import works with polish characters flawlessly. Great job 👍