Id: Failure when server returns "Value has more than 255 unicode characters"

[tbertels]

To reproduce:

• Edit a element property (in this case source) and add more than 255 characters (typically multiple addresses).
• Try to save

Result:
iD shows an error dialog: "Errors occurred while trying to save"

Then after clicking OK and trying to save again, we get an OAuth dialog:

Granting access leads to a File not found server error:

This last error may be due to osm itself, but why do we get the OAuth dialog in the first place?

And isn't it possible to show a more helpful message to the user? Including the name of the element if not null and the name of the property would help. This may need some work on the osm API though.

[mmd-osm]

Some of the input fields already have a max length of 255 characters, in case of "Source" (maybe others?), it seems to be missing, so that a user can enter much longer strings.

And isn't it possible to show a more helpful message to the user? Including the name of the element if not null and the name of the property would help. This may need some work on the osm API though.

This can be much better handled by iD by no letting you enter longer strings in the first place. Nothing to do on the OSM API side here.

[quincylvania]

This can be much better handled by iD by no letting you enter longer strings in the first place.

Indeed. We should audit all tag inputs and operations to ensure that they adhere to the character limit.

[quincylvania]

Related: #4061

[mmd-osm]

I left a comment on https://github.com/openstreetmap/iD/commit/17ae12b3d727e395b8e7fb623c4417d8c24aeb29#commitcomment-36988467, hope it doesn't get lost somehow.

Unless there's some other substr implementation in place, the default one wouldn't handle unicode chars as expected. In the worst case this could lead to some very long description being cut off too early.

[bhousel]

Unless there's some other substr implementation in place, the default one wouldn't handle unicode chars as expected. In the worst case this could lead to some very long description being cut off too early.

Thanks @mmd-osm , following our conversation on Slack I'm reopening this so that we can reimplement string counting and truncation functions that match what the API code does.

Our notes:

https://stackoverflow.com/questions/54369513/how-to-count-the-correct-length-of-a-string-with-emojis-in-javascript

led to
https://github.com/orling/grapheme-splitter#examples

But while that contains some great tests, we don't really want to split on graphemes, and using the spread operator to split on codepoints is probably what we really want.:

But Bublé's support for transpiling the spread operator is buggy:
https://github.com/bublejs/buble/issues/215

So I'll just take the code from that issue (which mimics what Babel does) and wrap it up into a function.

[quincylvania]

Okay, I think I finally have this working correctly.

iD now counts tag and role lengths in unicode characters instead of native JavaScript UTF-16 code units. I had to implement the checks in code rather than relying on the maxlength attribute we used previously since browsers count maxlength inconsistently. Doing it in code is safer in any case.

I also added unicode string normalization as a bonus since we're now cleaning all string input. This can reduce the total length of a string without changing the encoded content.

[jleedev]

I'm unable to type spaces when writing a changeset description, and this change looks related.

[quincylvania]

I'm unable to type spaces when writing a changeset description, and this change looks related.

@jleedev Whoops, this change is definitely related! iD now trims whitespace from the ends of textarea fields. Generally this is done only after the field loses focus, but the changeset comment is special.

[tbertels]

I did notice that the last space gets stripped when the changeset description loses focus, but I'm still able to type spaces. Tested with Firefox and Chrome.

[quincylvania]

@tbertels This issue relates to the development preview. I was able to reproduce it in Firefox on macOS.