Yourls: URL Encoding gets decoded before saving to db.

[Arujei]

When shorting this URL:
/Page.jsp?tI=ND6AWbgS5U4ky08ndJoxQg%253D%253D&uI=sA2%252BUi1VNUjpVsdS0GJGtw%253D%253D

it gets saved as:
/Page.jsp?tI=ND6AWbgS5U4ky08ndJoxQg==&uI=sA2+Ui1VNUjpVsdS0GJGtw==

Looks the URL encoding gets decoded.

[LeoColomb]

Linked with #1303

Please specify what is your environment:

• YOURLS version
• PHP version
• ...

[Arujei]

YOURLS 1.7
PHP 5.4.13

[stewx]

I am having the same issue with YOURLS 1.7 and PHP version:
PHP 5.3.10-1ubuntu3.6 with Suhosin-Patch (cli) (built: Mar 11 2013 14:34:31)

It's breaking my links.

[ozh]

Indeed.

I think we have to detect the context in which URL are added to the DB: shortening using API, shortening using bookmarklets, shortening using manual input, or editing in the admin interface.

I'm not sure anyway how to handle the situation in a generic and foolproof way: if for example someone shortens http://example.com/hello world, that long URL has to be converted to http://example.com/hello%20world. Similarly if someone enters http://example.com/hello world%20yay, the end result should be http://example.com/hello%20world%20yay and not double-encoding the second %20.

[stewx]

My context was using the API.

Is it possible to pass a parameter in the API that would prevent URL-decoding?

[ozh]

You could use a custom plugin to prevent all URL decoding, which might or might not suit your specific needs, something like:

<?php
/*
Plugin Name: Don't encode/decode
Plugin URI: http://yourls.org/
Description: Don't encode/decode submitted URLs
Version: 1.0
Author: Ozh
Author URI: http://ozh.org/
*/

yourls_add_filter( 'encodeURI', 'stewx_no_decode' );
function stewx_no_decode( $decoded, $original ) {
    return $original;
}

Again, might or might not meet your needs. Test before with common use cases.

[stewx]

Is there a reason we can't leave URLs exactly as they are passed in?

[ozh]

Yes. The one explained above, see related issues mentioned here.

[stewx]

I see what you mean, but this is still a bug IMO, since a URL-encoded URL that has been sent in is getting linked to as a URL-decoded URL.

[ozh]

Never said there was no bug, and as a matter of fact the issue has not been closed

[stewx]

That plugins work great for my purposes. Thanks!

In my opinion, people should have to encode URLs before sending them to YOURLS. Trying to force decoding/decoding is always going to cause some issues.

[hell0world]

great~thanks!
after applying the simple plugin don't encode/decode, there will be an issue with button edit. json_encode returns false always. seems because there is % in URL. any suggestion to get ride of it? I am not sure what's up!@ozh, anysuggestion?

I am wonder if your guys are all good once disabled encodeURI as above.....

[Arujei]

I'd have to agree with @stewx that urls should be encoded when sending to YOURLS. Any URLS that I am using sharing from other sites are usually straight out of the address bar or a share link that is generated. I hardly see the need to encode URLS. At the very least this should be a configurable option that we can disable at our discretion.

@ozh thanks for the plugin suggestion I'll probably go that route if this continues to be an issue.

[rmerit]

I'm having the same issue with %26 being decoded to an ampersand in Google Maps addresses:

https://maps.google.com/maps?q=5th+Ave+%26+W+42nd+St,+New+York,+NY+10036

The plugin @ozh posted above solves the problem. I think the best way to handle URLs like this is to bookmark them in the browser and then copy the bookmarked content to YOURLS (with no further encoding or decoding being needed), rather than the text in the location bar.

There is a related problem with trying to edit such URLs (with or without the plugin), although I'm not concerned by it.

...

Actually, I've noticed a side-effect of using the plugin is that square brackets are getting stripped out of the URL. My fix (which will probably cause other side effects) is to encode them in the filter function:

$result = strtr( $original, array (
  '[' => '%5B', ']' => '%5D'
) );

return $result;

I know it's wrong to encode square brackets in a URL if they enclose an IPv6 addresses. Otherwise, they're supposed to be enoded, but Firefox (for one) doesn't always do that.

[davidolrik]

Similarly if someone enters http://example.com/hello world%20yay, the end result should be http://example.com/hello%20world%20yay and not double-encoding the second %20.

I disagree here, if I enter a url with the characters %20 in it I expect it to be encoded as %2520 - you can't mix entering stuff to be encoded, and stuff that shouldn't be encoded.

In your example above you _should_ double encode the second %20, as you can't have a url both be unencoded and encoded at the same time.

[SirLouen]

davidolrik, I disagree with you. Why just never encoding and suggest manual encodings by users (or a tool to help encoding, but no pre-suppose encodings when not needed that fool the whole system?
Or maybe a tick button in line ("Encode URL?")
From what I've seen the issue goes beyond to other functionalities (like editing updating information in the admin panel)
https://github.com/YOURLS/YOURLS/issues/1973
For example when I change the Title, the not-coded url fallsback to an encoded url (breaking the functionality that the fix from ozh provided)
To put in place a widely used platform, Bit.ly does not encode pre-encoded urls for example.

[davidolrik]

@SirLouen I agree, that no encoding at all is a better choice - but if you choose to encode, you should encode all of the string and not try to guess whether or not some part of the string is already encoded.

[SirLouen]

Exactly trying to guess the proper string, is the problem because then you have to implement a lot of rules and plugins to adapt to infinite possible combinations.
Now I'm looking how to solve this issue in different parts of the code, because it only works for me during the creation with the plugin mentioned here.

[ozh]

The solution is not "never encode" nor "never decode". It depends on the context in which URL are added to the DB: shortening using API, shortening using bookmarklets, shortening using manual input, or editing in the admin interface.

[davidolrik]

The solution is to always do the same everywhere, and I'm leaning towards never encode.

[Ben-Mack]

@ozh Can you put Don't encode/decode to the plugin list? It's take me hours to found this solution.
( Another vote for no encoding at all, as I can see that's what goo.gl bit.ly do )

Thank you for develop this great url shortener!

[LeoColomb]

@Ben-Mack Thanks for your feedback!
Would you aim to create a PR on awesome-yourls to add the plugin? 💯

[kristianserrano]

I don't know if this is the same issue or not, but when I try to shorten a permalink generated by Trumba, it encodes any % as %25 in the URL.
For example,
http://web.library.emory.edu/news-events/upcoming-classes.html?trumbaEmbed=view%3Devent%26eventid%3D128996992
gets rewritten as this:
http://web.library.emory.edu/news-events/upcoming-classes.html?trumbaEmbed=view%253Devent%2526eventid%253D128996992

Any suggestions on how to work around or fix this?

[e-anima]

The Plugin in from ozh is so great. As i had a lot of parameterized urls that uner no cirumstances are allowed to be changed in any way, thsi is the solution.

[ozh]

Closing: 2691 and associated PR will supersede