Aspnetcore: Slashes incorrectly encoded when building URL with catchall param?

[aspnet-hello]

_From @Daniel15 on Saturday, September 10, 2016 10:27:27 PM_

Consider this controller:

public class TestController : Controller
{
  [Route("foo/{*path}")]
  public IActionResult Index(string path)
  {
    var url = Url.Action("Index", new { path = "hello/world" });
    return Content("Path = [" + path + "], built URL = " + url);
  }
}

When you hit http://example.com/foo/some/url, ASP.NET MVC Core will display:

Path = [some/url], built URL = /foo/hello%2Fworld

Whereas ASP.NET MVC 5 will display:

Path = [some/url], built URL = /foo/hello/world

Notice that the URL returned by Url.Action encodes / as %2F, whereas the previous MVC version did not do this.

Is this an expected change? If so, how do I render the URL such that the slash is not encoded?

_Copied from original issue: aspnet/Routing#363_

[aspnet-hello]

_From @SonicGD on Tuesday, November 29, 2016 10:55:46 AM_

Have same problem right now. Any news on this? @Daniel15, did you bypass it somehow?

[aspnet-hello]

_From @Daniel15 on Tuesday, November 29, 2016 11:36:18 AM_

I never got time to look into it, these routes on my site are still broken 😞

[aspnet-hello]

_From @SonicGD on Wednesday, November 30, 2016 9:21:40 AM_

So i did some investigation... Character encoding happens in https://github.com/aspnet/Routing/blob/dev/src/Microsoft.AspNetCore.Routing/Internal/UriBuildingContext.cs#L100

and UrlEncoder always filters '/' char - https://github.com/dotnet/corefx/blob/master/src/System.Text.Encodings.Web/src/System/Text/Encodings/Web/UrlEncoder.cs#L125

I don't see any way to configure or disable this behavior.

Maybe @NTaylorMullen or @rynowak will help us?

[aspnet-hello]

_From @SonicGD on Wednesday, November 30, 2016 9:58:29 AM_

Also, even if i generate correct url - i doesn't match route. ex:

[HttpGet("/{parentUrl}/files/{catPath}/{fileName}.html")

where catPath is path like 'video/others'.

Only workaround i found is to use catchall param and then parse it with regex. And it looks ugly :cry:

[aspnet-hello]

_From @DeadSith on Sunday, December 11, 2016 9:02:58 AM_

The only way to bypass it now is to set urls in a tags using href, not built-in tag helpers. It makes it harder to refactor code, but at least you get good-looking urls.

[aspnet-hello]

_From @Daniel15 on Saturday, April 29, 2017 11:01:09 PM_

This is still broken :(

I worked around it by using Url.Action then calling .Replace("%2F", "/") on the resulting string. Ugly but it works: https://github.com/Daniel15/Website/commit/26b1d2ad8c307afda07d70769265ea71ce775d2c

[aspnet-hello]

_From @DamianEdwards on Tuesday, May 2, 2017 10:52:01 AM_

@rynowak @Eilon @danroth27 comments?

[aspnet-hello]

_From @Eilon on Tuesday, May 2, 2017 4:47:21 PM_

We'll see if we can take another look at this. The problem, if I recall, isn't so much with how to encode slashes, but rather what to do when they come back in. I believe that servers/reverse-proxies such as IIS (and no doubt others) will decode the slashes on the way in, so you don't get round-tripping of data. We spent a lot of time back in MVC 1.x-5.x trying to make this work and never could. It's possible that today things are different.

[aspnet-hello]

_From @Daniel15 on Tuesday, May 2, 2017 11:47:40 PM_

I believe that servers/reverse-proxies such as IIS (and no doubt others) will decode the slashes on the way in

Why would the slashes be encoded in the first place? If I have a route like foo/{*path}, hitting /foo/bar/baz is a valid path. I never had issues with that in older ASP.NET MVC versions, other than IIS6 which is not really a problem these days.

[aspnet-hello]

_From @Eilon on Wednesday, May 3, 2017 9:24:07 AM_

I believe that an incoming URL in the form http://localhost/foo/bar is different from http://localhost/foo%2fbar. The first URL has a path with two segments: foo and bar. The second URL has a path with one segment: foo/bar (once it's decoded).

[aspnet-hello]

_From @kaa on Wednesday, May 3, 2017 9:45:29 AM_

Arguably, if it was a "normal" path segment, but this is specifically a "catch all" path segment which by its nature is going to contain slashes in unencoded form. For a normal path segment the current implementation is the expected behavior of course.

Currently there is no way to create an url using the UrlHelper that fits your first form, as far as I understand. This is what we're asking for.

[aspnet-hello]

_From @Daniel15 on Wednesday, May 3, 2017 10:15:13 AM_

Yeah, @Eilon I should have clarified that this is a catchall segment, so it should capture everything including slashes.

The actual routing is working fine, it's just the URL builder that's incorrectly HTML encoding the slashes.

[doggy8088]

@DamianEdwards @Daniel15 I think I met the same issue in the Azure Function Proxy too.. Check this out: https://github.com/Azure/azure-webjobs-sdk-script/issues/2249

[mkArtakMSFT]

@danroth27, any thoughts regarding this?

[mkArtakMSFT]

@javiercn, can you please investigate what options we can have here? Just come up with some proposal about this and let's discuss it with the team.
//cc: @Tratcher

[javiercn]

I spoke about this with @rynowak a while ago and I spoke to @Tratcher too.

There are some security implications with not completely escaping the path as we need to deal with things like "/Path/../../../" but Kestrel and IIS take care of those things on the inbound path.

We can special case the way we encode a catchAll parameter when we build the url by just creating a new instance of PathString (calling the constructor) and then calling ToString() on the PathString to produce the properly encoded value.

The fix will probably happen somewhere around here
https://github.com/aspnet/Routing/blob/dev/src/Microsoft.AspNetCore.Routing/Template/TemplateBinder.cs#L228

[javiercn]

This would be a breaking change, so we need to think about if we are willing to take it.
@mkArtakMSFT I'm sending this back to triage.

[juho-hanhimaki]

I run into this issue in API projects quite often and the workaround is pain in the ass. So my vote to fix this in future.

[doggy8088]

@javiercn May I ask why this issue would be a breaking change?

[javiercn]

@doggy8088 Because the '/' are getting encoded right now into '%2F' and they won't be if we do this change, and that has the potential to break people's apps depending on how their webservers are configured.

[doggy8088]

@javiercn It’s obviously a bug on encoding when using catchall routing. I think there is nobody will relying on this feature and the workaround right now is not affecting if you patched this bug. If user’s backend server is using IIS, no matter what they apply this patch no not, they won’t be affected either. If user use Apache Web server, there is same things. No one will be affected. So I don’t think it’s a breaking change.

[doggy8088]

@javiercn Please have a look at my study here: https://github.com/Azure/azure-functions-host/issues/2249#issuecomment-353762199

[DevelAx]

Can anyone clarify if it is going to be fixed soon? I couldn't grasp it from the discussion. I'm doing migration from MVC 5 to Core and have to use workaround everywhere the catchall is used.

[Eilon]

@DevelAx I'm not sure we can commit to when, if ever, this might be fixed. It's a very tricky problem and I'm not sure there's a perfect solution. We might have to do some think-outside-the-box solution to this to make sure we preserve compatibility, but maybe have some new way of enabling new functionality. Maybe an option to choose how to encode, or some convention to specify parameters that should be encoded one way or another. Not sure.

[doggy8088]

@DevelAx I made a demo repo here: https://github.com/doggy8088/CatchAllRouteProblem

Here is the ASP.NET Core 5 controller sample code:
https://github.com/doggy8088/CatchAllRouteProblem/blob/master/MVC5/Controllers/HomeController.cs

Here is the ASP.NET Core 2.0 controller sample code:
https://github.com/doggy8088/CatchAllRouteProblem/blob/master/Core2MVC/Controllers/HomeController.cs

These two controllers are exactly identical. You can check the execution result below.

The following screenshot is the ASP.NET MVC 5 result:

The following screenshot is the ASP.NET Core 2.0 result:

As you can see, the built URL format is wrong. That's the problem.

[DevelAx]

@Eilon Thank you for the reply! I'll be waiting for it. Hope you won't forget about this issue.

@doggy8088 You're right. It worked perfectly in MVC 5. Now it has to be always kept in mind not to use default helpers while using catchall option in routes.

[doggy8088]

@Eilon The problem only exists in ASP.NET Core. The ASP.NET MVC 5 is working perfectly! Can you explain why this is a tricky problem? I thought ASP.NET Core 2 and ASP.NET MVC 5 are share the same code base. (almost) The routing logic should be exactly the same, doesn't it?

[DevelAx]

@doggy8088 The odd thing is that we have the catchall option for routes. But this option is not supported by route-helpers in Core.

[doggy8088]

@DevelAx It do support, It just has a bug in it. 😞

[Eilon]

@doggy8088 the code bases are almost entirely different between the two products.

[valeriob]

I'm migrating AspNet Mvc applications and got bitten by this issue, what's the suggested work around ?

[DevelAx]

@valeriob for example, if you want to use Url.Link with <a> tag, you could create a helper class, like this:

public static class MyUrlHelper
{
    public static string LinkFixedSlash(this IUrlHelper urlHelper, string routeName, object values)
    {
        return urlHelper.Link(routeName, values).Replace("%2f", "/");
    }
}

Example of usage in a view:

<a href="@Url.LinkFixedSlash(RouteNames.PostsRoute, new { path = post.UrlPath })">
    @Model.PostHeader
</a>

And don't forget to add to the view the namespace where your helper class is scoped.

---

P.S.
This makes the syntax more familiar:

public static class MyUrlHelper
{
    public static string Link(this IUrlHelper urlHelper, string routeName, object values, bool decodeSlashes)
    {
        string link = urlHelper.Link(routeName, values);

        if (decodeSlashes)
            link = link.Replace("%2f", "/");

        return link;
    }
}

Usage:

<a href="@Url.Link(RouteNames.PostsRoute, new { path = post.UrlPath }, true)">
    @Model.PostHeader
</a>

[doggy8088]

Hope this bug can be fixed in ASP.NET Core 2.1. There are tons of potential ASP.NET MVC devs are going to migrate their website to ASP.NET Core. The inconsistence of the routing feature are really annoying. 😢

[javiercn]

Costing here is for the investigation work

[Eilon]

@mkArtakMSFT - should we move to preview2? We came up with a design yesterday but the work won't be done by EOD today...

[rynowak]

The plan we discussed was to move to 2.2.0

[Eilon]

Ah right. @mkArtakMSFT let's move to 2.2.0 and un-assign.

[mkArtakMSFT]

We've decided to add a new feature to support this scenario in 2.2.0
We will add a new syntax {**catchAll} that will parse catchAll segments as PathString instances.
Users will be able to generate urls without encoded slashshes in catch all values by passing a PathString as the value (or if its on the ambient values)

[DevelAx]

The same trouble with RedirectToRoute & RedirectToAction methods.
This
return RedirectToRoute(RouteNames.HomeRoute, new { path = "xxx/yyy" });
generates url with _"xxx%2fyyy"_.

Is there a workaround for it?

[DevelAx]

I couldn't find any easy workaround so I wrote a custom MyRedirectResult class for this purpose:
https://gist.github.com/DevelAx/f873f1beaf86b1f31ee8bb679762f088

[doggy8088]

@DevelAx In your sample, I think you don't have to implement a MyRedirectResult. It can simply use return Redirect(URL); to do the same thing.

[DevelAx]

@doggy8088 return Redirect(URL) converts "xxx/yyy" to "xxx%2fyyy" therefore it doesn't work for catchall paths. Also, it requires for non-ASCII characters to be encoded, in MyRedirectResult it's done automatically.

[doggy8088]

@DevelAx The RedirectResult has many method signatures, for passing "URL as a string" in the first parameter shouldn't URLEncode the url at all.

[DevelAx]

@doggy8088 If you try to pass a non-ASCII string to default RedirectResult, for example, "我的网站", you'll likely get the exception:
System.InvalidOperationException: Invalid non-ASCII or control character in header. But this is not the major problem: you can't redirect to a URL like "xxx/yyy" because default RedirectResult converts all slashes to %2F, so the URL will be broken as "xxx%2fyyy", that's why I created MyRedirectResult.

[doggy8088]

@DevelAx I mean this one below. Passing "xxx/yyy" in the default RedirectResult won't converts all slashes to %2F, the URL won't broken.

[DevelAx]

@doggy8088 guess what... you're right, it's my mistake. So, it can be useful only for the first case in order not to do encoding for each redirect, example:

string url = UrlEncoder.Default.Encode("/我的网站/About").Replace("%2F", "/", StringComparison.OrdinalIgnoreCase);
return Redirect(url);

But this case can be considered a bit offtopic for this thread.

[mkArtakMSFT]

@jamesnk, @rynowak, @kichalla please consider this as part of the Dispatcher work:

We've decided to add a new feature to support this scenario in 2.2.0
We will add a new syntax {**catchAll} that will parse catchAll segments as PathString instances.
Users will be able to generate urls without encoded slashshes in catch all values by passing a PathString as the value (or if its on the ambient values)

[mkArtakMSFT]

Closing this as done!

[russgove]

Hey, so how do i call an azure function that has a url segment with slashes from typescript?

I have my function signature set like this:
public static async Task GetRoleToTransactionsForRoleName(
[HttpTrigger(AuthorizationLevel.Function, "get", Route = "RoleToTransactions/{**RoleName}")]HttpRequestMessage req,
TraceWriter log,
string RoleName)
{

still getting errors

[doominator42]

I can't get this to work. My route is /somepath/{**name} and here is how i'm trying to generate the url:

// name = "x/y"
Url.Action("ActionName", "ControllerName", new { name }) 
// results with "/somepath/x%2Fy"

also tried

// name = "/x/y"
Url.Action("ActionName", "ControllerName", new { name = new PathString(name) }) 
// results with "/somepath/%2Fx%2Fy"

Anyone has managed to make it work and can help me?

[Eilon]

Hi, it looks like you are posting on a closed issue/PR/commit!

We're very likely to lose track of your bug/feedback/question unless you:

1. Open a new issue
2. Explain very clearly what you need help with
3. If you think you have found a bug, include detailed repro steps so that we can investigate the problem.