Swashbuckle.aspnetcore: $ref values must be RFC3986-compliant percent-encoded URIs

[emiliovmq]

Hi team,

First, thanks for this amazing tool...

Recently, we started the validation process of our APIs (version 2.0) using Swagger-Hub (https://app.swaggerhub.com) & Swagger Editor (https://editor.swagger.io). While with Swagger-Hub everything went well - some minor warnings -, with Swagger Editor we surprisingly started reciving errors (with structures that haven't changed from version 1.0 and successfully passed the validation process) of type: Semantic error at paths... $ref values must be RFC3986-compliant percent-encoded URIs...all the errors are related to simple generic structures used in our responses and the references to them seem to be using not allowed characters...like this '[' and ']:

schema:
$ref: '#/definitions/Page[Voyage]'

-- here is the error (Semantic error at paths./api/v201804/Voyages/Pages.get.responses.200.schema.$ref
$ref values must be RFC3986-compliant percent-encoded URIs)

and, basically, this is the definition

Voyage:
description: Represents a voyage
required:
- id
- vesselId
- startDate
- endDate
type: object
properties:
-- description of the properties

'Page[Voyage]':
description: Represents a page of resources
required:
- data
- paginationContext
type: object
properties:
data:
description: 'Defines the data. Type: array'
type: array
items:
$ref: '#/definitions/Voyage'
paginationContext:
$ref: '#/definitions/PaginationContext'
description: 'Defines the pagination context. Type: PaginationContext'

PaginationContext:
description: Represents the pagination context
required:
- page
- size
- pages
- totalRows
type: object
properties:
-- description of the properties

Could you, please, tell us what we are going wrong?
Thanks

[darwinyo-enterprise]

same trouble here

[domaindrivendev]

I wasn’t aware of this requirement before and therefore thr value that’s generated by default doesn’t account for it. So, let’s keep tracking here as a bug.

In the meantime, you can override the default behavior via the “SchemaId” confit setting (see readme)

[domaindrivendev]

Correction: the setting is CustomSchemaIds not SchemaId

[protegesolutions]

Same issue. And if you fix the compliance by escaping [ and ] with %5B and %5D respectively, you end up with "5B" and "5D" in the properties because the generator only removes the % symbols. It's a bit of a mess.

[gtd4]

Having a similar issue
For some reason our webapp generates the following
$ref: '#/definitions/KeyValuePair[String,IEnumerable[String]]'

and has KeyValuePair ... as part of our model section

In the definitions section it has
KeyValuePair... and a warning next to it saying the definition isn't being used but there are at least 3 refs to it.

I tried the suggestion around the schemaIds but that didn't fix it.

Is there a way to set model names i.e replace KeyValuePair with something else

[protegesolutions]

@gtd4 I reckon you're seeing this because the $ref is "invalid" (note quote marks!).

As a test, try manually modifying the $ref - eg, removing the parameters ([]) - until the definition shows as "used". Then work backwards from there.

Something I didn't know was that even if you have Errors - rather than Warnings - you can still submit your definition for client- or server-side code generation. To me, an error should/would prevent this, ala a compilation error. Anyway.

[gtd4]

@protegesolutions sorry I should have mentioned that I tried to manually edit on SwaggerEditor and match the references and that fixed the problem. Because this definition is created automatically I was hoping their would be a way to edit/swap the keyvaluepair ... via code or some kind of annotation or something.

[protegesolutions]

@gtd4 Ah, gotcha. If you're generating code from the definition, you should be able to just go ahead with the $ref error. We moved from using editor to generator and the $ref/definition stuff seems to be fine. Someone's just gotten a tad overzealous with the rules, I reckon, because to conform to the $ref validation, you end up with broken generated code.

[hanssens]

My property was indeed a KeyValuePair<string, string> as others mentioned before. The outputed result in the swagger.json was:

  phone_numbers:
    uniqueItems: false
    type: array
    items:
      $ref: '#/definitions/KeyValuePair[String,String]'

It took me a while to figure out from this thread what was meant with using the CustomSchemaIds. Experimenting with it, it's just a matter or playing around with the C# type name, incl. generic arguments, and transforming the name into something that's url-compliant. Like, for example:

swagger.CustomSchemaIds((type) => type.ToString()
    .Replace("[", "_")
    .Replace("]", "_")
    .Replace(",", "-")
    .Replace("`", "_")
);

Some notes on why the type.ToString():

• Can't use .FullName because this will not solve the problem but only adds additional rubbish to the $ref
• Can't use .Name, because this will break if you have the same type with different generic arguments (also will include the backtick '`' symbol, by the way)

Not proud of this. And it makes a huge mess of the definition. But it does seem to fix the "error". Maybe it help others, until a better solution can be conjured up. Would love for that to happen! So +1.

[kgrosvenor]

Was there any good solution for this, i'd like to be able to override the default schema id for generic objects but nothing cleanly replaces '[' and ']' ?

[kgrosvenor]

Ideally want to end up with

Paged[Voyage]
PagedVoyage

[kgrosvenor]

My property was indeed a KeyValuePair<string, string> as others mentioned before. The outputed result in the swagger.json was:

  phone_numbers:
    uniqueItems: false
    type: array
    items:
      $ref: '#/definitions/KeyValuePair[String,String]'

It took me a while to figure out from this thread what was meant with using the CustomSchemaIds. Experimenting with it, it's just a matter or playing around with the C# type name, incl. generic arguments, and transforming the name into something that's url-compliant. Like, for example:

swagger.CustomSchemaIds((type) => type.ToString()
    .Replace("[", "_")
    .Replace("]", "_")
    .Replace(",", "-")
    .Replace("`", "_")
);

Some notes on why the type.ToString():

• Can't use .FullName because this will not solve the problem but only adds additional rubbish to the $ref
• Can't use .Name, because this will break if you have the same type with different generic arguments (also will include the backtick '`' symbol, by the way)

Not proud of this. And it makes a huge mess of the definition. But it does seem to fix the "error". Maybe it help others, until a better solution can be conjured up. Would love for that to happen! So +1.

With your example i ended up with this which is not desired by any means:

GroHealth.Web.ViewModels.Utils.PagedList_1_GroHealth.Entities.HealthMetrics.BloodGlucose_

@domaindrivendev How exactly can i create this, it's very similar to what is returned by default minus the square brackets.

[pelakm]

How about using something like this?
swagger.CustomSchemaIds((type) => HttpUtility.UrlEncode(type.FriendlyId(false)));

[gtrifidis]

I did face similar issues as already mentioned. Even if the URL was encoded, the validator still was complaining for not finding a reference to the definition

Semantic error at paths./api/Attribute/Filter.post.responses.200.schema.$ref $refs must reference a valid location in the document

Which meant that I had to also update the definitions to match the new encoded URL.

What I did to get around this, was to convert the "[" for any operation response that had a schema reference to a generic type, into a dot ".", using an IOperationFilter. I also removed the "]"

Using an IDocumentFilter, I updated all Definitions of generic type, using the same pattern as above.

So the generic type List[MyObject] becomes List.MyObject which is still unique, but also compliant.
In a more complex generic type a KeyValuePair<String,String> should probably become KeyValuePair.String.String

As long as the Definition key is unique and compliant, the validator seems happy.

Sharing in case someone finds it helpful.

[domaindrivendev]

As mentioned above, the CustomSchemaIds config method provides the flexibility to play around with different strategies. Additionally, in the next version (previews available on myget.org), the default implementation has been simplified to _hopefully_ provide preferable names for generic types out-of-the-box. For example, List<Product> will be described as a ProductList, KeyValuePair<string, int> will be described as a StringStringKeyValuePair etc.

For reference, here's the code:

private string DefaultSchemaIdSelector(Type modelType)
{
    if (!modelType.IsConstructedGenericType) return modelType.Name;

    var prefix = modelType.GetGenericArguments()
        .Select(genericArg => DefaultSchemaIdSelector(genericArg))
        .Aggregate((previous, current) => previous + current);

    return prefix + modelType.Name.Split('`').First();
}

[ashley-manos-finsuite]

As mentioned above, the CustomSchemaIds config method provides the flexibility to play around with different strategies. Additionally, in the next version (previews available on myget.org), the default implementation has been simplified to _hopefully_ provide preferable names for generic types out-of-the-box. For example, List<Product> will be described as a ProductList, KeyValuePair<string, int> will be described as a StringStringKeyValuePair etc.

For reference, here's the code:

private string DefaultSchemaIdSelector(Type modelType)
{
    if (!modelType.IsConstructedGenericType) return modelType.Name;

    var prefix = modelType.GetGenericArguments()
        .Select(genericArg => DefaultSchemaIdSelector(genericArg))
        .Aggregate((previous, current) => previous + current);

    return prefix + modelType.Name.Split('`').First();
}

Thanks, this worked a treat

[fguemara]

And for those who need the integration with NSwag Studio and get the old naming (TypeOfType etc.),
based en the previous, here is the code :
` c.CustomSchemaIds(DefaultSchemaIdSelector);

            string DefaultSchemaIdSelector(Type modelType)
            {
                if (!modelType.IsConstructedGenericType) return modelType.Name;

                var prefix = modelType.GetGenericArguments()
                    .Select(DefaultSchemaIdSelector)
                    .Aggregate((previous, current) => previous+ current);

                return modelType.Name.Split('`').First() + "Of"  + prefix;
            }`

[ja6ez]

for anyone who still has a problem with this, can try to resolve the way i did. It is basically not accepting any special characters in your definitions which is referred under your schema in the API. So all I did was to change to a meaningful definition name and use it in the schema. The error is gone. Here is the example of the problem and its solution.

Problematic definition vs ref

"schema":
{
"$ref": "#/definitions/Tuple[List[Int32]. List[Int32]]"
}

under defintions: (model sections)
"Tuple[List[Int32]. List[Int32]]" : { ... }

Here is the solution to the above example problem - Rename definition and reference it as defined

"schema":
{
"$ref": "#/definitions/TupleListIntegers"
}

under defintions: (model sections)
"TupleListIntegers" : { ... }
The error goes off.