Autorest: KeyNotFoundException: The given key was not present in the dictionary.

[yahorsi]

Hi Guys,

Today we've got the following exception:

AutoRest code generation utility [version: 2.0.4413; node: v12.14.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
   Loading AutoRest core      'C:\Users\yahor_si\.autorest\@[email protected]\node_modules\@microsoft.azure\autorest-core\dist' (2.0.4413)
   Loading AutoRest extension '@microsoft.azure/autorest.typescript' (~4.2.0->4.2.4)
   Loading AutoRest extension '@microsoft.azure/autorest.modeler' (2.3.51->2.3.51)
FATAL: System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.ThrowHelper.ThrowKeyNotFoundException()
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at AutoRest.Modeler.SwaggerModeler.Build(ServiceDefinition serviceDefinition) in c:\publish\autorest.modeler\src\SwaggerModeler.cs:line 142
   at AutoRest.Modeler.Program.<ProcessInternal>d__2.MoveNext() in c:\publish\autorest.modeler\src\Program.cs:line 60
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NewPlugin.<Process>d__15.MoveNext()
FATAL: typescript/imodeler1 - FAILED
FATAL: Error: Plugin imodeler1 reported failure.
Process() cancelled due to exception : Plugin imodeler1 reported failure.
  Error: Plugin imodeler1 reported failure.

The command line we use looks like this

$SwaggerSpec  = "src/app/Api/TeamGateApiSwaggerSpec.json"
$OutputFolder = "."
$SecureSourceFolder = "src/app/Api/Secure"
$UnsecureSourceFolder = "src/app/Api/Unsecure"

# Generate Client with Auth support
autorest --add-credentials --source-code-folder-path=${SecureSourceFolder} --override-client-name=TeamGateApiSecured   --input-file=${SwaggerSpec} --typescript --output-folder=${OutputFolder}

And swagger spec is attached in the zip
TeamGateApiSwaggerSpec.zip

Any ideas on how we could fix it are really welcome!
Thanks!

PS: And of course thank you for the cool tool!
PPS: Sorry if that is a wrong repo for this issue

[yahorsi]

The problem it seems to be caused by the fact that in Swashbuckle 5 enums are no more generated inline by default. So, in order to fix we could opt-in for old behavior

[older]

I'm also affected by this bug. Here is
very simple json definition demonstrating the problem. Swagger editor generates C# client from this definition without any errors.

[fearthecowboy]

This is terrible, terrible swagger.

  "definitions": {
    "ResponseCode": {
      "enum": [
        "Success",
        "Error"
      ],
      "type": "string"
    },
    "ResponseObject": {
      "type": "object",
      "properties": {
        "ResponseCode": {
          // this is using allOf where a simple $ref would be far better.
          // this causes a lot of headache, not to mention that autorest 
          // doesn't generate code that has inheritance with enums. 
          "allOf": [ 
            {
              "$ref": "#/definitions/ResponseCode"
            }
          ]

          // instead, just refer to the enum definition directly. 
          "$ref": "#/definitions/ResponseCode"
        }
      }
    }
  }

[older]

This might be terrible swagger, but it seem to be valid one?

This is similar to what is generated by Swashbuckle 5, so to change it to something more autorest-friendly I would have to manually override schema generation for all classes with enum properties.

[fearthecowboy]

" but it seem to be valid one"

It's "valid" in that it's _resolvable_ JSON Schema, but it's not something that I'd encourage.

I'm certainly not going write code to go and unroll those. The best I can do is give you a configuration hack to correct the swagger:

Make a file called config.yaml:

---
# your input file goes here
input-file: myswagger.json

# directives can modify the swagger on the fly
directive:
  - from: swagger-document
    where: $.definitions..properties[?(@.allOf.length === 1 )]
    reason: unroll properties that have allOf declarations that are alone
    # check to see if the property found at $ 
    #  - which we know to be an allOf with a single item
    #  - has only one property
    transform: 'return (Object.keys($).length === 1 ) ? $.allOf[0] : $'

then run:

autorest config.yaml <...>

[yahorsi]

Sorry, maybe this discussion is going in the wrong direction.
I suggest to get the following input:

• Swashbuckle is the de facto standard swagger/open API implementation for the dot net
• Swashbuckle, in the latest stable release 5.0 generates separate types for enums
• Autorest fails in that scenario

In other words, autorest, for the typescript language, will fail if the project uses at least a single enum.

PS: The default Swashbuckle behavior is opt-in and easy to change to the previous

[fearthecowboy]

Separate types for enums is fine. We do that all the time.

Using allOf to refer to one is downright strange (especially since a direct $ref is far simpler, and so much cleaner to read.)

Writing the property as :

"ResponseCode": {
  "allOf": [ 
      {
        "$ref": "#/definitions/ResponseCode"
      }
  ]
}

should just say

"ResponseCode": {
  "$ref": "#/definitions/ResponseCode"
}

Sorry, if Swashbuckle creates that, but I'm not going to go thru and unravel that.

[older]

@fearthecowboy But would you agree that it is a bug in autorest? Anyway, I think that unhandled KeyNotFoundException is not the best way to respond to formally valid input. Maybe NotImplementedException with some meaningful message would be better?

The default Swashbuckle behavior is opt-in and easy to change to the previous

@yahorsi Could you provide a link so it is easier to find how to do it?

[fearthecowboy]

I'll agree that it's a lousy message, but the error that's getting thrown is happening in a component that is not being updated anymore.

The V2 modeler component, along with the V2-era of generators are not being updated because we're writing a new ones to support the Azure Core style runtimes.

In order to fix that, I'd have to go make the fix, publish the modeler, update all the generators to take a dependency on the new modeler, and publish all them.

At this point, it's not going to happen, simply because:

• it can be worked around with a configuration change that I gave you above
• I don't have the time to go rebuild and publish all those, when we're working to replace them anyway.

[yahorsi]

@older There is UseInlineDefinitionsForEnums call, can't find docs but it was added just recently.
@fearthecowboy Good to hear you're guys working on new version. Do you have any plans already on when to ship it? Is it OSS?

[fearthecowboy]

It's all OSS; the language generators are being written in a variety of languages (python in python, ts in ts, java in java, go in typescript, c# in c#)..

they should be showing up in the next couple of months.

[older]

Using allOf to refer to one is downright strange (especially since a direct $ref is far simpler, and so much cleaner to read.)

@fearthecowboy Here's swashbuckle author explains rationale behind this (I resolved my own issue using advice from this thread, just wanted to add this as a reference).

[spareilleux]

If you use NSwag, you can do something like:

_SimplifyAllOfPropertiesProcessor.cs_

using System.Collections.Immutable;
using System.Linq;

using NJsonSchema.Generation;

namespace YourNamespace.Swagger
{
    public class SimplifyAllOfPropertiesProcessor : ISchemaProcessor
    {
        public void Process(SchemaProcessorContext context)
        {
            var schema = context.Schema;
            var allOfProperties = 
                schema.Properties
                      .Where(x => x.Value.AllOf != null && x.Value.AllOf.Count == 1)
                      .ToImmutableArray();

            foreach (var allOfProperty in allOfProperties)
            {
                allOfProperty.Value.Reference = allOfProperty.Value.AllOf.First().Reference;
                allOfProperty.Value.AllOf.Clear();
            }
        }
    }
}

_Startup.cs_

public void ConfigureServices(IServiceCollection services)
{
   services.AddSwaggerDocument(config => config.SchemaProcessors.Add(new SimplifyAllOfPropertiesProcessor());

   // ...
}

[older]

The allOf behavior has been made opt-in in Swashbuckle v5.1.0. Didn't have time to test if myself, just adding link for reference.

[lmazuel]

Close, we believe it's fixed, please re-open an issue if you still have a problem.

[BlackRider97]

AutoRest C# code generation

autorest --csharp --input-file=swagger.json --output-folder=Autogenerated --clear-output-folder --verbose --namespace=MyNameSpace.Autogenerated --use:@microsoft.azure/[email protected] --version=2.0.4413

1. This is working and no error

          {
            "in": "body",
            "name": "boundary",
            "description": "Boundary object.",
            "schema": {
                 "$ref": "#/definitions/Boundary"
            }
          }

1. This is not working and returned System.ArgumentOutOfRangeException error

          {
            "in": "body",
            "name": "boundary",
            "description": "Boundary object.",
            "schema": {
              "allOf": [
                {
                  "$ref": "#/definitions/Boundary"
                }
              ]
            }
          }

FATAL: System.ArgumentOutOfRangeException: Non-negative number required.
FATAL: System.ArgumentOutOfRangeException: Non-negative number required. Parameter name: newSize at System.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument, ExceptionResource resource) at System.Array.Resize[T](T[]& array, Int32 newSize) at System.Collections.Generic.Stack`1.Push(T item) at AutoRest.Modeler.OperationBuilder.<>cDisplayClass13_0.<BuildMethodReturnType>b0(Stack`1 typeStack) in /opt/vsts/work/1/s/src/OperationBuilder.cs:line 382 at System.Collections.Generic.List1.ForEach(Action1 action) at AutoRest.Modeler.OperationBuilder.BuildMethodReturnType(List`1 types, IModelType headerType) in /opt/vsts/work/1/s/src/OperationBuilder.cs:line 369 at AutoRest.Modeler.OperationBuilder.BuildMethod(HttpMethod httpMethod, String url, String methodName, String methodGroup) in /opt/vsts/work/1/s/src/OperationBuilder.cs:line 197 at AutoRest.Modeler.SwaggerModeler.BuildMethod(HttpMethod httpMethod, String url, String name, Operation operation) in /opt/vsts/work/1/s/src/SwaggerModeler.cs:line 404 at AutoRest.Modeler.SwaggerModeler.Build(ServiceDefinition serviceDefinition) in /opt/vsts/work/1/s/src/SwaggerModeler.cs:line 106 at AutoRest.Modeler.Program.<ProcessInternal>d__2.MoveNext() in /opt/vsts/work/1/s/src/Program.cs:line 60 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at NewPlugin.<Process>d__15.MoveNext() FATAL: csharp/imodeler1 - FAILED FATAL: Error: Plugin imodeler1 reported failure. Process() cancelled due to exception : Plugin imodeler1 reported failure. Error: Plugin imodeler1 reported failure.

I think it is related to this only