Autorest: Autorest python failing for x-ms-pageable extension while passing for CSharp

[DheerendraRathor]

Input Json: https://github.com/Azure/azure-rest-api-specs/blob/master/arm-recoveryservicesbackup/2016-12-01/swagger/backupManagement.json
Autorest version: 1.0.1-20170517-2300-nightly

Failure error:

FATAL: System.Collections.Generic.KeyNotFoundException: Couldn't find the item property specified by extension
   at AutoRest.Python.Azure.TransformerPya.GetPagingSetting(CodeModelPya codeModel, CompositeType body, Dictionary`2 extensions, String valueTypeName, IDictionary`2 typePageClasses, String methodName)
   at AutoRest.Python.Azure.TransformerPya.NormalizePaginatedMethods(CodeModelPya codeModel)
   at AutoRest.Python.Azure.TransformerPya.TransformCodeModel(CodeModel cm)
   at Generator.<ProcessInternal>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NewPlugin.<Process>d__11.MoveNext()
FATAL: python/generate - FAILED
Process() Cancelled due to exception : Plugin python reported failure.
Error: Plugin python reported failure.
    at C:\Users\dhratho\.autorest\plugins\autorest\1.0.1-20170517-2300-nightly\node_modules\autorest-core\lib\pipeline\pipeline.js:88:19
    at next (native)
    at fulfilled (C:\Users\dhratho\AppData\Roaming\npm\node_modules\autorest\lib\polyfill.min.js:10:59)
    at process._tickCallback (internal/process/next_tick.js:109:7)

This error is coming for Python generator while working fine for CSharp generator.

Diagnostics:

• In paths with operation_id Operations_List, response schema name is ClientDiscoveryResponse which is pageable and hence NextLink and Value fields are present in schema.
• In schema properties NextLink and Value are present, but python generator is looking for "value" field which is not present and hence failing. (Autorest.Python.Azure TransformerPya.cs#129).
• But on the other hand in CSharp generator (Autorest.CSharp.Azure TransformerCsa.cs#184 GetPagingSetting) no such check is present.
• On further analysis it turns out that this check is present only in Python generator, which presents inconsistency across generator transformers.

[lmazuel]

The Python code is looking into the "itemName" Swagger conf, or default to "value". This means that your pageable operation does not have a return type that satisfies this behavior.

This is consistent with the pageable extension declaration:
https://github.com/Azure/azure-rest-api-specs/blob/088e6d861c135aec2e82a8393cc0591d58128f5c/documentation/creating-swagger.md#Paging-x-ms-pageable

[lmazuel]

More precisely, your definition is:

    "ClientDiscoveryResponse": {
      "description": "Operations List response which contains list of available APIs.",
      "type": "object",
      "properties": {
        "Value": {

meaning, you are using Value (note the uppercase). The default value if itemName is not present is value (lowercase, see previous link).

@olydis @fearthecowboy should itemName content be case-sensitive?

• If yes, then this is a Swagger issue: Swagger should contain itemName: Value
• If no, then this is an Autorest Python issue: Value should be consider correct for default value

[olydis]

indeed a Swagger issue, not sure why C# even succeeds o.O

[DheerendraRathor]

@olydis C# generator doesn't check whether property with itemName actually exists. Surprising thing is only Python generator is making this check and even Python generator is missing check for nextLinkName (though code exists, it is just not enforcing check).

I believe python is making these checks as in-built json module in python is case sensitive.

Although this can be made consistent across languages by adding these checks appropriately.

[lmazuel]

@DheerendraRathor @olydis
nextLinkName is optional, because it's possible that this does not exist. You can have a "list" operation that does not support paging at the RestAPI level, then there is no nextLink possible, but you want the nice paging experience at the SDK level, so you use this extension.
However, itemName is critical, since it tells you where is the array you consider to be a list. Python code is very different than C# code for paging, since we return an "iterator", that will automatically call the next page for you when needed. You will never see a "next" method and then we generate only ONE "list" method. The purpose of the iterator is to return element by element: we don't return an array. This means that locating precisely the array in the JSON is critical.
I don't know why C# is not testing that, I don't know what are the constraints in C# compared to Python. Shouldn't be a linter rule that check itemName content or value is in the Swagger?

[olydis]

Shouldn't be a linter rule that check itemName content or value is in the Swagger?

excellent idea, I'll file an issue!

[olydis]

I'll close, since this was just a Swagger issue, correct? Feel free to continue discussing or reopen if there is indeed a bug we have to fix in generators.