Autorest: Spec for Paging that supports header extraction

[lmazuel]

Right now pageable supports the following syntax:
https://github.com/Azure/autorest/blob/master/docs/extensions/readme.md#x-ms-pageable

"x-ms-pageable": {
   "nextLinkName": "nextLink",
   "itemName": "values",
   "operationName": "Paging_nextOperationWithQueryParams" 
},

And nextLinkName is the name of a JSON key at the root of the returned JSON payload.

We want to be able to extract that nextLink from a header instead.

Proposition:

• Adding a nextLinkIsHeader: bool parameter, to say that nextLinkName is a header name from returned headers.
• By default value is "false" (meaning comes from JSON)

Alternative proposition (that I prefer less):

• Adding nextLinkOrigin: str parameter, that states the origin using in vocab from Swagger (body/query/header)
  I prefer it less, because I think it's overcomplicated, since except body/headers there is not that much possibility anyway.

This is needed for synapse, so I'd like to move fast on it and make it happen fast. I don't expect a lot of work on the m4 side (pass it through).

[daviwil]

I like your first proposal, though I think if we used in we could specify it to only ever be body or header in this case. Would it be too confusing to call the property in also? nextLinkIsHeader seems too tied to the body/header dichotomy in case there's some other place it could come from in the future (albeit not very likely).

[lmazuel]

@daviwil You mean:

"x-ms-pageable": {
   "nextLinkName": "nextLink",
   "nextLinkIn": "header",
   "itemName": "values",
   "operationName": "Paging_nextOperationWithQueryParams" 
},

I'm fine with that, usually I prefer more expressiveness, but in this situation I was ok with a bool. If you feel strong about an enum scenario I'm glad to follow you.

[daviwil]

Yep, that looks better to me! I prefer to err on the side of future extensibility when it's not very intrusive, like this case.

[pakrym]

We want to be able to extract that nextLink from a header instead.

Where is this requirement coming from?

[daviwil]

@pakrym, @lmazuel linked me to this bit of the Synapse spec: https://github.com/Azure/azure-rest-api-specs/blob/f9caf92527ccff06c5b66380e6f2b4f50f5e82b3/specification/synapse/data-plane/Microsoft.Synapse/preview/2020-02-01-preview/roleAssignments.json#L100. We realized this is referring to a continuation token in the header so we might need to tweak the proposed design slightly. I'll bring it up in scrum and post back notes from the discussion.

[daviwil]

At the scrum we discussed whether we should even support this header in a special way since it's just a header and not a Swagger extension. We need to have a larger conversation about what kind of custom extensions we will support in data plane specs (or if at all).

[joheredi]

Just for reference, Tables also does paging through headers, slightly different though, since they give you 2 continuation tokens
https://docs.microsoft.com/en-us/rest/api/storageservices/query-entities#response-headers