Azure-sdk-for-js: Azure schema registry does not work if schema JSON contains newlines

[dhoepelman]

• Package Name: @azure/schema-registry
• Package Version: 1.0.0-beta.1
• Operating system: OS X 11.2
• [x] nodejs
  
  
  
  • version: v14.15.4
  
  
• [ ] browser
  
  
  
  • name/version:
  
  
• [x] typescript
  
  
  
  • version: 4.1.3
  
  
• Is the bug related to documentation in
  
  
  
  • [ ] README.md
  
  
  • [ ] source code documentation
  
  
  • [ ] SDK API docs on https://docs.microsoft.com
  
  

Describe the bug

The azure schema registry does not function if the schema content contains newlines

To Reproduce
Steps to reproduce the behavior:
1.

await new SchemaRegistryClient(..., new DefaultAzureCredential()).getSchemaId({
  group: 'test'
  name: 'Test',
  serializationType: 'avro',
  content: '{\n' +
      '  "type": "record",\n' +
      '  "name": "Test",\n' +
      '  "fields": [{ "name": "X", "type": { "type": "string" } }]\n' +
      '}\n'
})

// will always return rejected promise with schema not found

or

2.

await new SchemaRegistryClient(..., new DefaultAzureCredential()).registerSchema({
  group: 'test'
  name: 'Test',
  serializationType: 'avro',
  content: '{\n' +
      '  "type": "record",\n' +
      '  "name": "Test",\n' +
      '  "fields": [{ "name": "X", "type": { "type": "string" } }]\n' +
      '}\n'
})

// will throw error
// RestError: {"Code":400,"Detail":"Schema validation failed: Invalid character after parsing property name. Expected ':' but got: \". Path '', line 1, position 4.

Expected behavior

Schema newlines (and whitespace) should not affect the API, as long as it results in the equivalent JSON

At the very minimum, this should have some major warnings in the SDK docs and typings

[nguerrera]

@dhoepelman Thanks again for the feedback. I will take a look. I believe this is a response from the service judging from the 400 in the thrown error. Agree this is totally unexpected either way. I'll confirm if it's the service behaving this way and forward the feedback along to the service owners if so. If not, I'll fix it on our side. :)

[nguerrera]

Heads up, @hmlam, I'll let you know what I find, but maybe you've seen something like this already?

[dhoepelman]

It's not on the server side, because replicating the underlying call with curl works fine if the schema contains newlines

curl --location --request PUT 'https://<my schema registry>.servicebus.windows.net/$schemagroups/<my-schema-group>/schemas/Test?api-version=2017-04' \
--header 'Authorization: Bearer <portal.azure.com token>' \
--header 'x-schema-type: avro' \
--header 'Serialization-Type: avro' \
--header 'Content-Type: application/json' \
--data-raw '{
  "type": "record", 
  "name": "Test", 
  "fields": [{ "name": "X", "type": { "type": "string" } }]
}
'

[nguerrera]

OK, interesting and surprising, I'll dig in. Thank you for the added detail

[nguerrera]

So far, I haven't managed to reproduce this.

When I registered the schema using code above, it succeeded.

[nguerrera]

I'm going to retry with 1.0.0.beta.1, maybe this was fixed since.

[nguerrera]

Yep, something fixed this between 1.0.0-beta.1 and now! 😁

I'm fairly convinced it was https://github.com/Azure/azure-sdk-for-js/pull/11650. There had been some issues with the swagger and autorest around the body being a raw string. Also, there really hasn't been any other interesting changes since beta 1.

C:\Temp\repro
> cat test.js
const { SchemaRegistryClient } = require("@azure/schema-registry");
const { DefaultAzureCredential } = require("@azure/identity");
const dotenv = require("dotenv");
dotenv.config();


new SchemaRegistryClient("https://nicholgsrplayground.servicebus.windows.net", new DefaultAzureCredential()).registerSchema({
  group: 'azsdk_js_test_group',
  name: 'Test',
  serializationType: 'avro',
  content: '{\n' +
      '  "type": "record",\n' +
      '  "name": "Test",\n' +
      '  "fields": [{ "name": "X", "type": { "type": "string" } }]\n' +
      '}\n'
}).catch((e) => {console.error(e.stack); process.exit(1)});

C:\Temp\repro
> npm install @azure/schema-registry
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ @azure/[email protected]
updated 1 package and audited 33 packages in 1.215s
found 0 vulnerabilities


C:\Temp\repro
> node test.js
RestError: {"Code":400,"Detail":"Schema validation failed: Invalid character after parsing property name. Expected ':' but got: \". Path '', line 1, position 4. TrackingId:83146526-9197-49cc-acd5-16678d07e1e9_G11, SystemTracker:nicholgsrplayground.servicebus.windows.net:$schemagroups\/azsdk_js_test_group\/schemas\/Test, Timestamp:2021-05-05T21:57:18"}
    at new RestError (C:\Temp\repro\node_modules\@azure\core-http\dist\index.js:2362:28)
    at handleErrorResponse (C:\Temp\repro\node_modules\@azure\core-http\dist\index.js:3246:17)
    at C:\Temp\repro\node_modules\@azure\core-http\dist\index.js:3182:18
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

C:\Temp\repro
> npm install C:\src\azure-sdk-for-js\sdk\schemaregistry\schema-registry\azure-schema-registry-1.0.0-beta.2.tgz
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ @azure/[email protected]
removed 2 packages, updated 2 packages and audited 1 package in 1.08s
found 0 vulnerabilities


C:\Temp\repro
> node test.js

[dhoepelman]

Ah ok, npm registry lists 1.0.0-beta.1 as the latest version. Is there any planning to release a new version?

[nguerrera]

Yes, we'll try to push a release. I'll update here when I have an ETA.