Fastify: Not handling Chinese characters correctly

[bhanuc]

Replicate issue:

'use strict'

const fastify = require('fastify')();

fastify
  .get('/', function (req, reply) {
    reply
      .send({ hello: '菜鸟驿站' })
  })

fastify.listen(3000, err => {
  if (err) throw err
  console.log(`server listening on ${fastify.server.address().port}`)
})

It returns -> {"hello":"菜鸟驿站"}, I tried to debug the issue in the core, but seems string is fine even till the call of wrapReplyEnd fn, but could not follow the flow afterwards.

[evanshortiss]

@bhanuc look like setting the charset in the content-type header will do it. Had a similar issue recently, but not with fastify.

Here's a quick fix:

'use strict'

const fastify = require('./')();

fastify
  .get('/', function (req, reply) {
    reply
      .header('Content-Type', 'application/json; charset=utf-8')
      .send(JSON.stringify({ hello: '菜鸟驿站' }))
  })

fastify.listen(3000, err => {
  if (err) throw err
  console.log(`server listening on ${fastify.server.address().port}`)
})

Looks like this line could be updated to include the charset?

[delvedor]

@evanshortiss nice catch!
Since we are refactoring the send code I'll add your suggestion in https://github.com/fastify/fastify/pull/330 :)

[StarpTech]

After some research, I found out that this is not a bug. The default encoding of JSON is UTF8. If you view the response directly in the browser and you don't set the correct page charset the browser will misinterpret it. Try to request your API with CURL or Postman you will see the correct output. We shouldn't add it because of several reasons:

https://github.com/request/request/issues/383

• The standard does not define a charset parameter it requires that JSON is always encoded in Unicode and UTF8 is the default encoding. https://tools.ietf.org/html/rfc4627#section-3

https://github.com/akka/akka-http/issues/1482#issuecomment-337363859

This is a well-known bug in various browsers. Some have fixed it by now but not all. We are currently not considering fixing this because it's specified very clearly that no charset parameter is needed. You can still create a custom content type if you need very specific behavior and the defaults don't suite you.

That's correct e.g in firefox the content is displayed correctly.

[mcollina]

Thanks for chiming in @StarpTech! I think we should not add the charset and close this issue.