Nest: TCP Microservice doesn't work with unicode characters in message

Created on 18 Mar 2019 · 8Comments · Source: nestjs/nest

I'm submitting a...


[ ] Regression 
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

If TCP Microservice message handler tries to send a string that contains unicode character (e.g. é), the next error will appear:
[Server] There is no matching event handler defined in the remote service.

Expected behavior

No errors should appear.

Minimal reproduction of the problem with instructions

@Controller()
export class TCPController {
  @MessagePattern('getUnicodeString')
  async getUnicodeString(payload: any) {
    return 'é';
  }
}

What is the motivation / use case for changing the behavior?

Environment


Nest version: 6.0.0


For Tooling issues:
- Node version: 11.10.0  
- Platform: Windows 

Others:

In addition

Trying to understand where this error comes from, I found out that the root of the problem is in json-socket package.
So as we know Nest.js TCP Microservice server sends to client messages in the following form

export interface WritePacket<T = any> {
  err?: any;
  response?: T;
  isDisposed?: boolean;
}

When data transmission is done server sends the finishing message

{ isDisposed: true }

So if the service from the example above just sends one string of one character é, TCP Microservice server will send two mesages. The fist one

{
  err: null,
  response: 'é'
}

and the second one

{ isDisposed: true }

And here json-socket comes into play. Before each JSON.stringified message it puts the string length value and specific delimeter (#). For instance

'28#{"err":null,"response":"é"}'

Among other things this is made to distinguish messages when they come in one packet like this one

'28#{"err":null,"response":"é"}19#{"isDisposed":true}'

Under the hood json-socket uses the given length value and substring function to extract neccesary message. But considering the fact that json-socket calculate the string length with Buffer.byteLength function, the length for é character will be 2 istead of 1. This circumstance causes error as instead of first expected message

'{"err":null,"response":"é"}'

we'll get

'{"err":null,"response":"é"}1'

Since this is not a valid json, json-socket on the client will return back to the server the next error

'148#{"success":false,"error":"Error: Could not parse JSON: Unexpected number in JSON at position 27\\nRequest data: {\\"err\\":null,\\"response\\":\\"é\\"}1"}'

The server won't be able to define what type of message it is, as there's no MessagePattern sent within it. Hence we get our error
[Server] There is no matching event handler defined in the remote service.

Could you comment on this issue?

BTW there's an opened pull request in the json-socket repository (https://github.com/sebastianseilund/node-json-socket/pull/46), that must address this issue. It was opened one year ago. Furthermore, it seems like the json-socket repo is not mainained for more than one year.

microservices type todo 💚

Source

ivibe

Most helpful comment

Fixed in 6.0.3. Thanks @ivibe

kamilmysliwiec on 26 Mar 2019

👍2

All 8 comments

Basically, this issue was reported here https://github.com/nestjs/nest/issues/858 already. However, you have provided further information, so let's track this bug here.

BTW there's an opened pull request in the json-socket repository (sebastianseilund/node-json-socket#46), that must address this issue. It was opened one year ago. Furthermore, it seems like the json-socket repo is not mainained for more than one year.

We will very likely take this code and maintain it directly from within @nestjs/microservices soon.

kamilmysliwiec on 19 Mar 2019