Phoenix: Revisit default channel protocol encoding

[chrismccord]

We can still use json, but the format could be easily compacted to save bits on the wire for every message

[jeregrine]

We might be able to also enable websocket protocol compression via cowboy
to get some speedups if the browser supports it.

https://ninenines.eu/docs/en/cowboy/1.0/guide/ws_protocol/

Cowboy's Websocket implementation also includes the x-webkit-deflate-frame

compression draft which is being used by some browsers to reduce the size
of data being transmitted. Cowboy will automatically use compression as
long as the compress protocol option is set when starting the listener.

On Fri, Feb 24, 2017 at 12:50 PM, Chris McCord [e-mail]
wrote:

We can still use json, but the format could be easily compacted to save
bits on the wire for every message

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/phoenixframework/phoenix/issues/2120, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AAGKFsU5165HIqP4k6jCC-RHcXjcEW1Oks5rfyZ_gaJpZM4MLjDQ
.

[chrismccord]

I believe this should already happen w/ compress: true

[jeregrine]

Right but we don't provide any guidance in the docs and etc. And it's not
something I think we'd enable for all connections by default.

On Fri, Feb 24, 2017 at 2:19 PM, Chris McCord [e-mail]
wrote:

I believe this should already happen w/ compress: true

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/phoenixframework/phoenix/issues/2120#issuecomment-282394165,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGKFg-mhCEmZfocPEXmMnc9uurDa5fDks5rfztEgaJpZM4MLjDQ
.

[dschniepp]

@chrismccord do you have a specific protocol in mind like Googles Protocol Buffers? What do you think is the main use case for channels, communication with JavaScript apps or more advanced apps?

[josevalim]

@dschniepp the goal is still to use json but format the json in a way it is less verbose than today. It is also worth pointing out that the serializer for Phoenix channels is cutomizable, so anyone can use protocol buffers if they want to.

[chrismccord]

What do you think is the main use case for channels, communication with JavaScript apps or more advanced apps?

Both! :)

As José said, we will use json by default so the browser client does not need to ship an extra decoder, but the goal from day one was for channels to be transport agnostic. As long as you can get the bits on the wire and encode/decode them with the required information, phoenix can accommodate your usecase.

[dschniepp]

Ok, got it. I thought you want to change the format itself and in case the majority of people use channels in non web based apps googles protocol buffers or something similar could reduce quite some bits on the wire.

[lcacciagioni]

From my perspective, I think that it would be good if we have documentation on how to use custom encoders/decoders for protocols like Thrift, Protocol Buffers or Avro over phoenix channels. Even a howto or a blog post can help into this.

[michalmuskala]

Something like https://github.com/inaka/jem.js might also be worth considering - I'm not sure about the efficiency on the JavaScript side of this, though.

[OvermindDL1]

Something like https://github.com/inaka/jem.js might also be worth considering - I'm not sure about the efficiency on the JavaScript side of this, though.

Hah, I love the look of that, and I was curious about its efficiency, it does not seem bad at all, once the javascript gets JIT'd it was within an order of magnitude normal unsafe JSON and about the even better than safe JSON libraries, it looks decently worth using, and easy enough to update for any newer BEAM types as time comes up too. Could even encode a way for anonymous functions to call local javascript functions of the same names, hmm... (this part not for phoenix, but for another library it would be useful)

[mbaileys]

Jem.js is interesting but its goal is to save processing rather than bits on the wire so it ends up being larger than JSON. The tiny LZ-String library might be a more efficient drop-in for bidirectional compression. Removing the base64 code it's less than 1KB of compressed JS.

Tested here : https://jsfiddle.net/ekfwzsw3/1/

[xadhoom]

Throwing my 2c here,

I've used :zlib quite successfully from client to server (but same can be easily done on the opposite direction), with on-the-fly compression switching on client side (to debug socket frames) which is handled transparently on server side.
The js lib used is pako.js .

The main reason was to compress webRTC stats messages, which are quite verbose and frequent.

[xadhoom]

Also, x-webkit-deflate-frame is deprecated (not part of the standard).
Now browser should use permessage-deflate, which is supported on Cowboy 2.
Or not?

[chrismccord]

The second version of the write protocol is compacted as a keyless json array rather than redundant keys as a json object. Further compression can be handled outside of core by third-party libs as it would require us to ship extra js deps.