Subscriptions-transport-ws: 0.7.0 breaks with Graphcool Freecom, while 0.5.4 works

[eddiewang]

If you look at the github repo here: https://github.com/graphcool-examples/freecom-tutorial

We use this library to implement a subscription with websockets. With the latest version, this breaks with a 400 error:

That said, downgrading to 0.5.4 works fine. We aren't using the library in any weird way, so I'm wondering what is the cause of the error.

[mistic]

@eddiewang the protocol in the new 0.0.7 version was updated and it has a lot changes. It looks like to me that your server at subscriptions.graph.cool is not implementing this new protocol :) you can try using the new provided server for your graph cool server or update your own server :)

[Urigo]

@eddiewang 0.7 supposed to be backward compatible with 0.6.
Can you please check if it works for you with 0.6?

If not, we know Graphcool should have update to that first and we didn't broke something we did not want to break. but maybe also we broke something unexpected in 0.7 so it is important to know.

In anyway, Graphcool and Scaphold should try to update their transport to the new one as soon as they can and give us feedback on that..
@vning93 @mikeparisstuff @sorenbs @schickling @marktani

[twelvearrays]

I was having the same issue with 0.7.0 with my graph.cool setup. I rolled back to 0.6.0 and everything is working correctly.

[sorenbs]

Thank you @twelvearrays for confirming that the breaking change was introduced between 0.6 and 0.7.

@Urigo - do you have a list of stuff that changed in 0.7 that we can keep in mind when looking into this?

[dotansimha]

The issue here is different - because the graph.cool server is running 0.6.0 server, and you are trying to communicate it with 0.7.0 (which has a different protocol) - you are having this error.

The actual issue here is that you are trying to use 0.7.0 on your client (which uses ONLY the new protocol), but you are trying to communicate with 0.6.0 server - which support only the old protocol.

For users who work with graph.cool - at the moment, you solution is to keep using 0.6.0, and update it only after graph.cool updates their subscriptions service to use the new protocol.

The backward compatibility support for 0.7.0 is only for the old protocol. the case here is not related to backward compatibility because you are trying to use a new client with an old server (forward compatibility?? 😆 )

The new protocol has new features and support for query/mutation/subscriptions, along with better lifecycle hooks, updated dependencies and more (see CHANGELOG).

[helfer]

We should probably version the protocol if there are differences that make it incompatible with older versions.

[TheAncientGoat]

Couldn't find this issue via Google, Graphcool support pointed me here though - going to add the exact error to hopefully increase SEO rank

Websocket connection to  'wss://subscriptions.graph.cool/v1' failed: Error during WebSocket handshake: Unexpected response code: 400

[schickling]

Update: We're currently upgrading the Websocket connection services to support the new protocol changes. We'll be rolling out the new version over the coming days. It shouldn't break old versions.

[dotansimha]

@helfer
If there is a mismatch in the protocol, the server respond with 1002 error code, which the browser translates to Protocol Error (according to the WebSocket RFC).

[mavilein]

Hey @helfer and @Urigo,
i am working at Graphcool. I just looked into this issue and there are indeed multiple breaking changes. Here is an overview:

• The value of the header Sec-WebSocket-Protocol was changed from graphql-subscriptions to graphql-ws. link to source
• All message types have been renamed. link to source
• Also the protocol contains at least one more new message type. link to source

So i have a few questions now:

1. When were those changes introduced exactly?
2. How could we have known beforehand? These are pretty major changes and now we are caught by surprise by this.
3. We plan to support the old and new protocol side by side. Do you see any issues with that? Are there also incompatible changes with in the message structures itself?

[dotansimha]

@mavilein

The changes was introduced in 0.7.0, and the protocol has changed because this transport now support execution of query/mutation, not only subscriptions.
The current version also supports the old protocol messages, and the new messages types are just an addition to the old protocol.
I think the only issue now is the Sec-WebSocket-Protocol that has changed, because the protocol should work the same.

The current version of the transport support both protocol, the old and new.

We had some discussions here in issues and PRs about the implementation of the new protocol, and how to do these changes without breaking the current API, so maybe it's better if you'll be an active part in the future when talking about changes like this.

Do you experience any other errors using the new package on the server and the old package on the client?

[Urigo]

@mavilein welcome!
adding to what @dotansimha said, those changes were talked about a lot on the repo, on the Apollo Slack and on our contributors club.
Also that's one of the reasons we really tried to minimize the breaking changes of this particular version and we still completely support the old protocol....
Also I've talked about it with some of you from Graphcool and Scaphold on DM on Slack about 2 months ago and also at GraphQL Europe.
I guess that if within those 2 months you could have tested those changes, we might have been able to identify the change in Sec-WebSocket-Protocol and we or you could have addressed that during that development process.

Going forward there are a few things we can do better:

1. We are going to introduce the 1.0 version soon (just deprecating the old protocol and renaming the transport package to graphql-transport-ws). After that, we will go with semver which will make breaking changes better handled. It was hard to do that beforehand without the official support in graphql-js and finalizing it's API with Facebook.
2. As you are using this library, it would be great if you could watch the repo, be part of the discussions and give feedback. We are hungry for your feedback as you are very knowledgeable and a big user of this library. You are also already a contributor for this library by the way and any issue you wrote here in the past we addressed also immediately.
3. I will keep sending you messages and updates directly. I can also start messaging you directly @mavilein or any single point of contact that you decide on for the transport on your team.
4. You can also join the subscriptions channel on Apollo and keep track of all the discussions that are happening around that.

You are a big part of this library and I would love to hear any feedback about how we can do better and improve this library and it's releasing process!

[mavilein]

Hey @dotansimha and @Urigo ,
thanks for your nice and thorough responses! I am subscribed to this repo and am looking forward to future discussions =)
We had a little last misunderstanding: At Graphcool we are maintaining our own implementation of this protocol. I am currently working on implementing the most recent version of the protocol.

I have one last question around the semantics of the protocol:
In the old protocol the server responded with a message of type subscription_success when a subscription was created. In the new protocol i don't see the an immediate replacement. Should the server send a message of type data in this case? If so with which payload? Or should it send no message at all?

[sorenbs]

Great news for @eddiewang @twelvearrays and everyone else who want to use the latest version of the subscriptions client with their Graphcool project - it now works :-D

With the small caveat that you need to manually override the timeout setting when instantiating the SubscriptionClient:

const wsClient = new SubscriptionClient('wss://subscriptions.graph.cool/v1/__PROJECT_ID__', {
  reconnect: true,
  timeout: 20000
})

This is all you need to do to use the latest version of subscriptions 🚀

---

Summary of our findings:

In addition to the change in protocol and header as outlined above we found 3 changes in behaviour. The first two we have implemented workarounds for, the third we are providing a simple workaround for developers while we wait for a solution in the client.

1. https://github.com/apollographql/subscriptions-transport-ws/issues/173 The protocol specifies that message id should be string, but Apollo Client sends a number. We used to work around this by accepting a number, and immediately turning it into a string on the server. When we responded with data we would send back the string representation of the number, and the Client prior to 0.7 accepted this. It no longer does. We now return a number when receiving a number.
2. https://github.com/apollographql/subscriptions-transport-ws/issues/174 The protocol specifies that the operationName is an optional string. It used to be that the client would send a null value when no operation name was specified. Now it sends "" (the empty string) which we treated as invalid. We now treat the empty string as null.
3. https://github.com/apollographql/subscriptions-transport-ws/issues/167 The keep-alive mechanism is important to keep the connection open in all network topologies. There seems to be issues with the current implementation in 0.7. We advise developers to manually configure the client timeout to 20 seconds as per above.

Thanks to everyone working on the client who has helped us debug these issues!

[twelvearrays]

@sorenbs I will update today in my project. Thanks

[dotansimha]

@mavilein
The subscription_success was removed in the new protocol, because the client does not need an indication for the success of the operation (only for the error, if there is any). I the server need to fail the subscription is can throw an error, causing the GQL_ERROR to be sent, and the client will receive GQL_DATA when the server publishes data.

[dotansimha]

@sorenbs
Can you please explain the issue with the timeout?
I know that there is an issue with the docs, but the timeout actually used for settings the timeout between keep-alive messages (the default is 5000ms, but the server should also send the keep-alive messages, otherwise this parameter is not in use at all).

[sorenbs]

@dotansimha resolved in https://github.com/apollographql/subscriptions-transport-ws/issues/167 :-)

[Urigo]

@sorenbs thanks for the super valuable feedback!

thanks to @dotansimha , we've just released 0.7.2 with the following changes accourding to your great suggestions.

So that means that the points from your feedback are:

1. We've corrected the message id to be a string
2. Operation name has been fixed in Apollo Client and will be released soon
3. Increased the KeepAlive to 30 seconds

@sorenbs do you think we can close the issue?

[Urigo]

also correction - [email protected] has been released with the needed fix

[sorenbs]

Please do :-D

[marktani]

Thanks so much guys for the great work 💪

[delgermurun]

Upgraded to 0.7.2 still getting an error.

WebSocket connection to 'wss://subscriptions.graph.cool/v1/blabla' failed: Connection closed before receiving a handshake response

[eliezedeck]

On Firefox, 0.7.2 generates:

The connection to wss://subscriptions.graph.cool/v1/blabla was interrupted while the page was loading.

[sorenbs]

@delgermurun and @eliezedeck

I believe this might be a separate issue. Can you both dm me in the Graphcool slack so we can look into this together?

[delgermurun]

Works fine now. Thanks

[Urigo]

@sorenbs @marktani @schickling I'm going to release a new version with a lot of great changes by @mistic .
there shouldn't be a breaking changes but just to make sure, I've just released [email protected] so you could verify that everything works on your end before I release 0.8.0.
For a list of changes in this version, check out the changelog

Waiting for you prompt response, thanks!

[marktani]

I did a quick sanity check with one of our examples and it works great 👍

However, I am receiving a deprecation warning for addGraphQLSubscriptions.

[Urigo]

@marktani thank you so much for checking!
yes it will be deprecated soon with the new network interface, people can keep track on it here - https://github.com/apollographql/subscriptions-transport-ws/issues/169

I've updated the message to be more understandable though, thanks for the feedback..

0.8.0 has been released!