Rocket.chat: [sandstorm] Mobile and Descktop clients can't connect to Sandstorm RC grain

[jadeqwang]

Right now, I can't seem to figure out how to get my local RC client on my Mac to connect to my Sandstorm RC grain. Probably because it isn't possible yet.

@Sing-Li ?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[engelgabriel]

Did you try this?

[jadeqwang]

Hey @engelgabriel, that gets me to the prompt to enter a host URL, but I'm not sure what to put into the "host URL."

For instance, I tried using a Share link that would ordinarily be able to open that instance in a new browser window, but I get the error "Not a valid instance url." I also try using the webkey (which is how Sandstorm connects to things like mobile apps). Same error.

I think there's some plumbing underneath that needs to get looked at.

[engelgabriel]

It checks for a specific http header, to make sure it is a RC server. Maybe Sandstorm is removing this unknown header?

[engelgabriel]

We will change from a custom header to a /info rest api

[engelgabriel]

relates to https://github.com/RocketChat/Rocket.Chat.Cordova/issues/26

[nathanhourt]

So is it currently impossible/infeasible to connect the apps to a RC grain on Sandstorm? Any idea on when this might be doable?

[engelgabriel]

We just need a fixed URL to connect to, we have already removed the dependency on custom http headers

[Sing-Li]

@jparyani - can you please describe the magic involved ? You are the only known human living today who had successfully integrated a mobile app into Sandstorm (quoting @kentonv )

[nathanhourt]

By "fixed URL" do you mean a URL that doesn't change, or that something needs to be fixed before it'll work? :)

I've already tried pointing the server to the underlying subdomain SS creates for the grain, but that just hands back Unauthorized errors.

[jparyani]

@nathanhourt The subdomains that Sandstorm creates are ephemeral (they only last as long as your web browser is open to that tab), and as you saw, they check for a cookie that is only given to the client that created that session.

Sandstorm requires api access to go through a webkey (see https://docs.sandstorm.io/en/latest/developing/http-apis/#about-webkeys). Further up on that page, it shows you how to add an offer template to your app, which is our recommended way of displaying to users how to set this up. If you don't add an offer template, users can still generate a webkey manually through the Sandstorm UI by clicking the key icon.

The one thing that Rocket.Chat mobile apps will probably have to change is to handle how authorization with webkeys works. An example webkey is https://alpha-api.sandstorm.io#49Np9sqkYV4g_FpOQk1p0j1yJlvoHrZm9SVhQt7H2-9. The part after the # is a bearer token that needs to be sent as an authorization header. For example, a curl statement accessing the webkey above would look like:

curl -H "Authorization: Bearer 49Np9sqkYV4g_FpOQk1p0j1yJlvoHrZm9SVhQt7H2-9" https://alpha-
api.sandstorm.io

[engelgabriel]

@jparyani the mobile app doesn't use any APIs, it is just a webview :) We just need access to the webapp itself without the sandstrom "wrapper". Is that possible in any way?

[kentonv]

In theory your "API" could just serve up regular web content. There are a couple quirks, though.

Sandstorm generally does not want apps hosting web content at a fixed URL for security reasons: Our sandboxing, and our added CSRF defense, depend on randomized per-session hostnames. APIs, however, obviously must have a fixed location, so we have the API endpoint for that. But, since we don't want regular web browsers rendering HTML content from our API endpoint (since it would defeat our sandboxing), we require the Authorization: bearer header (which web browsers never send for a normal browsing context) and we serve the content with headers like Content-Security-Policy and Content-Disposition to try to convince browsers that they absolutely should not render HTML from this endpoint. We also drop all cookies.

Now, if you have a mobile app that is just a webview locked to the rocket.chat server, then arguably it's OK for you to bypass Sandstorm's sandboxing -- or more accurately, it is your app's responsibility to implement any sandbox, not Sandstorm's, because your app is not hosted by Sandstorm. Crucially, though, your app's webview should _not_ be rendering any other sites and should _not_ be sharing a cookie jar with the user's main web browser.

In this scenario, what your app can perhaps do is fetch web content directly from the API endpoint and pass it to the WebView with the following modifications:

• Add the appropriate Authorization header to all requests, as described above.
• Strip off the Content-Security-Policy and Content-Disposition headers before passing along to the webview.
• Do NOT allow the WebView or any nested iframe to browse to any third-party host, unless you are confident you've mitigated the CSRF risks somehow. It's probably better to open the user's regular web browser instead.

Despite the long explanation, I think the above should be easy to implement? I suggest triggering all this specifically when the user enters a webkey as the host -- i.e. a URL that contains a # character as described above.

[memento]

Hi !
I'm also interested ! ;)

[engelgabriel]

I propose a hackathon of the Sandtorm and Rocket.Chat teams to get this working :)

[memento]

...and I'm offering the reward cookies ;)

[jonathandey]

+1 for this!

[kentonv]

Is anyone on the Rocket.Chat side working on this? How can we help?

[engelgabriel]

How can we get the API endpoint?
The user is expected to type that URL on the desktop/mobile client login form?

[engelgabriel]

Found this:

[engelgabriel]

OK, so:

https://api.oasis.sandstorm.io#R75AxS08WI_MLEks27sn0kyDOGEVqVPjN97KWudxA8H

should result on a request like this:

curl -i https://api.oasis.sandstorm.io -H "Authorization: Bearer R75AxS08WI_MLEks27sn0kyDOGEVqVPjN97KWudxA8H"

that gave me our index html indeed...

[patcon]

Ah, interesting... Didn't realize sandstorm allowed that so easily:

if anyone is using httpie command, this works:

http GET https://api.oasis.sandstorm.io/api/version Authorization:'Bearer R75AxS08WI_MLEks27sn0kyDOGEVqVPjN97KWudxA8H'

[engelgabriel]

Depends on:
https://github.com/RocketChat/Rocket.Chat.Electron/issues/97
https://github.com/RocketChat/Rocket.Chat.Cordova/issues/74

[rodrigok]

https://github.com/RocketChat/Rocket.Chat.Electron/tree/sandstorm

[engelgabriel]

@kentonv we will need your help here, please see the branch @rodrigok started and help us to try to make it work.

[kentonv]

For now, use the key icon in the top bar to create an API key, as in @engelgabriel's screenshots above. Let's get that much working first.

The UX of the key icon is not good. We have an alternative called "offer templates", which allows an app to tell the user inline what token to use:

https://docs.sandstorm.io/en/latest/developing/http-apis/#creating-an-offer-template

Check out Davros for an example of this in action (click on "clients" in the Davros sidebar -- the "server name" and "password" fields on this page are both offer templates).

However, offer templates make the most sense for command-line apps like git, where people are expected to copy/paste text. It's probably not a good UX for people trying to connect Rocket.Chat on their phone, where copy/paste works poorly.

A better option would be "QR code offer templates", as described here:

https://github.com/sandstorm-io/sandstorm/issues/1120

Basically, the user would click a button in the rocket.chat app on Sandstorm on their desktop labeled "connect mobile app". They would then be presented with a QR code. They scan the code with their phone, and rocket.chat on the phone handles the request and configures itself for the server. We have some work to do to define exactly how this works, but I think it is the ideal solution in the future.

@engelgabriel @rodrigok Are you having any specific problems that I can help with? I looked at the branch and it seems reasonable.

[engelgabriel]

@rodrigok can you describe better where you got stuck?

[rodrigok]

We can connect to the API and get the HTML content by we can't render because the Content-Security-Policy, we are just pointing the webview to the URL passing the auth header.

We can't get the HTML via AJAX because we need an local web server to server the page, and we don't have that.

[kentonv]

You have no way to strip headers from the response before the WebView renders them?

Hmm, what if you did something like in this SO question to intercept the requests and fulfill them yourself?

http://stackoverflow.com/questions/3134389/access-the-http-response-headers-in-a-webview

[kentonv]

For that matter, Rocket.Chat only really has one HTML page, right? Instead of intercepting the request, perhaps you could load that one page using a direct HTTP request and then inject the HTML into the WebView? All the other resources (images, javascript) should work OK despite Content-Security-Policy, right?

[rodrigok]

I have no ideia if the Ajax requests will work if we "inject" the html data, probably we will need a local webserver to serve the page as we do in mobile apps, but this add a lot of more complexity and I can't do that now.

If we solve this part, how the page will connect to the server via websocket?

[kentonv]

@rodrigok I think the AJAX requests should work fine, because as long as you have the correct Authorization header then we enable CORS on this endpoint.

[engelgabriel]

Do WebSockets connections work?

[kentonv]

@engelgabriel Yes, as long as you have the Authorization header.

[paulproteus]

For those who are curious: @jparyani has been spending some time in the last few days seeing how to address this.

[Sing-Li]

:+1: it'll be a major win - instant mobile chat/messenger server ... rentable by the minute.

Can't wait. Go @jparyani , go !

[ovizii]

Can't wait. Right now rocketchat on my Android is taking that webkey and asking for a user/pass which apparently rocketchat on sandstorm isn't generating yet :-/

[kentonv]

Technically #3437 doesn't fully resolve this issue but is just one part. There's also: https://github.com/RocketChat/Rocket.Chat.Cordova/pull/114

[compuguy]

Looks like this is still not working with the Android app, and the electron app.

[daira]

At Zcash we would like to use Rocket.Chat on Sandstorm, but will probably use it without Sandstorm unless this issue is fixed (or seems like it is going to be fixed soon). What's the current estimate for when this could be fixed?

[kentonv]

@jparyani Can you update people on the current state of the world?

[jparyani]

Currently the mobile app is working under Sandstorm. It requires you to click the key icon from the Sandstorm top bar and copy/paste that URL into the app.

I've been hesitant to recommend it though since it's still missing mobile notifications, which is what I imagine most people want the mobile app for.

@Sing-Li or other Rocket.chat devs, how do mobile notifications work today for self-hosted Rocket.chats? What would be the best way to get it working under Sandstorm?

[geekgonecrazy]

@jparyani out of the box Rocket.Chat uses our push gateway. When a notification needs to be sent it hits our gateway via an http request. https://gateway.rocket.chat I believe is the address. It then delivers to GCM / APN.

Alternatively any server could choose not to use our gateway. At that time the server would make a request to both the GCM endpoint as well as APN endpoint.

[JamborJan]

@jparyani just to let you know: it's not working like you describe it. The app is connectiong and downloads something from the server but then this happens and never stops:

Waiting for server connection, trying again in 21 seconds ... Try now

FYI: Desktop app doesn't work at all.

Tested with latest iOS version and latest rocket chat app version for iOS and rocket chat mac client.

[alesarrett]

I confirm the behavior described by @JamborJan with my Android 4.2.2 client

[SheilaRuth]

I just tried this with the iOS app with a key created in Sandstorm, and when I clicked "Connect" I got an error that the server is not enabled for mobile apps.

[rawray7]

we are also experiencing this same issue...cannot get iOS to connect to our grain. any thoughts on a timeline here? the sandstorm/rocketchat combo is a toolset we would love to use, but no mobile is kind of a deal-breaker for my team.

[jparyani]

Sorry, it turns out there's still an outstanding bugfix that hasn't been released for the mobile apps yet (https://github.com/RocketChat/Rocket.Chat.Cordova/commit/050af1659ae9f3161cedbf6dff084e8eaa618335). Next time it's updated, it should then work with Sandstorm.

[geekgonecrazy]

@RocketChat/core Looks like we still need to cut a release with the above fix?

[wtebbens]

if anyone managed to fix or configure this and get it working, we would be really interested!

[rawray7]

+1 - any update here on a new release or a workaround? i'm having a hard time convincing my team to use this with no mobile<>sandstorm compatibility.

[keypair]

Does it work now? Update please! Mobile support is crucial for so many users.

[memento]

If it works, could a good samaritan provide us with a quick tutorial ?

[keypair]

Users are asking how they can log-in into the mobile rocket.chat app without username and password? Please provide instructions.

[kentonv]

It looks like the app works! (Tested on Android.) (EDIT: And iOS!)

Here's what to do:

1. On your phone in a browser open Sandstorm and open the Rocket.Chat grain.
2. Open the hamburger menu (upper-left) and choose "Get Webkey". (On the desktop view, you would click the key icon in the topbar.)
3. Optionally enter a label (e.g. "phone") and then click "create".
4. You'll be presented with a URL, aka a "webkey". Long-press on this URL, then choose "copy link address".
5. Open the rocket.chat app.
6. Paste in the copied URL (the whole thing!) as your Rocket.Chat server address.
7. That's it! No password is needed, since an authorization token is encoded into the URL.

We know this is pretty clunky right now and we have another change coming to add a simpler one-click configuration to the Rocket.Chat app UI.

Note that currently, unfortunately, push notifications don't work. This is because the push notifications system on both iOS and Android assumes that each app has only one notification server worldwide. This works great for cloud services but is totally wrong for self-hosted services like Rocket.Chat and Sansdtorm. :( We're looking into whether Sandstorm can host a central notification relay as a work-around, but this might take a while to build.

[memento]

Epic
You're a hero
(Yes, it works)

[engelgabriel]

This is awesome! @Sing-Li can you help add this to our Docs?

How could we solve the "clunkyness"? Maybe he the URL with a QR code?

@kentonv can the Rocket.Chat instance on Sandstorm use our own central notification relay?

What is the status on the Desktop app?

[Sing-Li]

Wow - awesome work indeed!!
@engelgabriel sure - I'll add the instructions to the mobile app page.

[kentonv]

@engelgabriel Solving the clunkiness is pretty straightforward. We need to add a clickable link somewhere in Rocket.Chat's web UI which will auto-configure the app. So then the process will be:

1. Install the app, but don't open it.
2. In a browser, open Sandstorm, and open the Rocket.Chat grain.
3. Click the magic link, which will be handled by the app.
4. The app opens and auto-configures itself.

The link needs to be a Sandstorm "offer template". Don't worry about that for now, though. The main question we need to answer is: Where should we put the link inside the UI so that users can find it?

[kentonv]

One possibility is that if the user opens Rocket.Chat in a mobile browser, it could display a full-page interstitial with two options: "Continue in browser", "Open in app".

[engelgabriel]

Is the Desktop app working via the same way?

[kentonv]

@engelgabriel I don't think the desktop app was ever updated in the same way the mobile apps were. :/

[engelgabriel]

@kentonv we have been working a lot on getting the Electron apps built and publish more widely, maybe is a good time to focus on fixing the issues? What do you recommend @rodrigok to do so it works as the Cordova apps do?

[kentonv]

@engelgabriel Same changes as were made by @jparyani to the mobile apps, basically. Need to accept a webkey as the server address, and then use the key portion as the Authorization header.

[sethlester]

Is there a workaround for this for the desktop apps?

[yanivz]

Hi kentov, about point 6 (Paste in the copied URL (the whole thing!) as your Rocket.Chat server address.)
I've installed the rocket.chat on sandstorm (and want to link it as embed to another site [Dolphin Platform]),

1. How and where i am passing the link?
2. Today i see a prompt when entering to the website (where i linked Rocket.Chat). The page asks usr&pwd, when i click on cancel comes the message:

Missing or invalid authorization header.

This address serves APIs, which allow external apps (such as a phone app) to
access data on your Sandstorm server. This address is not meant to be opened
in a regular browser.

• i am login to sandstorm via google sso, i don't have user & password

[kentonv]

Hi @yanivz,

My instructions were specifically for the Rocket.Chat mobile apps, which have been extended to understand Sandstorm webkeys. Unfortunately it won't work for other apps that don't support this format.

[sethlester]

@kentonv , Can the same work that enabled sandstorm logins on mobile be done on desktop? Or is there some reason the desktop apps cannot support the format?

[awfm9]

How hard is it to port the fix from the mobile app to the desktop client? I would imagine it would be rather simple?

[kentonv]

@sethlester @awishformore It should indeed be roughly the same work -- look for changes by @jparyani on the mobile app, then apply something similar on desktop. Unfortunately the Sandstorm team no longer employs anyone full-time, so we'll need a volunteer to work on this.

[awfm9]

Yes, I saw and read up on the story. A rather sad development, if I might say. Competing platforms don't offer the same level of security, unfortunately. I should have some time to take a look in ~3 weeks.

[spacegoing]

I tried on my Android phone but it says invalid server version. Any idea for this?

[geekgonecrazy]

@spacegoing if you are using the Rocket.Chat+ app it doesn't yet support sandstorm you will need to use the other one

[awfm9]

I'm afraid we completely moved away from sandstorm because of UX and other usability issues. It's a shame when you see such a promising project not going where it could.

[spacegoing]

@geekgonecrazy Yes I installed the plus version. However I'm building my app from source (don't have access to google play). Would you please tell me which repo should fork (which one is rocket.chat)?

[rebeld]

@spacegoing:
Rocket.Chat+ (Native iOS): https://github.com/RocketChat/Rocket.Chat.iOS
Rocket.Chat+ (Native Android): https://github.com/RocketChat/Rocket.Chat.Android
Rocket.Chat (Cross platform mobile): https://github.com/RocketChat/Rocket.Chat.Cordova
Rocket.Chat+ (Desktop): https://github.com/RocketChat/Rocket.Chat.Electron

[geekgonecrazy]

Yes currently only the "Rocket.Chat" app in play store / Apple store support sandstorm. The "Rocket.Chat+" apps don't yet.

Thanks for posting the links @rebeld !

[yooomark]

@geekgonecrazy So is there a plan to support sandstorm on "Rocket.Chat+"?

[geekgonecrazy]

Rocket.chat+ aka now simply just "Rocket.Chat"

I do not believe it's on the roadmap. I believe the legacy app is the only one that supports it.

Best place to ask would be on the repo. https://GitHub.com/rocketchat/rocket.chat.android or https://GitHub.com/rocketchat/rocket.chat.ios

[boredland]

so currently only the legacy app supports it? this happened so many times for those sandstorm apps that we're most likely to move away from them very soon.

[mgax]

so currently only the legacy app supports it?

Is the legacy app available on iOS any more? I can only find these two apps:

• https://itunes.apple.com/ro/app/rocket-chat/id1148741252
• https://itunes.apple.com/ro/app/rocket-chat-experimental/id1272915472

Both display an error "The URL you entered is invalid" when pasing the webkey URL from sandstorm.

[geekgonecrazy]

The legacy app is actually fully deprecated. We no longer support it and stopped providing updates back a few months ago. I believe it will soon be fully removed from App Store.

You can of course still compile it if you need. But that app won’t work with any release after 0.71.0 so bare that in mind.

Regarding sandstorm... it actually needs more attention to get working again. We have some community effort to make it a community supported distribution again. Feel free to join in and help if you have familiarity with building sandstorm grains