Parse-server: Deploy LiveQuery on google app engine

[ranhsd]

Hi ,
My parse-server app is hosted on google app engine. Since my app needs real time capabilities i decided to go with ParseLiveQuery. I managed to install ParseLiveQuery on a different server (i use redisURL for this) and i wanted to know if it is possible to have the following setup:

1. parse-server runs on google app engine (works!)
2. redis run on google compute engine (works!)
3. ParseLiveQuery deployed as a separate app to google app engine (not working..)

On my localhost everything run as expeceted without any issues. I get responses from both Parse REST api and ParseLiveQuery Websocket but when i tried it on google app engine the ParseLiveQuery is not sending me events.
I read here about it and found that in order to support websockets on Google app engine you need to do some steps . I tried to do it but it's not working.

I wanted to know if someone tried to do it before or if someone can provide some hints about it.

Thanks,
Ran.

[flovilmart]

We'll be deploying soon live query to AppEngine. I'll let you know my findings.

[ranhsd]

Hi @flovilmart thanks! Really hope to hear some good news.. but meantime can you provide some hints or thoughts because i need it asap and if i will managed to do it i will also share it here.

Thanks,
Ran.

[flovilmart]

I added a CLI for live Query server in 2.2.22, in the mean time, you can run the live Query server on Compute Engine, just point the live query client to that URL.

[ranhsd]

Hi @flovilmart , on compute engine i managed to run it but if i want to autoscale it i think it's better to use the app engine. I use the latest CLI for that and it works as expected so currently the only issue that i am facing is with the google app engine.
Meanwhile i will do it on the compute engine. If you managed to run it on the app engine it will be great...
So what i have currently is: parse-server deployed on the app engine, Redis deployed on compute engine and now i need another machine to deploy the live query server. If i will now create multiple live query server machines and will connect them to the same Redis server how the load balancing will work? it will do it sequentially or by load or by number of connections? (or maybe something else?)

Thanks!

[flovilmart]

Not sure how the lod balancing work on compute engine, I would guess you have multiples options like round robin etc

[flovilmart]

@ranhsd also, did you connect directly to the instance with the provided IP/name you get with the reverse lookup?

[EricNetsch]

@ranhsd I am about to try this solution and deploy it to app engine today. I'll post my findings here.

@flovilmart The example posted here pulls the external IP address for websocket use. You can see the suggested integration here

// In order to use websockets on App Engine, you need to connect directly to
// application instance using the instance's public external IP. This IP can
// be obtained from the metadata server.
var METADATA_NETWORK_INTERFACE_URL = 'http://metadata/computeMetadata/v1/' +
    '/instance/network-interfaces/0/access-configs/0/external-ip';

[ranhsd]

Hi @EricNetsch - it will be great thank you ! I just wondering ... If we will be able to deploy LiveQuery to app engine it should autoscale the live query server automatically i assume because currently in order to scale it i use Redis with multiple compute engines so do you think that if the live query server will be deployed to GAE it will replace my current setup with Redis and multiple compute engines?

Thanks.
Ran.

[flovilmart]

On my side, we'll no longer use app engine soon. We're moving to container engine in the coming days

[ranhsd]

@flovilmart about container engine... can you please share how you are planning to use the container engine? are you creating the image on your own or take a predefined image like this one? and regarding to live query are you planning to create a dedicated instance on google container engine for it or to use the same one and just increase the nodes (for scaling to solution..)

thanks in advanced!

Ran.

[flovilmart]

We'll probably write an article about it once everything is correctly deployed. For now, we're still at the experimental phase in terms of the deployments

[ranhsd]

@flovilmart - Thanks .. looking forward for it.

[flovilmart]

@ranhsd I have awesome new for app engine!

there are a few steps to get it to work correctly, if you're not using the parse-server CLI that should be easy:

Add that to your index.js

we need app engine to run on a different port...

let liveQueryPort = 65080; //Any thing you want
var liveQueryServer = express().listen(liveQueryPort, function() {
  console.log('ParseLiveQuery listening on '+ liveQueryPort);
});
ParseServer.createLiveQueryServer(liveQueryServer);

Add a small CloudCode endpoint:

// Cloud Code entry point
var METADATA_NETWORK_INTERFACE_URL = 'http://metadata/computeMetadata/v1/' +
    '/instance/network-interfaces/0/access-configs/0/external-ip';
var request = require('request');

function getExternalIp () {
  var options = {
    url: METADATA_NETWORK_INTERFACE_URL,
    headers: {
      'Metadata-Flavor': 'Google'
    }
  };
  return new Promise(function(resolve, reject) {
     request(options, function (err, resp, body) {
        if (err || resp.statusCode !== 200) {
          console.error('Error while talking to metadata server, assuming localhost');
          return reject('error fetching');
        }
        return resolve(body)
      });
  });
}

Parse.Cloud.define("getWebsocketEndpoint", function(req, res) {
  getExternalIp().then(function(body) {
    res.success(body);
  }, function(err) {
    res.error(err);
  });
});

That should return the IP address of your server

Forward the port from the app.yaml

Change the port to the one you selected first

network:
  forwarded_ports:
  - 65080
  instance_tag: websocket

Add firewall rule:

gcloud compute firewall-rules create default-allow-websockets \
  --allow tcp:65080 \           
  --target-tags websocket \                           
  --description "Allow websocket traffic on port 65080"

Deploy your app!

in your client:

js:

Parse.Cloud.run("getWebsocketEndpoint").then((ip) => {
  let liveQueryURL = `http://${ip}:65080/parse`;
})

swift:

let client: Client
if let res = (try? PFCloud.callFunction("getWebsocketEndpoint", withParameters: nil)) as? String {
    client = ParseLiveQuery.Client(server: "http://\(res):65080/parse")
} else {
    client = ParseLiveQuery.Client.shared
}

[otymartin]

@flovilmart I'm curious why you are leaving app engine for container engine? Anything I should know as a user of Parse-server with App Engine?

[flovilmart]

Not really, that's just because we plan to use container engine more, but that may change back to in the future as well.

[flovilmart]

@ranhsd closing that one as the steps are all described here. That may be a good opportunity for a blog post or other for the steps to deploy (or even put it in the Wiki when I have some time)

[ranhsd]

Hi @flovilmart thanks a lot ! i will try it our and will let you know soon as everything will work. Meantime i started to work on a different deployment option with GKE (google container engine) In GKE i use kubernetes which allows me to auto scale , monitor and manage the whole cluster in a very nice way (via docker containers). I didn't try to deploy the live query servers yet but i believe that it will not be an issue. Currently the only thing that i am facing with is the ability to expose the parse-server (and in the future live-query) services in HTTPS for that i plan to use letsencrypt with nginx-ssl-proxy for auto generate the certificates for me every 3 months but since it's not so simple i think that it will take me some more time. Anyway, if i will managed to do it i will let you know and maybe we can add it to the parse-server wiki page. For now i go with GAE but in the future i think it's better to go with GKE and kubernetes in order to create more flexible solution (which leverage the microservices architecture)

Thanks again!

[otymartin]

@flovilmart final question does LiveQueryServer scale along with app engine or is extra work needed to help it scale?

[flovilmart]

Depends on your setup but it should correctly scale as long as you're using redis or another pub/sub (coming soon) to propagate the events