Parse-server: Parse-server on Heroku web and worker dyno

[adammlevy]

Issue Description

I am currently running ParseServer on Heroku where I have 1 web dyno for handling web traffic and 1 worker dyno for handling jobs. I have set up 5 cron jobs on the worker dyno which handle fetching JSON files and storing them in a db I have hosted on mLab. While watching my server logs I noticed that although the job starts on the worker node (as seen by log statements), anytime I make a call to Parse.* this is happening on the web dyno. Given both the web and worker dyno have their own instance of ParseServier running, shouldn't this prevent my cron job from hitting the web node?

Steps to reproduce

/// Procfile ///
web: node --optimize_for_size --max_old_space_size=960 --gc_interval=100 index.js
worker: node --optimize_for_size --max_old_space_size=960 --gc_interval=100 worker.js

/// index.js ///
var api = new ParseServer({
  databaseURI: ...
  appId: ...
}
...
var app = express();
...
var port = process.env.PORT || 1337;
var httpServer = require('http').createServer(app);
httpServer.listen(port, function() {
    console.log('parse-server-example running on port ' + port + '.');
});

/// worker.js ///
// same server config as index.js with additional code to set up cron jobs

// functions called by cron jobs
function fetchSomething() {
     var query = new Parse.Query(ParseObject);
                          // this query happens on the web node, I expect it to be called on the worker
                  query.find().then ....

}

Expected Results

Everything that happens on the worker node stays on the worker node, worker node never offloads work to the web node

Actual Outcome

All calls to Parse.* happen on the web node.

Environment Setup

• Server
  
  
  
  • parse-server version : 2.6.3
  
  
  • Operating System: N/A
  
  
  • Hardware: N/A
  
  
  • Localhost or remote server?: Heroku 1 web, 1 worker
  
  
• Database
  
  
  
  • MongoDB version: 3.2.13
  
  
  • Storage engine: N/A
  
  
  • Hardware: N/A
  
  
  • Localhost or remote server? : mLab
    
    

    Logs/Trace

  
  

Notice that the job starts on the worker node, push happens on the web node, and finally finishes on the worker node.

2017-10-17T05:11:40.966975+00:00 app[worker.1]: will push
2017-10-17T05:11:41.191817+00:00 app[web.1]: verbose: REQUEST for [POST] /parse/push: {
2017-10-17T05:11:41.191828+00:00 app[web.1]:   "where": {
2017-10-17T05:11:41.191829+00:00 app[web.1]:     "deviceType": "ios",
2017-10-17T05:11:41.191852+00:00 app[web.1]: } method=POST, url=/parse/push,
2017-10-17T05:11:41.252067+00:00 app[worker.1]:  push success
...

[steven-supersolid]

What are you passing to your constructor for serverURL? Should be something like 'http://localhost:' + PORT + '/parse'. If you pass the herokuapp url instead then all internal parse requests will hit the heroku load balancer and go to the web dyno instead of the worker

[adammlevy]

@steven-supersolid that was it! Thank you so much! Now I no longer see my jobs hitting the web dyno. Although I have it working now, can you foresee any problems I my experience running parse on a web dyno and worker dyno?

[steven-supersolid]

I do something similar and it's been fine for us. Our jobs must run in a single process so cannot use node cluster, this might cause issues if the parse request took a long time and another job was fired at the same time, but as everything is non-blocking I think it's fine.

You might want to experiment with deliberately hitting the web dyno so you that don't have to run a separate instance of parse-server. That would probably save a lot of memory on your job dyno so could potentially reduce the dyno class. In our case I wanted the job to be completely independent in case the web dynos became overloaded so did not pursue this further.

BTW if you are using node-schedule to schedule your jobs then please be aware of this bug as it caused concurrent executions which are a problem for us https://github.com/node-schedule/node-schedule/issues/370

[natanrolnik]

Closing it as the issue as solved. Thanks @steven-supersolid.