Parse-server: Cloud Jobs Guidance

[anthonysg]

Hello, from what I understand, cloud jobs are not working in parse-server. It states the following in the migration docs:

Jobs

There is no Job functionality in Parse Server. If you have scheduled jobs, port them over to a self-hosted solution using a wide variety of open source job queue projects. A popular one is kue. Alternatively, if your jobs are simple, you could use a cron job.

I am wondering if a tutorial could be made showing how to modify already-made cloud jobs to run using kue. Either that or it would be really nice if somebody could implement this functionality.

[gfosco]

I agree a guide would be handy. We've been leaning towards telling developers to build node scripts for this and trigger them with a cron scheduler... but yeah, an example would be great.

[grassland-curing-cfa]

My Cloud code (main.js) contains a Job function:
Parse.Cloud.job("jobRequestForValidation", function(request, status) {
... ...});

This caused failure of deployment of my app on Heroku.

So I had to remove the Job from the main.js so that deployment was successful.

[gfosco]

Going to close this for lack of activity. Guidance for cloud jobs will likely vary based on the deployment platform of choice (i.e. the Scheduler addon for heroku, or some cron script for a VPS).

[AmbroiseCollon]

Hi, I'm setting up a tutorial on Kue + Heroku. In order to finish it, I need help on this issue #1078. Thanks and I hope you'll like it ^^!

[mdtempalert]

I use the following for simple jobs:

if (!Parse.Cloud.job) {
    Parse.Cloud.job = function(name, func) { 
        app.get("/jobs/" + name, func);
    };
} 

Where app is my express instance in the same scope. This function exposes jobs as GET endpoints from my server. For instance, a job called MyJob could be run by hitting /jobs/MyJob.

[rendragon83]

Does anyone know if there are any news on any tutorials for this. I keep hitting rode blocks trying to set this up.

[gateway]

I have to kick this as well, I have seen some info on jobs but no wiki or docs on the best way or best practices for this..

[flovilmart]

Parse.Cloud.Job is exposed.

Every job will be exposed through: /jobs/:jobName and protected by the masterKey access.

Job status is recorded in the _JobStatus whenever the jobs starts, ends, error etc... as the old jobs would do.

Jobs are not scheduled, you should handle scheduling calling the job HTTP endpoint yourself (with the masterKey).

You should be able to run the jobs from the parse-dashboard to test your endpoints and see the status of your jobs.

[gateway]

@flovilmart thanks for your reply, is their some sort of priority we can define for jobs and whats the expected time out that a job might see. I can early have another server do the scheduling via cron and using CURL to run the job. How would be pull _JobStatus back to see the progress, completion etc?

[flovilmart]

There is no priority as there is no schedule, the Job is just the same thing a a cloud function, the only difference is that it's status can be updated for success, error and messages.

You can read the job status in the dashboard or make a query on it "_JobStatus".

[noder199]

Jobs are not scheduled, you should handle scheduling calling the job HTTP endpoint yourself (with the masterKey).

So how should we do this if we want the job(s) to be scheduled? (Open to anyone's answer) I am using google app engine, and I understand there is some sort of scheduler but it is not cron-based. Would that work?

[flovilmart]

You can use app engine cron.yaml To schedule the calls to the jobs end points.

[gateway]

I should of asked, but do jobs time out after x minutes?

[afgarcia86]

This was pretty difficult for me to figure out so I am just going to post a full url example here in hopes it saves someone else time.

Basically I used Postman to Post to a url like http://localhost:1337/parse/jobs/:jobName while having the following headers set. X-Parse-Application-Id and X-Parse-Master-Key.

[flovilmart]

@gateway they don't
@afgarcia86 that is how to run them, use the same method in a scheduler if you need to.

[chadpav]

FYI, I've successfully used a web cron service as an external scheduler so that I didn't have to maintain an additional bit of infrastructure.

https://www.setcronjob.com

P.S. - I have no affiliation w/ this service. It's just the one I happened to use and it worked for me.

[devKC]

@flovilmart - for AWS worker tier, cron.yaml is to be formatted as:

version: 1
cron:

• name: "backup-job"
  url: "/backup"
  schedule: "0 */12 * * *"

Where would I include the parse app id and masterkey? Would it look like below?

url: "/backup"
headers: {
X-Parse-Application-Id: appId
X-Parse-Master-Key: masterKey
}
schedule: "0 */12 * * *"

Any guidance would be greatly appreciated!

[flovilmart]

On Google cloud, I'm using something similar with app engine CRON. I have deployed a very simple JavaScript web app that listen on /job/:jobName and forwards the HTTP call to parse server. This way, I use app engine cron for scheduling as I can't pass the headers either.

You should probably look for something similar on AWS.

[devKC]

@flovilmart - just checking, when you are forwarding the HTTP call to parse server, are you passing along the app_id/master_key values in headers by using something like headers option for require('request')? If so, how secure is it to send the request with keys in the headers? I would assume I wouldn't hardcode the key values in the headers options but use env var, but since the values are being sent in the post request I was wondering if it is recommended to protect the values, i.e. encryption, etc.

[flovilmart]

I make HTTPS requests from the 'worker' so yes I pass my masterKey in the header.

[acinader]

@devKC AWS worker tier? huh? you got a link to what you are talking about? What I've been doing for scheduled jobs is to set an event in AWS Cloud Watch which invokes a lambda that then sends the request. Its problematic in a few ways: 1. its way too complicated, every time i have to touch it, i need to re-wrap my head around it. 2. it doesn't guarantee that the event will only fire once, so I have to code around that, so if there is some AWS worker tier thing...I need to read up on it....

[devKC]

@acinader - http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-tiers.html

[AmbroiseCollon]

I finally went on using agenda which is based on MongoDB and therefore plays nicely with ParseServer. After trying Kue, node package crontab, standard cron, agenda ended up being the best option for me.

[danielchangsoojones]

If anyone is using Parse Server and Heroku, I just used Heroku Scheduler to schedule jobs. It bypasses the need to create a Cloud Job. It's very quick and easy. Here's a tutorial.

[flovilmart]

We implemented a job runner, https://github.com/ampme/Parse-server-job-runner , it just needs to run periodically and will work with the jobSchedule objects (re-introduced in 2.5.0 and next dashboard release)

[aliasad106]

@flovilmart How will it be possible to use it with Cron ?

[flovilmart]

You should ask on the parse-server-job-runner repository