Parse-server: True beginner's migration guide

[bmueller]

A lot of people chose Parse because they had no experience with developing backends. The current migration guide is a great start, but much of it is just an executive summary for people who already know what they're doing (particularly the section "Deploying to Heroku/MongoLab").

What's really needed is a true beginner's guide and best practices that takes you through all the steps of setting up a Heroku account correctly, installing node, setting up mongoDB, migrating everything over, making sure everything's working, etc. Ideally, you'd be able to follow the steps to do a complete migration of a standard Parse app with little to no experience in the platform/languages. (Sort of like the original Parse tutorials for developing your first iOS app with Parse integration.)

Otherwise, a lot of developers will be left out in the cold here, either abandoning their apps completely, spending weeks/months learning new platforms/languages, or spending thousands hiring someone else to do it for them.

[kingmatusevich]

Agreed

[sinosoidal]

+1

[jdmcd]

+1

[nitrag]

+1, has anyone done pricing of moving to Heroku though?? Their DB's aren't cheap if you want to keep your data safe and replicated.

I'm think we (developers) need to start budgeting for minimum $100/month in hosting. Not cheap for us Indie guys that haven't even launched a product yet. Experts chime in but I just need to get back to the 30 req/sec benchmark. Then be able to scale from there.

Let's also not restrict our mindset to Heroku-only and help beginners like me make the transition to the most appropriate solution.

[phyominthu]

+1

[bmueller]

Agreed, @nitrag. Looking over their pricing page, I don't really have any clue what I'd need for a simple app that has ~5k users and is doing less than 1 request per second. Would the hobby tier work for that?

[michael-mansour]

+1 yes a detailed guide explaining how to deploy Parse Server on Windows Azure/AWS/Google App Engine would be great !

[nitrag]

Looks like we'll also need to crowd-open-source a web-based dashboard.
That's is handy for me while I am still developing.

On Fri, Jan 29, 2016 at 1:04 PM, Michaël [email protected] wrote:

+1 yes a detailed guide explaining how to deploy Parse Server on Windows
Azure/AWS/Google App Engine would be great !

—
Reply to this email directly or view it on GitHub
https://github.com/ParsePlatform/parse-server/issues/38#issuecomment-176889908
.

[phyominthu]

@bmueller According to this comment, a 5$/month Digital Ocean VPS could work for you. I have a few small apps and I'm interested in Digital Ocean right now. You may want to checkout its pricing plans too.

[jamesyu]

The current guide is definitely our first stab at explaining the core principles, and I agree we can make better guides for each vendor.

We're thinking of these vendors: AWS, Digital Ocean, Heroku, Azure. Any others you guys would be interested in?

[kingmatusevich]

Linode is interesting as well, and the Digital Ocean guide could easily be worked out to also match linode's similar offerings.

[bmueller]

@jamesyu great to hear, I think a lot of us would be totally lost without something a lot more in-depth for all the newbies who have no experience doing any of this stuff. Really appreciate the effort you're putting into this.

[arxidon]

Perhaps Google App Engine as well

[mattcreager]

Hey Folks, Creager here from Heroku, happy to help prepare a beginner-friendly guide to getting the parse-server up and running on Heroku! In the meantime, we're working on adding a Heroku Button to the parse-server-example!

[meilers]

The cheapest option I see right now is to use Google App Engine + MongoLab

Any one have better ideas?

On Jan 29, 2016, at 1:04 PM, Michaël [email protected] wrote:

+1 yes a detailed guide explaining how to deploy Parse Server on Windows Azure/AWS/Google App Engine would be great !

—
Reply to this email directly or view it on GitHub https://github.com/ParsePlatform/parse-server/issues/38#issuecomment-176889908.

[jamesyu]

Here's one for GAE that someone wrote: https://medium.com/google-cloud/deploying-parse-server-to-google-app-engine-6bc0b7451d50

[nitrag]

@mattcreager @jamesyu and Parse/Facebook staff.

It means a lot to us that you guys are still dedicated to supporting the developers and helping us find the best path forward. Getting us on the right path early (before you move on at FB) will give us the best shot at continuing to improve and grow Parse and the community.

Waht we'd like to see in your guides

• Deploying to the different platforms mentioned above BUT also pro's and con's of each. What struggles/limitations can you help us foresee so we can be prepared for that
• Reliability: How do we achieve the similar level of stability and data reliability we've had with Parse. With whatever platform we choose, what features/services will we need to ensure no downtime for our users.
• Performance benchmarks: What level of server processing power should be considered minimum for supporting X users or X req/s? Can you baseline this since this is now something we will have to manage ourselves?
• Operationally - What to look out for, such as the indexing information already provided. Tips to optimize and keep the platform running healthy/stable.

Again, thank you for you time and support.

[hslightnin]

@meilers https://www.digitalocean.com/pricing/ $5 here gets us close to what Parse was for free.

+1 for a ELI5 guide for multiple providers.

[nitrag]

@hslightnin Yes but, correct me if I'm wrong, you can't scale that if you dump the DB, parse-server, etc on the same server. With digitalocean or AWS wouldn't you need a load balancer, multiple nodejs application servers and a mongoDB replicated to be completely safe? What your talking about is potential major downtime. I was hinting at this in my previous post and waiting for some professionals to chime in.

[hslightnin]

True , but those of us who would never outgrow the free Parse subscription need something that isn’t going to be $400/mo.

[francocorreasosa]

I use DigitalOcean to run my Parse Server and it's great. Not too different to run any other NodeJS and MongoDB app.

[jpv123]

This is a step by step guide I made for the transition. I hope this helps someone. I just want to say that I don't know anything about node.js and yesterday was my first day using it.

Please try this guide with a not production parse app first. It’s recommended to clone your repo and try with this copy so if something goes wrong, you don't loose any data. I’m not responsible for anything that can happen with your data, app, computer or anything.

First Time Only:
Install source tree (you can use git from console if you want to)
Install node.js. This also installs npm.
Install Heroku toolbelt
Open Terminal
npm install express-generator -g //Installs a generator for express (where Parse-server works)
Create a Mongolab account
Create a Heroku account
Create a Bitbucket account and setup source tree with this account (for git with source tree)

Every Time a new parse server is needed
Open Terminal
cd my/directory/where/folder/will/be/with/node.js/
express nameOfMyApp //Creates a template of node.js. MAKE SURE IS ALL LOWERCASE!
cd nameOfMyApp
npm install //Install dependencies
npm install express —save //Install dependencies
npm init
Open Source tree
Select create local repository
Select your folder of nameOfMyApp and create repo
heroku login
enter your credentials of heroku
heroku create
echo ‘web: node app.js' >procfile //Create procfile for Heroku
In source tree, commit and push to Heroku master. Wait for everything to be successful
heroku ps:scale web=1 //Set a node for Heroku
heroku open
Verify it’s working and no error message appears. You should se a Welcome to Express screen
npm install parse-server —save //Installs Parse Server. If you see a warning of kerberos ignore it

For new parse databases
Create in mongolab new deployment
For the new deployment create a user and a password

For migrations from Parse to mongodb
Enter to parse new dashboard
Go to app settings -> General -> Migrate to external database
From mongolab, search for the MongoDB URI
replace dbuser and dbpassword with your user and password you created
Copy the whole url into parse
Wait for the transition to finish. This can take a few minutes to hours. It depends on the amount of data. Once it finishes, go to mongolab and check that all your info is there.

Final steps:
Open app.js. You can do it with the program of your preference
Add var ParseServer = require(‘parse-server').ParseServer; at the top of the file
Add this before 404 call

// Specify the connection string for your mongodb database
// and the location to your Parse cloud code
var api = new ParseServer({
databaseURI: ‘mongodb://yourMongoDBURL',
cloud: 'parse-server/cloud/main.js',
appId: ‘Your App Id',
masterKey: ‘Your Master Key'
});

// Serve the Parse API on the /parse URL prefix
app.use('/parse', api);

var port = process.env.PORT || 1337;
app.listen(port, function() {
console.log('parse-server-example running on port ' + port + '.');
});

Commit and push to heroku

You’re finished!! Now, to use Parse just point your apps to the new Server. You will need at least version 1.12 for iOS, 1.13.0 for Android, 1.6.14 for JS, 1.7.0 for .NET

[Rhadammanthis]

@francocorreasosa Could you provide some sort of guide about how you achieve that? I had never used Node JS nor Mongo before and I'm having lots of troubles with it. I'd be forever in your debt.

[francocorreasosa]

@Rhadammanthis You'll need to install nginx and do a reverse proxy to the port 5000. Then install nodejs and npm. Next, create a project folder and cd it, do a npm init and npm install pm2 express parse-server --save. Then create a server.js file and paste this:

var express = require('express');
var ParseServer = require('parse-server').ParseServer;

var app = express();

// Specify the connection string for your mongodb database
// and the location to your Parse cloud code
var api = new ParseServer({
  databaseURI: 'mongodb://localhost:27017/dev',
  cloud: '/home/myApp/cloud/main.js', // Provide an absolute path
  appId: 'myAppId',
  masterKey: 'mySecretMasterKey',
  fileKey: 'optionalFileKey'
});

// Serve the Parse API on the /parse URL prefix
app.use('/parse', api);

// Hello world
app.get('/', function(req, res) {
  res.status(200).send('Express is running here.');
});

var port = process.env.PORT || 1337;
app.listen(port, function() {
  console.log('parse-server-example running on port ' + port + '.');
});

Then do pm2 start server.js -i 2

You're done

[meilers]

@francocorreasosa you did all that on digitalocean? via terminal? And are you hosting your mongo DB on digitalocean as well?

[francocorreasosa]

@meilers Yes, in my case I am hosting the DB on the same droplet but if you want you can host the mongo instance in another DB and connect it within the DigitalOcean's Private Network.

[docherty]

My 2¢ : There is a group of users who were drawn to Parse because it was so accessible/easy to use. Everything was in one place and it solved a lot of potential headaches. I am one of those users. Since the news of the shut down broke, I've been looking at the opportunity to switch to something like a Docker container on AWS. However, this just feels like an opportunity to fail in a production environment (I know nothing about professional security, adequate redundancy planning etc.). I've hacked together a working server but I'm sure professionals would laugh/shake their head at my incompetence.

So, I think it would be really helpful to people like me if we could put together a suggested "architecture" for purely managed services where all the security/uptime issues are taken care of by 3rd parties. I haven't found a direct replacement for the breadth of services Parse offered but a combination of 3-4 (?) might work well for a lot of people. I'm happy to contribute my findings/approach but thought the experts on here might be able to provide objective guidance. Off the top of my head, I'm thinking something like this (not done the research yet): Firebase (replaces Core Data)+Heroku??? (Cloud code)+Flurry (analytics)+UrbanAirship (push notifications)...you get the idea.

@gfosco have you got any suggestions where this kind of content could be put/discussed?

BTW something I noticed which isn't obvious in a comment above: the "Migrate to external database" button is only available in the "beta" admin web interface.

[poseidonsw]

I got mine up and running on digital ocean in a couple of hours as well, minus the login. Had an issue with my SB when it migrated I think, but all up and running now.

[dcdspace]

@docherty I agree it would be helpful to have a list of suggestions on how to rebuild our stack. Would we need to use an external service like Firebase for Core or could we survive simply on Parse Server running on Heroku?

Does anyone have any thoughts on OneSignal for Push? It is completely free. Also I wouldn't use Flurry as they are part of Yahoo, and I've learned my lesson from Facebook with Parse.

[abremner]

I've contacted Digital Ocean to request they add Parse Server to their list of One-Click Applications. That, with MongoDB, would be a big help for those of us without that in-depth backend experience.

Edit: Please vote for this request https://digitalocean.uservoice.com/forums/136585-digitalocean/suggestions/11629764-parse-com-as-a-1-click-app

[mkll]

_Just rewrote a @jpv123's comment above for better readability:_

This is a step by step guide I made for the transition. I hope this helps someone. I just want to say that I don't know anything about node.js and yesterday was my first day using it.

Please try this guide with a not production parse app first. It’s recommended to clone your repo and try with this copy so if something goes wrong, you don't loose any data. I’m not responsible for anything that can happen with your data, app, computer or anything.

First Time Only

1. Install source tree (you can use git from console if you want to)
2. Install node.js. This also installs npm.
3. Install Heroku toolbelt
4. Open Terminal
5. npm install express-generator -g //Installs a generator for express (where Parse-server works)
6. Create a Mongolab account
7. Create a Heroku account
8. Create a Bitbucket account and setup source tree with this account (for git with source tree)

Every Time a new parse server is needed

1. Open Terminal
2. cd my/directory/where/folder/will/be/with/node.js/
3. express nameOfMyApp //Creates a template of node.js. MAKE SURE IS ALL LOWERCASE!
4. cd nameOfMyApp
5. npm install //Install dependencies
6. npm install express —save //Install dependencies
7. npm init
8. Open Source tree
9. Select create local repository
10. Select your folder of nameOfMyApp and create repo
11. heroku login
12. enter your credentials of heroku
13. heroku create
14. echo ‘web: node app.js' >procfile //Create procfile for Heroku
15. In source tree, commit and push to Heroku master. Wait for everything to be successful
16. heroku ps:scale web=1 //Set a node for Heroku
17. heroku open
18. Verify it’s working and no error message appears. You should se a Welcome to Express screen
19. npm install parse-server —save //Installs Parse Server. If you see a warning of kerberos ignore it.

For new parse databases

1. Create in mongolab new deployment
2. For the new deployment create a user and a password

For migrations from Parse to mongodb

1. Enter to parse new dashboard
2. Go to app settings -> General -> Migrate to external database
3. From mongolab, search for the MongoDB URI
4. replace dbuser and dbpassword with your user and password you created
5. Copy the whole url into parse
6. Wait for the transition to finish. This can take a few minutes to hours. It depends on the amount of data. Once it finishes, go to mongolab and check that all your info is there.

Final steps

Open app.js. You can do it with the program of your preference
Add var ParseServer = require(‘parse-server').ParseServer; at the top of the file
Add this before 404 call

// Specify the connection string for your mongodb database
// and the location to your Parse cloud code
var api = new ParseServer({
databaseURI: ‘mongodb://yourMongoDBURL',
cloud: 'parse-server/cloud/main.js',
appId: ‘Your App Id',
masterKey: ‘Your Master Key'
});

// Serve the Parse API on the /parse URL prefix
app.use('/parse', api);

var port = process.env.PORT || 1337;
app.listen(port, function() {
console.log('parse-server-example running on port ' + port + '.');
});

Commit and push to heroku

You’re finished!! Now, to use Parse just point your apps to the new Server. You will need at least version 1.12 for iOS, 1.13.0 for Android, 1.6.14 for JS, 1.7.0 for .NET

[docherty]

@dcdspace Re Firebase: The reason I'm looking at it (rather than mongoDB) is that I want to replicate the functionality of Parse.User.current() in a hybrid (JS SDK) app. I use this feature to sync data between the device and the server really easily. The Firebase SDK has a similar feature (I need to do more testing but that's what I remember from a while ago). If I just wanted a datastore I'd probably use a managed mongoDB service like mongoLab.

Re. Flurry: I know what you mean but ultimately businesses are typically owned by someone else - Heroku is Salesforce. So while I agree it's a bummer, any service could disappear/change. In the case of Parse/Facebook you have to agree that they've done a pretty good job of trying to help users transition (we have a year, they're paying guys to support the community, they've open sourced their code etc.) I doubt that would have happened if it was a small independent company and the money ran out. Flurry is owned by Yahoo but it's an integral part of their ad business.

I've created a repo here: https://github.com/docherty/parse-migration-to-managed-services with my notes which I'll keep updating and eventually turn into a tutorial for the few people that want to piece together a "managed" replacement for Parse.com .

[niraj-shah]

If it helps, I've created a tutorial on installing Parse API Server on AWS. I have also released an Amazon Machine Image (AMI) with everything already installed, just needs configuration. See: http://bit.ly/parse-server

[dcdspace]

Thanks @docherty , that is an awesome repo and will be very useful to me and others I'm sure.

[hslightnin]

know this would be a long shot but any chance you could package this up like backendless?
Standalone Backendless

[JcMinarro]

Here is a HowTo Step by Step to create a Parse Server on your own Server.
https://medium.com/@jcminarro/run-parse-server-on-your-own-server-using-digitalocean-b2a7d66e1205#.e33bkf4m3

[natario1]

@jamesyu also GAE is missing. The tutorial from medium is not enough in my opinion, totally ignores migration. A comprehensive guide should also consider holding MongoDB in a dedicated GCE cluster, see https://cloud.google.com/launcher/solution/click-to-deploy-images/mongodb?q=mongodb . That can cost less that the 180$ for MongoLab and is just as easy.

For example - just to name one of the things that a typical parse user is not understanding - when configuring the above mentioned cluster, I don't understand if I should have disk space for both data (Parse's Data Storage) and images pointed from that data (Parse's File Storage), or if the latter should go in my GAE Storage space. The migration tool is going to point to the mongo in GCE.

Another thing that is very very unclear is how ParseFiles are managed by the migration tool. That's missing in your guide.

So my suggestions are:

• GAE + GCE
• Better ParseFile understanding. How are these managed by the migration tool?
• Better storage understanding, mostly related to files pointed from ParseFile columns. Where will they be hosted? When assigning storage, should we 10x the data storage or both data and file storage for those files?
• Deeper info on "_You need to manage your indexes yourself. Hosted Parse automatically adds indexes based on the incoming query stream._"
• More generally, please take into account that most of your users do not know (but now are forced to) anything about backends. Some phrases could be re-worded in my opinion.
• Consider writing in your guide that push and dashboard are probably coming.

[Kinzi]

Another great guide for digital ocean:
http://julienrenaux.fr/2016/01/29/complete-parse-server-migration-guide/

[arcopo]

I wrote up a more comprehensive tutorial than the one parse give, and actually realised it's really easy to deploy your own parse server to heroku and mongo lab [https://medium.com/@timothy_whiting/setting-up-your-own-parse-server-568ee921333a#.s09l7a5vh]

[meilers]

@mkll thank you very much for that. Do you think it's cheaper to setup a mongoDB directly in mongoLab and then just add its URI in app.js instead of using Heroku's mongoLab add-on?

[dcdspace]

Super helpful tutorial @arcopo thanks!

[mkll]

@meilers, your thanks should be addressed to @jpv123, I just rewrote his comment. :)
As for me, I still collect the info and didn't do anything to migrate from Parse.

[DrJid]

This is a very nice beginner friendly write up by Timothy Whiting. Even if you've never used Heroku, Mongodb or node.js - https://medium.com/@timothy_whiting/setting-up-your-own-parse-server-568ee921333a#.dfndszn7w

[brennen]

I posted a guide on running Parse Server on a DigitalOcean Ubuntu Droplet (or really any Ubuntu system) earlier this morning. It's definitely not a full migration guide, but does cover installing Node and running the example code in parse-server-example.

Parse is a new topic to me personally, so I've got some learning to do, but we'd love to see a beginner-friendly migration guide supported within the scope of the open source project that includes our platform (well, and by logical extension generic Linux VPS provider installations). I can likely devote some energy to helping with that.

[markuswinkler]

First kudos to the parse.com developer team to open source their server. That's an incredible move on their end!
Second, it would be really helpful as mentioned above to get some real world numbers for what service can handle what load and how to set it up for scaling (possibly in the millions of users).
What's the advantage of AWS vs. Heroku vs. GAE vs. Digital Ocean?
I am leaning towards AWS because they just have the biggest server farm there is (imho), but all the guides above are for Heroku or Digital Ocean, is AWS too hardcore to setup?

[niraj-shah]

@markuswinkler actually, the guide I posted was for AWS setup on Ubuntu 14 (http://bit.ly/parse-server), with a pre-built AMI you can easily launch without having to setup the software yourself.

[markuswinkler]

fantastic! Very much appreciated.
I am not too familiar with AWS. How would I scale from there if one server is not enough? I don't have any frame of reference how much a single server can handle or process.

[thewizster]

Great discussion and info here... Much appreciated everyone!

[auggernaut]

Any word on whether Parse is going to open source the dashboard? Browsing/editing data in a GUI was a huge benefit to us when prototyping. It's gonna be a drag to build something like it from scratch...

[hslightnin]

Looks like the first company has started.

[JcMinarro]

@auggernaut If you migrate your app from Parse to your own server, Parse still shows you data from your own server, and you are able to see/edit data. I think that they are not going to close it (I don't remeber where I read it)
@gfosco @drew-gross @wangmengyan95 Do you know anything about it?

[natario1]

@JcMinarro they are going to close it, but they are also probably open sourcing that as well.

[drew-gross]

The parse.com dashboard will still be able to view and edit the data on your self-hosted database, however that won't be true after January 28, 2017. We are working on open sourcing our dashboard so that it can be used with parse-server.

[JcMinarro]

@drew-gross Thank you for clearfy it ;)

[Taylorsuk]

Thanks for all the info in this thread very helpful and is putting me at ease slightly.
I was probably in peaceful ignorance to what the parse service was build on, however my question is: Can a single digital ocean droplet offer me the same reliability? Can they be mirrored, load balanced etc to guarantee 9.9999% uptime?
A 'gold standard' Digital Ocean setup including security, data view and backup strategy would be my ideal.
If I can the knowledge I would love to write it - however I'm completely reliant on you kind folks atm. Thanks for all your hard work!

[drew-gross]

You would have to look into the SLA of Digital Ocean for uptime. Load balancing will up to you after you move to parse-server, and different providers will have different levels of automation in load balancing.

[chrsolr]

1+ for a open source dashboard.

[asContagious]

A dashboard would be great, but also the ability to run multiple apps from one server instance.

[DoDSoftware]

If you want to use Heroku, create an account then go to https://devcenter.heroku.com/articles/deploying-a-parse-server-to-heroku and click "Deploy to Heroku", copy the master key and app id from your parse app and you're set. Had me going in under 3 mins after spending hours trying to follow tutorials online

[ua24]

I succeded use DelightedD0D and DigitalOcean with Ubuntu on localhost virtual machine guides.

[DoDSoftware]

For anyone having trouble getting going with Heroku, I posted the method I used here:http://stackoverflow.com/questions/35389389/how-can-i-host-my-own-parse-server-on-heroku-using-mongodb/35389397#35389397

[estebantcs]

i found this tutorial is very easy to follow https://learnappmaking.com/how-to-migrate-parse-app-parse-server-heroku-mongolab/

[brennen]

One more entry from us at DigitalOcean: How To Migrate a Parse App to Parse Server on Ubuntu 14.04. Quite a bit more detailed than my first go at the problem.

[Taylorsuk]

I have yet to see something that I am been secure in moving to. I have not seen anything with regard to failover, no-one seems concerned about this - are we that trusting of DO, AWS and Heroku for our uptime.

Using Mongolab is expensive - Parse recommend that 10x size is required. Therefore the MongoLab sandbox plan is fine for those who have <50Mb parse export?

There is little talk about the cloud code and files. Where are they stored and how are they managed?

[mohmagdy]

i have got a begginner Guide from one udemy courses :+1:
The guide is available now at:

http://www.robpercival.co.uk/parse-server-on-heroku/

[gfosco]

Now that there's a bevy of content online, different migration and deployment tutorials, I feel like we can close this general issue. My favorite is the two Digital Ocean guides, they are super thorough, and the migration guide really covers setting up a production-level instance. Excellent work by the community here!

[Sadmansamee]

Dokku Parse Server with pm2?
How can we use dokku parse server with pm2 on digital ocean or else? I mean what will be configuration process?

[mohmagdy]

we @ http://appgain.io have used PM2 with parse server to achive multi parse server instances
we are opening our parse hosting service soon to private beta , if you are interested plz let me know

[DoDSoftware]

@mohshaheen If you want people to be interested in your services, you should list more information on your landing page. As it stands the page is quite barren with just an input to subscribe and nothing even explaining what we would be subscribing to. Honestly, it looks very unprofessional, not at all like a service I would trust my production apps to. I get that its a beta and don't get me wrong, I appreciate what you're trying to do there, it's just that the presentation and information could use some work.