Freecodecamp: Seeding database fails

[ojongerius]

If I try to seed a database it fails. How to reproduce:

• docker-compose down # make sure none of the related containers are running
• docker system prune -a # remove all stopped containers, images and networks not in use
• docker-compose run --rm freecodecamp npm install# install deps
• docker-compose run --rm freecodecamp npm run only-once # seed

Of interest is that entities not found, db may not be properly seeded is ironic when you try to seed a db 😉

Related but not identical #17256 -which shows a timeout connecting to the db, while my setup connects to the db just fine (fcc:server db connected +0ms).

▶ docker-compose run --rm freecodecamp npm run only-once
Starting freecodecamp_mailhog_1 ... done
Starting freecodecamp_db_1      ... done

> [email protected] only-once /app
> npm run prelint-js && echo '/****/' && echo 'Seeding Database' && echo '/****/' && node seed/index.js && echo '/****/' && echo 'Seeding Completed' && echo '/****/'


> [email protected] prelint-js /app
> npm run ensure-env


> [email protected] ensure-env /app
> node ./config/ensure-env.js



rev-manifest present




pathMigration present


/****/
Seeding Database
/****/
Warning: Accessing createClass via the main React package is deprecated, and will be removed in React v16.0. Use a plain JavaScript class instead. If you're not yet ready to migrate, create-react-class v15.* is available on npm as a temporary, drop-in replacement. For more info see https://fb.me/react-create-class
Warning: Accessing PropTypes via the main React package is deprecated, and will be removed in  React v16.0. Use the latest available v15.* prop-types package from npm instead. For info on usage, compatibility, migration and more, see https://fb.me/prop-types-docs
  fcc:server db connected +0ms
/app/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/utils.js:123
    process.nextTick(function() { throw err; });
                                  ^

Error: Error: entities not found, db may not be properly seeded
    at AnonymousObserver._onError (/app/server/boot/home.js:18:21)
    at AnonymousObserver.Rx.AnonymousObserver.AnonymousObserver.error (/app/node_modules/rx/dist/rx.js:1836:12)
    at AnonymousObserver.Rx.internals.AbstractObserver.AbstractObserver.onError (/app/node_modules/rx/dist/rx.js:1772:14)
    at AnonymousObserver.o.onError (/app/node_modules/rx/dist/rx.js:1999:22)
    at AnonymousObserver.tryCatcher (/app/node_modules/rx/dist/rx.js:63:31)
    at AutoDetachObserverPrototype.error (/app/node_modules/rx/dist/rx.js:5891:52)
    at AutoDetachObserver.Rx.internals.AbstractObserver.AbstractObserver.onError (/app/node_modules/rx/dist/rx.js:1772:14)
    at InnerObserver.error (/app/node_modules/rx/dist/rx.js:4530:14)
    at InnerObserver.Rx.internals.AbstractObserver.AbstractObserver.onError (/app/node_modules/rx/dist/rx.js:1772:14)
    at InnerObserver.o.onError (/app/node_modules/rx/dist/rx.js:1999:22)
    at InnerObserver.tryCatcher (/app/node_modules/rx/dist/rx.js:63:31)
    at AutoDetachObserverPrototype.error (/app/node_modules/rx/dist/rx.js:5891:52)
    at AutoDetachObserver.Rx.internals.AbstractObserver.AbstractObserver.onError (/app/node_modules/rx/dist/rx.js:1772:14)
    at AutoDetachObserver.o.onError (/app/node_modules/rx/dist/rx.js:1999:22)
    at AutoDetachObserver.o.onError (/app/node_modules/rx/dist/rx.js:1999:22)
    at /app/node_modules/rx/dist/rx.js:1904:72
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] only-once: `npm run prelint-js && echo '/****/' && echo 'Seeding Database' && echo '/****/' && node seed/index.js && echo '/****/' && echo 'Seeding Completed' && echo '/****/'`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] only-once script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2018-05-29T00_05_10_316Z-debug.log

[ojongerius]

@rajatkantinandi ^^

[ptsurbeleu]

I looked into the runtime state and it looks like #17150 could be the reason why seeding doesn't work anymore. Here is the execution paused in boot/home.js right after runtime enters server/boot/home.js:

module.exports = function(app, done) {
  const { About } = app.models;
  const router = app.loopback.Router();
  let challengeCount = 0;
  debugger
...

and after I let it go with continue (c) command, the runtime throws an error while trying to access entities in the db:

According to git blame changes from #17150 could be related to the issue with seeding, since its code introduces call to entities while the server is being bootstrapped:

[ptsurbeleu]

@Bouncey, can you take a look pls whether #17150 is the reason why seeding not working anymore? ^^

[Bouncey]

Ohhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh. Now I know what is broken, thank you for your investigation @ptsurbeleu!!

The models are imported from the app during seed to destroy/create blocks and challenges. This has a side-effect of spinning up the server, which in its current state, requires challenge info whilst bootstrapping.

I will see about patching this today

[ojongerius]

A hacky workaround for the moment is:

Edit home.js and comment out line 12 until 24.

You can this manually or if you have patch installed you could do this automatically by creating a file named home.patch with the following contents:

--- server/boot/home.js.orig    2018-06-01 11:01:27.000000000 +1200
+++ server/boot/home.js 2018-06-01 11:01:39.000000000 +1200
@@ -9,6 +9,7 @@
   const { About } = app.models;
   const router = app.loopback.Router();
   let challengeCount = 0;
+  /*
   cachedMap(app.models)
     .do(({ entities: { challenge } }) => {
       challengeCount = Object.keys(challenge).length;
@@ -22,6 +23,7 @@
         done();
       }
     );
+  */

   function addDefaultImage(req, res, next) {
     if (!req.user || req.user.picture) {

And then run patch like so:

▶ patch -p0 < home.patch
patching file server/boot/home.js

However you changed home.js, your next step is to run the seeding by either:

• Docker: docker-compose run --rm freecodecamp npm run only-once
• Other: npm run only-once

When seeding is done, restore home.js by doing git checkout server/boot/home.js and start freeCodecamp as you normally would, ie:

• Docker: docker-compose up
• Other: npm start

Contributions for a permanent fix are welcome!

[rajatkantinandi]

@ojongerius
Thanks.
Your solution works fine.
Just after saving the file & seeding the db we have to restore the file to original version otherwise the address localhost:3000 shows "Can't get /"
I think this step should be mentioned on contributing.md for now until there's a permanent fix.

[korydondzila]

I can work on this.

[ojongerius]

🎉

[Nirajn2311]

I'm sorry to say this, but it didn't work for me until I added stripe APIs to my .env file.

[ojongerius]

@Nirajn2311 that's a different issue though. The root cause of this issue has been fixed in #17472