Rescript-compiler: Allow build to be re-entrant

[alexfedoseev]

After installing new dependency via yarn, bsb binary is not available.

❯ yarn install --save some-dep
❯ $(yarn bin)/bsb -clean-world -make-world
Error: spawnSync node_modules/bs-platform/lib/bsb.exe ENOENT

[fakenickels]

Also having that sometimes, the solution has been deleting node_modules and run yarn install again

[jsiebern]

Running this once or twice also "fixes" it:
yarn add --force bs-platform

Sometimes an error is thrown on the first time, sometimes it's not

[jsiebern]

Btw, it seems that this happens every 2nd add. On the first it says rebuilding bs-platform and it works fine. When I add another package after that it breaks.

[bobzhang]

Note this seems to be fixed in the master, will cut a beta release for testing

[bobzhang]

Published beta 4.0.15, tests are welcome

[MarcelCutts]

I previously ran into this issue and no longer do so in 4.0.15

Environment

Mojave + [email protected]

Methodology

1. bsb -init newpackbug -theme react
2. Check base project is functional yarn && yarn start, no issues
3. Add a dependency yarn add bs-jest --dev
4. Add bs-jest to bs-dev-dependencies
5. yarn clean && yarn start, no issues
6. Repeat above for reason-apollo and graphql_ppx packages

The above skeleton project setup process previously resulted in this bug, and now it does not.

[bobzhang]

Thanks @MarcelCutts

[jsiebern]

Late to the party but can also confirm that it works with the beta!

[alexfedoseev]

@bobzhang Hmmm I still see this w/ 4.0.15: https://cl.ly/99cdce1ee984

[bobzhang]

@alexfedoseev we published beta, 4.0.17, hopefully it have your issues addressed

[MoOx]

I just update to 4.0.18, then installed some new deps, and got it again. Using --force helped.

[bobzhang]

Do you have a detailed log.
Also would be nice to have NodeJS version, yarn version

[dfalling]

Hopefully my issue's the same one. This happens for all of my deploys on Heroku. I have to destroy my Heroku cache before deploys or they'll fail.
../jscomp/stubs/ext_basic_hash_stubs.c:1:10: fatal error: caml/hash.h: No such file or directory

remote: -----> Restoring cache        
remote:        - node_modules        
remote:                
remote: -----> Building dependencies        
remote:        Installing node modules (yarn.lock)        
remote:        yarn install v1.14.0        
remote:        [1/4] Resolving packages...        
remote:        [2/4] Fetching packages...        
remote:        info [email protected]: The platform "linux" is incompatible with this module.        
remote:        info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.        
remote:        [3/4] Linking dependencies...        
remote:        warning " > [email protected]" has unmet peer dependency "prop-types@^15.5.4".        
remote:        warning " > [email protected]" has incorrect peer dependency "reason-react@^0.3.0 || ^0.2.0".        
remote:        [4/4] Building fresh packages...        
remote:        error /tmp/build_b932ee2eb6f4fb44b03770ba7b8e87c5/node_modules/bs-platform: Command failed.        
remote:        Exit code: 1        
remote:        Command: node scripts/install.js        
remote:        Arguments:         
remote:        Directory: /tmp/build_b932ee2eb6f4fb44b03770ba7b8e87c5/node_modules/bs-platform        
remote:        Output:        
remote:        ninja binary is already cached and installed:  /tmp/build_b932ee2eb6f4fb44b03770ba7b8e87c5/node_modules/bs-platform/lib/ninja.exe        
remote:        No working prebuilt buckleScript compiler        
remote:        The OCaml compiler, version 4.02.3+BS        
remote:        Standard library directory: /tmp/build_aba5af2e96d2124210414c0d07be0758/node_modules/bs-platform/vendor/ocaml/lib/ocaml        
remote:                
remote:        Use the compiler above        
remote:        [1/6] Making bsb_helper.exe        
remote:        ../jscomp/stubs/ext_basic_hash_stubs.c:1:10: fatal error: caml/hash.h: No such file or directory        
remote:         #include "caml/hash.h"        
remote:                  ^~~~~~~~~~~~~        
remote:        compilation terminated.        
remote:        [2/6] Making bsppx.exe        
remote:        ../jscomp/stubs/ext_basic_hash_stubs.c:1:10: fatal error: caml/hash.h: No such file or directory        
remote:         #include "caml/hash.h"        
remote:                  ^~~~~~~~~~~~~        
remote:        compilation terminated.        
remote:        [3/6] Making bsb.exe        
remote:        ../jscomp/stubs/ext_basic_hash_stubs.c:1:10: fatal error: caml/hash.h: No such file or directory        
remote:         #include "caml/hash.h"        
remote:                  ^~~~~~~~~~~~~        
remote:        compilation terminated.        
remote:        child_process.js:637        
remote:            throw err;        
remote:            ^        
remote:                
remote:        Error: Command failed: /tmp/build_b932ee2eb6f4fb44b03770ba7b8e87c5/node_modules/bs-platform/lib/ninja.exe        
remote:            at checkExecSyncError (child_process.js:616:11)        
remote:            at Object.execFileSync (child_process.js:634:13)        
remote:            at provideCompiler (/tmp/build_b932ee2eb6f4fb44b03770ba7b8e87c5/node_modules/bs-platform/scripts/install.js:258:12)        
remote:            at Object.<anonymous> (/tmp/build_b932ee2eb6f4fb44b03770ba7b8e87c5/node_modules/bs-platform/scripts/install.js:268:5)        
remote:            at Module._compile (internal/modules/cjs/loader.js:689:30)        
remote:            at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)        
remote:            at Module.load (internal/modules/cjs/loader.js:599:32)        
remote:            at tryModuleLoad (internal/modules/cjs/loader.js:538:12)        
remote:            at Function.Module._load (internal/modules/cjs/loader.js:530:3)        
remote:            at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)        
remote:        info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.   

[bobzhang]

@dfalling do you have similar issues with regard to 5.0.0-dev.4? it would be nice that I can have a reliable way to reproduce it

[dfalling]

This was happening so consistently for me I added heroku repo:purge_cache -a ido-ui to my deploy step so my builds would consistently pass. I'll drop that workaround and switch to 5.0.0-dev.4 to see if it's still an issue.

[dfalling]

Two builds with my hack removed and so far so good. I'll report back if any builds start failing again, but this is already better than it's been for a while.

[jsiebern]

It was gone for a while, now it happens to me sometimes with 5.0.0-dev.4. But I can't really reproduce it consistently either unfortunately.

[bobzhang]

@jsiebern would you paste your log here when it is broken, thanks. Do you have any intuition what's yarn doing here

[MoOx]

@bobzhang just got it again, this time with 5.0.0. What I did: I just changed a dep hash (reason-react from git...), run yarn (which ran correctly...?)

❯ yarn
yarn install v1.15.2
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies..
...(uncessary warning for react native & expo deps that you don't care)
[4/4] 🔨  Building fresh packages...
success Saved lockfile.

After that, I just tried bsb -make-world and boom bsb.exe was gone! Crazy stuff.
I opened node_modules/bs-platform, everything is here... except bsb.exe 🤯

Can this be related to the fact that this project is in a monorepo covered by yarn workspaces & the fact that yarn workspace do some "tricks" to only put necessary modules in the "local package node_modules folder" (in this monorepo (bs-react-native) some other folder use bs-platform 4)?
Part of this tricks involve (if I am correct) doing some links for node_modules/.bin, to be able to link to local bin or bin from root folder (which tries to get most node_modules, to avoid redundancy - which is the point of yarn workspaces...)

[bobzhang]

I guess yarn will remove some files. I really would like to fix such issue, but I need a solid way to reproduce it, thanks!

[webern]

Same problem here. Same solution rm -rf node_modules && yarn.

macOS

yarn --version
1.15.2

node --version
v11.13.0

which node
/usr/local/bin/node

node_modules/.bin/bsb -version
5.0.2

[MoOx]

According to a recent experience with a reason module that's actually a node binary (cli tool) that is being built on postinstall - to generate some .bs.js files (because dependencies etc), I faced issue where built artifact (.bs.js) where removed... By yarn? I suspect yarn integrity feature to clean files that aren't supposed to be there & to not rerun postinstall hooks. Sometimes even when using --force...
That may be a reason for this mess.

[MoOx]

@bobzhang does this explanation make you think about something that could be improved/avoided?

https://github.com/yarnpkg/yarn/issues/5680#issuecomment-381775304

[MoOx]

Han, I just saw this https://github.com/BuckleScript/bucklescript/issues/2799 lol

[lensbart]

I get the same error when yarn adding a dev dependency and adding it to bs-dependencies in bsconfig.json:

yarn re:watch
yarn run v1.9.4
$ bsb -make-world -clean-world -w
Error: spawnSync /Users/bart/Documents/GitHub.nosync/codebase/node_modules/bs-platform/lib/bsb.exe ENOENT
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Reinstalling bs-platform via the above suggested yarn add -D --force bs-platform eventually did the trick.

• Node v12.10.0
• Yarn 1.9.4

[bobzhang]

finally fixed once for all, no postbuild needed for next release