Server: package.json file of transitive dependency seems to be somehow embedded in the built JavaScript files

[danxuliu]

Running make on _master_ changes the built JS files. "_Maybe some change was commited to the source files without rebuilding_" I thought, but no, it is more interesting than that. Keep reading if you want to follow me in this journey of mystery...

Let us start with something reproducible and clone Nextcloud server in a container with the same Docker image used by Drone to check the JavaScript files:

docker run --rm --tty --interactive nextcloudci/node:node-4 bash
git clone https://github.com/nextcloud/server
cd server
git checkout 0e06239ae144227598def41a1c4482d6b12385a6

Now, we build the JavaScript files like Drone does to check them:

npm ci
npm run build

The result?

git status
nothing to commit, working tree clean

So far so good. But when you run make it runs npm install instead of npm ci. What happens in that case? Before doing that we move the _node_modules_ downloaded with npm ci to another directory, we will use them later.

mv node_modules node_modules-with-ci
npm install
npm run build
git status

        modified:   apps/accessibility/js/accessibility.js
        modified:   apps/accessibility/js/accessibility.js.map
        ...

"_So, for some reason, npm install installs something different than npm ci and that is why the built files are different_". If you check you would see that _fsevents_ is installed by npm ci (even on GNU/Linux, although it is for Mac OS only) but not by npm install. "_Ha! There it is!_". Is it? Now watch this trick:

cp node_modules-with-ci/@nextcloud/event-bus/package.json node_modules/@nextcloud/event-bus/package.json
npm run build
git status

Untracked files:
        node_modules-with-ci/

Tadaaa! Modified files are gone! Wait, wait, but there is more. Now I just add a white space to the _package.json_ file and...:

echo " " >> node_modules/@nextcloud/event-bus/package.json
npm run build
git status

        modified:   apps/settings/js/vue-settings-admin-security.js
        modified:   apps/settings/js/vue-settings-admin-security.js.map
        ...

Modified files are back (but just a subset of them)! Cool, isn't it?

Now if someone has a Hogwarts acceptance letter please explain me what is going on, because I do not have the slightest clue :-P

@nextcloud/javascript

[skjnldsv]

we do not understand either
easiest would be to just use npm i in the tests ^^'

[juliushaertl]

I had another look at the actual diff that is produced, it seems that the package.json is integrated into the bundle for @nextcloud/event-bus:

The package.json for a npm dependency is not static and depends on how the packages were installed.

The issue seems to be that ProxyBus imports the package.json:

node_modules/@nextcloud/event-bus/dist/ProxyBus.js:

    3 Object.defineProperty(exports, "__esModule", {
    4   value: true
    5 });
    6 exports.ProxyBus = void 0;
    7 
    8 var _package = _interopRequireDefault(require("../package.json"));

@ChristophWurst I guess we should check if there is a different way to get the version number out of package.json so that is is bundled in directly without depending on the package.json.

[skjnldsv]

@ChristophWurst I guess we should check if there is a different way to get the version number out of package.json so that is is bundled in directly without depending on the package.json.

I do it via webpack :)
https://github.com/nextcloud/contacts/blob/a01c511a44f37a971964ffe2e6669e1e6e64595c/webpack.common.js#L5-L7

[ChristophWurst]

I do it via webpack :)
https://github.com/nextcloud/contacts/blob/a01c511a44f37a971964ffe2e6669e1e6e64595c/webpack.common.js#L5-L7

Thanks. Will give that a try. TBH I assumed the way I did it before you just compile the version number into the even bus bundle. I'm surprised this is loaded dynamically.

But… why does this make a difference for "npm i" vs "npm ci"? Is this really the same issue?

@danxuliu try this for more confusion: run npm i or npm ci and compare the outputs from npm ls. They don't have the same packages. Like, for example davclient.js is missing in the ci one.

[danxuliu]

But… why does this make a difference for "npm i" vs "npm ci"? Is this really the same issue?

Because the main difference between npm install and npm ci is in the _package.json_ files. You can check that following the steps above and, after doing npm install, do diff --recursive --brief node_modules-with-ci node_modules.

try this for more confusion: run npm i or npm ci and compare the outputs from npm ls. They don't have the same packages. Like, for example davclient.js is missing in the ci one.

Even better is that _davclient.js_ actually is in _node_modules_ after npm ci; it is missing only according to npm ls.

[juliushaertl]

https://github.com/nextcloud/nextcloud-event-bus/pull/12

[juliushaertl]

Even better is that davclient.js actually is in node_modules after npm ci; it is missing only according to npm ls.

Seems we are not the only ones experiencing this: https://npm.community/t/npm-ls-does-not-provide-the-same-output-after-npm-ci-as-it-does-for-npm-i-when-there-are-git-dependencies/8758

[ChristophWurst]

Seems we are not the only ones experiencing this: https://npm.community/t/npm-ls-does-not-provide-the-same-output-after-npm-ci-as-it-does-for-npm-i-when-there-are-git-dependencies/8758

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

:man_facepalming:

[danxuliu]

Fixed by https://github.com/nextcloud/nextcloud-event-bus/pull/12 and https://github.com/nextcloud/server/pull/17510