P5.js: issues with npm run release

[lmccart]

When running npm run release I am noticing a few problems:

• The p5.js-release repo files seem to be disorganized. It looks like there is a combo of older files and new ones, placed inside a lib/lib folder.
• npm version bumps up, but the lib submitted trails by one version. for example npm is at 1.1.2 but the files inside are at 1.1.1
• The latest package.json file with the updated version gets committed but not pushed (maybe this is expected behavior?)
• No new release tag is created

[limzykenneth]

I'll have a look, seems unusual to me. If you can post the terminal output here it could be helpful.

[limzykenneth]

• The p5.js-release repo files seem to be disorganized. It looks like there is a combo of older files and new ones, placed inside a lib/lib folder.
• npm version bumps up, but the lib submitted trails by one version. for example npm is at 1.1.2 but the files inside are at 1.1.1

I've managed to fix the two above. From my testing (a bit incomplete as I don't want to push directly to this repo) the later two points should work, np should be pushing to github after releasing on NPM. I see that no package is released on NPM so maybe that's why it didn't create the release? In any case, the terminal output could be helpful, do let me know if there are other problems.

For the bower release repo, the copy steps are fixed but the release repo itself needs to be cleaned up manually.

[limzykenneth]

The new releases seems to work, except the CI here is still failing and the website also didn't build sucessfully. For CI here, could it be that this step didn't finish in time before the library starts building?

Also on the website the library's version is correct but for some reason data.json's version is one version behind. I think for now a manual fix on the website should bring everything back on track for the time being.

[lmccart]

Thanks @limzykenneth! Yeah, still trying to track down the different issues. Experimenting with patch releases which is kind of spammy but I just want to make sure we get everything worked out smoothly.

I was able to get it to build by including the parameterData.json in the repo, but I'm not sure that is the correct fix. As you mentioned above, it should be generating as part of the build process, so it could be a timing issue.

The website is failing with a lint issue, which I'm also trying to fix. Once I get these two resolved, I'll look again at the whole build sequence and see which steps are still missing or out of order.

[lmccart]

Update: I think everything is resolved now except the reference/data.json file trailing one version number.

[akshay-99]

Hi. From what I understand there's an issue due the build of parameterData.json which may have caused all this trouble and I am just so so sorry about that. I did not realise that it may cause issues with the release because for some reason the CI seemed to pass everything until today.

I looked at the line number mentioned by @limzykenneth and for now, I think changing the use of this to something synchronous might fix the issue. I don't know if it's also linked to the version number in data.json but I will see it and get back soon.

Somehow it did not give me all these issues during my test release but I realise that might just be due to me having a local pre-built copy of parameterData.json.

[akshay-99]

@lmccart @limzykenneth

So here's what I know so far:

1. parameterData.json is built whenever the reference documentation is built.
2. grunt build only builds the library, not the documentation, but needs parameterData.json to run successfully.

This meant that if grunt build is called before building the documentation, it would fail. This didn't cause the problem in all the previous runs of the CI because the CI runs npm ci ( install dependencies ) followed by npm test.
npm test builds the documentation before building the library and so everything would go smoothly.

But now due to the prepublish step, grunt build runs right after npm ci or npm install ( as described here ). Since the documentation build never happened before this, grunt build fails.

The new releases seems to work, except the CI here is still failing and the website also didn't build sucessfully. For CI here, could it be that this step didn't finish in time before the library starts building?

This line never ran as the documentation build didn't happen yet before the crash.

Including parameterData.json in the git repo is a fix because now it can rely on a pre-existing copy. But I really doubt whether it should be included. It's a huge auto-generated file and small changes to inline docs can cause large changes to parameterData.json.

Maybe we can re-configure the prepublish step to also build the docs before building the library. I think this could also solve the data.json issue as this file is also built alongside the reference documentation.

[lmccart]

@akshay-99 thanks for looking into this! I think what you've written sounds correct. I agree it would be best not to have paramaterData.json in the repo. Would you be up for submitting a PR that makes the proposed change?

[limzykenneth]

Didn't know prepublish runs for install as well (and it seems it's a common confusion point), maybe it should be prepublishOnly instead.

[limzykenneth]

For data.json did you notice this step run? It should generate data.json with the latest version number that will then be copied to the website.

[neefrehman]

Hi all! I hadn't read through the contents of this issue before opening https://github.com/processing/p5.js/issues/4708.

Mine seems to to be a side-effect of the current release issue fix.

[akshay-99]

For data.json did you notice this step run? It should generate data.json with the latest version number that will then be copied to the website.

Hmm. That's weird. I am doing another namespaced release, starting with everything clean to see what happens. I will update soon.

Hi all! I hadn't read through the contents of this issue before opening #4708.
Mine seems to to be a side-effect of the current release issue fix.

@neefrehman yeah it seems so. I will take a look

[akshay-99]

Okay. So I may have an idea what's up with the data.json thing. I am not sure but this is what I think as of now. Grunt first runs on npm run release and in the entire release process, it's the same instance of grunt running ( except for the prepublish command, which initializes a separate one ). The package version is captured at the very beginning (before np can bump it up). So when this step runs, even though np has bumped the version number, this instance of grunt won't know that.

The reason the p5.js and p5.min.js files have the correct version number is probably due to the fact that the prepublish command spawns up a different instance of grunt.

[limzykenneth]

When running locally the version number of data.json is updated correctly for me though

[akshay-99]

@limzykenneth

I released a namespaced version https://www.npmjs.com/package/@akshay-99/p5-test.
The version there is 1.1.11.
But my local data.json copy has

{
    "project": {
        "name": "@akshay-99/p5-test",
        "description": "[![npm version](https://badge.fury.io/js/p5.svg)](https://www.npmjs.com/package/p5)",
        "version": "1.1.10",
        "url": "https://github.com/processing/p5.js#readme"
    },

same on my fork of the p5-website repo here.

It would be weird if the local version doesn't match what's pushed. Can you please let me know what command you ran?

[limzykenneth]

Might be my problem as it seems your explanation make sense, the version number pass to yuidoc comes from this line which will be populated at the beginning of the script and not updated when npm bump the version.

A fix will be to grab the version synchronously on this line with fs.readFileSync, additionally since the yuidoc config is called as a function, it can fetch a fresh version number everytime it is called (at least I assume that's how grunt does it).

[akshay-99]

What I did was add this task to Gruntfile, grunt.registerTask('prerelease', ['yui', 'build']); and then configure prepublishOnly to grunt prerelease. It seems to work and I get the correct verson (1.1.12) both on https://www.npmjs.com/package/@akshay-99/p5-test as well as here

I removed the grunt.task.run('yui'); line from release-p5.js as it would have instead overwritten the correct data.json.

Is this method okay?

[limzykenneth]

That should work as well.