Gensim: CHANGELOG.md vs PyPI?

[gojomo]

PyPI shows gensim releases "3.8.2" and "3.8.3". But these aren't mentioned in the CHANGELOG.md, nor were they announced on the project discussion list. (There was a tweet announcing 3.8.3.) Should those steps – especially, updating CHANGELOG.md – be part of a release checklist, or maybe even some sort of automated-pre-release check?

[piskvorky]

Yeah that's weird. Both are missing. CHANGELOG is definitely an (important) part of our release steps: https://github.com/RaRe-Technologies/gensim/wiki/Developer-page#making-a-new-release

The step is manual though, not automated, so maybe @mpenkov skipped it / forgot.

[menshikh-iv]

I guess reason was in release from separate branch (not develop), here changelog are OK https://github.com/RaRe-Technologies/gensim/blob/3.8.3/CHANGELOG.md

[piskvorky]

Interesting. But that's pretty weird too – what else (other changes) are missing from develop?

[mpenkov]

From https://github.com/RaRe-Technologies/gensim/issues/2820:

Diff with current develop HEAD

The only thing that's missing is last-minute changes to get Py2.7 working on 3.8.3. We deliberately chose not to merge them into develop, because then we'd have more work to do when removing Py2.7 for real in the very immediate future.

[mpenkov]

Should those steps – especially, updating CHANGELOG.md – be part of a release checklist, or maybe even some sort of automated-pre-release check?

They are part of our checklist here.

[piskvorky]

They are part of our checklist here.

That's the same link as above :)

Diff with current develop HEAD

That link attempts to open a new PR. Here are the files that changed; there's more than just CHANGELOG:

$ git diff --name-only develop release-3.8.3
.gitattributes
.travis.yml
CHANGELOG.md
appveyor.yml
azure-pipelines.yml
continuous_integration/appveyor/install.ps1
continuous_integration/appveyor/requirements.txt
continuous_integration/appveyor/run_with_env.cmd
docs/src/conf.py
gensim/__init__.py
gensim/corpora/dictionary.py
gensim/corpora/sharded_corpus.py
gensim/models/doc2vec.py
gensim/models/fasttext.py
gensim/models/hdpmodel.py
gensim/similarities/nmslib.py
gensim/test/test_corpora_dictionary.py
gensim/test/test_fasttext.py
gensim/test/test_utils_any2vec.py
setup.py
tox.ini

EDIT: Looks mostly safe but we have to be careful not to leave behind relevant changes.

[mpenkov]

What action do we need to take? I can think of several options:

1. Do nothing
2. Merge the changelog only onto develop, so more people know about the 3.8.3 release (because develop is the default branch that people see when they come to github).
3. Merge the entire 3.8.3 branch onto develop, and then immediately undo all changes other than the change log
4. Any other options?

I'm OK with 1), but it does not address the original issue raised by @gojomo. I'm also OK with 2), but it's a bit of a half-measure. I don't like 3) because it's silly.

[piskvorky]

Option 1 (incomplete CHANGELOG) is not an option.

Ideally we'd have two branches that differ only in the py2 changes, so we can go back and forth easily in the future. Option 3 is the closest and cleanest.
OTOH, we don't care about py2, so that's more theoretical – in all likelihood unnecessary work.

So that leaves option 2. Only I'm not 100% sure CHANGELOG is the only difference. For example, how about the changes to test_utils_any2vec.py or tox.ini?

[mpenkov]

Ideally we'd have two branches that differ only in the py2 changes, so we can go back and forth easily in the future.

I'd recommend against doing that - we'd have to deal with the ongoing pain of maintaining Py2.7.

The changes to tox.ini were to make Py27 happy. One line enables Py27 builds, another one disables an optional pytest extension that refused to work with Py27.

The same thing goes for test_utils_any2vec.py. Py3 accepts 'bw' as a mode, Py2.7 does not.

If you're happy with the above explanation, let me know, and I will copy the changelog across.

[piskvorky]

Sure – as long as all relevant changes are in develop, that's it. Thanks.