Semanticmediawiki: 3.0.0 release

[JeroenDeDauw]

Tracking issue for the 3.0 release.

• Release notes
• Open issues for 3.0
• 3.0 issues already closed
• Changes since 2.5
• Contributors since 2.5 https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/2694#issuecomment-333473589

Breaking changes TODO list:

• https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1014 (includes PHP version bump https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/698)

rc.1 target: 2018-09-21, done: 2018-09-23

• [x] Shape rel notes
  
  
  
  • [x] Add missing commits tagged with releasenotes
  
  
• [x] Update INSTALL.md - not just version number
• [x] Update composer.json
• [x] Update version number
  
  
  
  • [x] extension.json
  
  
  • [x] COMPATIBILITY.md
  
  
• [x] Create tag
• [x] Update semantic-mediawiki.org
  
  
  
  • [x] Run composer update
  
  
  • [x] Touch LocalSettings.php
  
  
  • [x] Update the SMW configuration
  
  
  • [x] Check Special:Version
  
  
• [x] Ping testers group
• [x] Announce via mail
• [x] Do a release tweet on Twitter

rc.2 target: 2018-10-02, done: 2018-10-02

• [x] Shape rel notes
  
  
  
  • [x] Add missing commits tagged with releasenotes
  
  
• [x] Update version number
  
  
  
  • [x] extension.json
  
  
• [x] Create tag
• [x] Update semantic-mediawiki.org
  
  
  
  • [x] Run composer update
  
  
  • [x] Touch LocalSettings.php
  
  
  • [x] Update the SMW configuration
  
  
  • [x] Check Special:Version
  
  
• [x] Ping testers group
• [x] Announce via mail
• [x] Do a release tweet on Twitter

Release target: 2018-10-11, done: 2018-10-11

• [x] Finalize rel notes (include contributors (git shortlog -sn 2.5.0...3.0.0))
  
  
  
  • [x] Add missing stuff tagged with releasenotes
  
  
  • [x] Check open issues
  
  
• [x] Create a migration guide for admins (e.g. list format rework https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/3130, etc.) / https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki_3.0.0/Migration_guide
• [x] Update version number (3.0.0)
  
  
  
  • [x] extension.json
  
  
  • [x] README.md in /
  
  
  • [x] README.md in /docs/
  
  
  • [x] COMPATIBILITY.md (release date etc.)
  
  
  • [x] INSTALL.md
  
  
• [x] Create branch 3.0.x
• [x] Create tag 3.0.0
• [x] Update "branch-alias" in composer.json
• [x] Update https://github.com/SemanticMediaWiki/IndividualFileRelease
• [x] Update semantic-mediawiki.org
  
  
  
  • [x] Run composer update
  
  
  • [x] Touch LocalSettings.php
  
  
  • [x] Update the SMW configuration
  
  
  • [x] Check Special:Version
  
  
  • [x] Update release template
  
  
• [x] Start development of new version
  
  
  
  • [x] Update "composer.json" (update dev alias)
  
  
  • [x] Update "extension.json" (new alpha version)
  
  
• [x] Announce on wiki
• [x] Add release to Wikidata
• [x] Update version on en, mw and wikiapiary
• [x] Announce via mail
• [x] Do a release tweet on Twitter
• [x] In case no 2.5.9 release is done revert back the version info on the 2.5.x branch

Note
Due to https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/2859 we have to do minor releases for SBL, SC, SESP and SIL. SRF should traditionally be done anyways.

[JeroenDeDauw]

@mwjames I suggest that we switch to 3.x-dev after releasing 2.4.0. MediaWiki is dropping PHP 5.3 and 5.4 support, so it's certainly time we do the same. Thoughts?

[mwjames]

so it's certainly time we do the same. Thoughts?

Thoughts added in #1366 to keep this issue bound to specific release management tasks.

[JeroenDeDauw]

I suggest we drop 5.5 support by the time we get to this release (which I doubt will be before halfway through 2017, at which point 5.5 will have been unsupported by the PHP team for over a year). See also https://seld.be/notes/php-versions-stats-2016-2-edition

[kghbln]

I suggest we drop 5.5 support by the time we get to this release

No objections. Still the wiki reality is a bit different from what seld.be is experiencing. Continuing stats from here: https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/698

2016-11-18

7.1    1     0,06%
7.0  119     7,26%
5.6  425    25,93%
5.5  279    17,02%
5.4  376    22,94%
5.3  404    24,65%
5.2   36     2,20%
    1639

[JeroenDeDauw]

It's time to once again shout: SUCH A DROP PHP 5.5 SUPPORT

New stats? :)

[kghbln]

New stats? :)

All PHP versions:

7.1    8    0,47%
7.0  149    8,78%
5.6  519    30,57%
5.5  256    15,08%
5.4  351    20,67%
5.3  380    22,38%
5.2   35    2,06%
    1698    

PHP versions currently supported by SMW:

7.1   8 0,86%
7.0 149 15,99%
5.6 519 55,69%
5.5 256 27,47%
    932 

I guess those using PHP 5.5 are still on Ubuntu 14.04 LTS. So two thirds of the people are already on 5.6+

[mwjames]

So two thirds of the people are already on 5.6+

We should make an early email announcement that 3.0 will require:

• PHP 5.6 (expected to receive security updates until 31 Dec 2018 according to [0])
• MW 1.27 (June 2019 as EOL according to [1])

[0] https://secure.php.net/supported-versions.php
[1] https://www.mediawiki.org/wiki/Version_lifecycle

[kghbln]

We should make an early email announcement that 3.0 will require: ...

I could do this if @JeroenDeDauw has no objections and/or additional thoughts.

[JeroenDeDauw]

I'd make such an announcement closer to the release, assuming we follow the pattern here and 3.0 is still at least half a year off. Not sure what we gain by announcing this now. It's in the compat docs already https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/docs/COMPATIBILITY.md#platform-compatibility.

[JeroenDeDauw]

No strong objections from my side though

[kghbln]

Let's do it in August then.

[JeroenDeDauw]

Hmm... SMW still supports 5.5. Breaking that just before the release for the sake of it seems a waste. How about bumping it right after? That is assuming that the 3.0 release will be in the near future. If it's still half a year off, I suggest bumping the number ASAP.

[kghbln]

SMW 3.0 will be PHP 5.6+ no?

[JeroenDeDauw]

Well that was the idea, but it has been half a year since we started 3.x-dev, and it still supports 5.5

[mwjames]

That is assuming that the 3.0 release will be in the near future. If it's still half a year off

Changes happen with a certain pace, and the number of people working on changes is directly proportional to the time required on making that release.

From my side, I'm not in a hurry, if users want to see an earlier release then there is 3.0 task list of what needs to be done before considering a code freeze.

Well that was the idea, but it has been half a year since we started 3.x-dev, and it still supports 5.5

I'm not sure I get the point here? Is this about the CI environment testing on PHP 5.5? Does it really matter that we are still testing on 5.5 as long as tests pass?

Do you have a specific feature in mind that isn't supported in PHP 5.5 but you want to use? What do we gain from switching to PHP 5.6 now on the 3.x-dev and since we won't go full steam on PHP 7.x, using declare(strict_types=1) is out of the question, or did I miss something?

[JeroenDeDauw]

Just to be clear: I'm not saying things are going to slow or that you should hurry.

I'm not sure I get the point here? Is this about the CI environment testing on PHP 5.5? Does it really matter that we are still testing on 5.5 as long as tests pass?

It means you are not getting any of the 5.6 benefits and constantly need to keep compat with an older version of PHP. In this case the difference might not be huge, but it is there.

Variadic stuff is nice https://secure.php.net/manual/en/migration56.new-features.php

[mwjames]

Just so there is no misunderstanding and users starts dropping in on Q1/2018 and asking about a possible release. Issues that read [Task 3.0] ... are required to happen with release 3.0 which means that if there are not resolved then there will be no 3.0 release as long as there are open.

3.0 is a release where things can break therefore my insistence to do so and not excuse ourselves with "dammit we are in Q1... we need to release" argument or wait another 10 years to clean up stuff from a decade ago.

[mwjames]

Variadic stuff is nice https://secure.php.net/manual/en/migration56.new-features.php

This doesn't really heat my plate, for strict typing (7.+) yes, but this is just syntactic sugar.

[mwjames]

Special:Ask

Special:Browse

Special:SemanticMediaWiki

In-page factbox

Input assistance

[JeroenDeDauw]

@mwjames We got 15 months worth of awesome new features and bug fixes all unavailable to most users, so I'd say it's high time to do a release. (Which might be unsurprising since I'm for releasing basically weekly.) Is there anything that really should be included in 3.0? I checked https://github.com/SemanticMediaWiki/SemanticMediaWiki/milestone/6 and figured most of the things there can wait. Cool new features can just as well go in 3.1 or 3.2, which can be released shortly (much less than 15 months) after 3.0. Breaking changes are a different matter, as releasing 4.0 a month after 3.0 would be a bit silly if it can easily be avoided.

I have some time to help with the release process in the coming weeks.

[kghbln]

I also believe that we should release sooner than later, so no more big new features is perfectly fine with me. I think the biggest bunch still to be done is https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1014, https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/2965, https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/3074, https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/3101 and https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/3124 - not to mention my personal backlog of terror: https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/2762.

I have some time to help with the release process in the coming weeks.

I think this will be utterly cool. Perhaps you could have a peep at some of the issues at SRF. SRF is currently only getting emergency fixes which is kinda sad but understandable at the same time.

[JeroenDeDauw]

To me none of those look like blockers. To me blockers are regressions compared to the previous release or missing upgrading instructions. I think it makes sense to look at the breaking changes #1014 since else they would be put on hold for even longer. Seems like everything else can go in a 3.1 release.

[mwjames]

I'll do a quick post to avoid coming back next week and see the repo
in shatters and things getting out of hands for a possible 3.0 release
in Q3.

To me none of those look like blockers.

We are in disagreement here and besides I have a list of some changes
(20+ some old, some new) that are scheduled to be part of 3.0 either
because those extend existing features or change some architectural
base concepts that were added during the 3.0 development. I'm also not
going for the argument of adding tests later (we don't want to repeat
the same mistakes that took place between SMW 0.1 and 1.9 with no
tests which then had to be added in retrospect to make the software
thereafter not falling apart.)

Well, the timeline has shifted from Q1 to now Q3 (tentatively) but
that was due to circumstances that were unplanned for such as the
introduction of the ElasticStore. We always have encouraged people
to send PRs but so far that didn't happen (except Stephan) which by my
account means no rush.

(Which might be unsurprising since I'm for releasing basically weekly.)

I have responded to that concept in the past which I believe is unfit
for the SMW repo given its various sources that require updates as
well as labour necessary to get the job done.

Cool new features can just as well go in 3.1 or 3.2, which can be released shortly (much less than 15 months) after 3.0.
Seems like everything else can go in a 3.1 release.

Making a release always takes a toll and I'm not really in for making
another one (ie. 3.1) before next year.

Breaking changes are a different matter, as releasing 4.0 a month after 3.0 would be a bit silly if it can easily be avoided.

3.0 is big release and I'd like to take the chance to deprecated some
stuff that has been in the source forever and there are some
architectural decisions I'd like to revisit before I consider calling
a 3.0 freeze.

As for support of MW 1.31, I think I made a clear statement on that
matter of "reproducible test builds" but let me repeat when tests fail
for either MySQL, SQLSite, and Postgres on that platform then we are
surely not able to call support for it.

PS: I don't like to be rushed and I'm even less participative on doing
things by halves.

Cheers

On 7/6/18, Jeroen De Dauw [e-mail] wrote:

To me none of those look like blockers. To me blockers are regressions
compared to the previous release or missing upgrading instructions. I think
it makes sense to look at the breaking changes #1014 since else they would
be put on hold for even longer. Seems like everything else can go in a 3.1
release.

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1365#issuecomment-402989595

[JeroenDeDauw]

Hey James, I definitely don't want you to feel rushed. And I think we want the same thing: a good experience for the users, and maintainable software.

To me it just looks like the approach you are taking has a big hole in it: the users are not getting all the nice new improvements and need to wait two years to get any new features. The enemy of the good is the perfect.

If we already added x value compared to last release and still want to add y more value, we can first deliver x and then y, rather than waiting for x+y. If that is all that is going on, it is easy to see how that is better for the users.

There is the complicating factor of painful releases that need lots of work. I definitely see how these are currently painful. The reason for this seems twofold. If things are done half way and need another half doing before release, you end up with a pile of work you don't want to do. If those things where done a year ago or more, it gets even more painful, since then you need to figure out what happened all the way back then.

Looks at how many releases of the Maps extension I made since the last SMW release. Now imagine all those improvements where still sitting on master, being of no use to any users. And the thing is that all those releases combined cost me way less effort than what I expect we need for 3.0.

A software development practice that I find very important is having a releasable master branch. That means the code on it is tested. That it is not broken. That it is documented. And that no only half finished features are exposed without some config change. Does that not both sound like something desirable and something achievable?

I think 3.0 should be done soonish, to deliver value to the users, to gather feedback, and to enable switching to less painful releases.

[mwjames]

As outlined in [0, 1], 2018-09-17 is marked as code freeze therefore any release planning can happen hereafter, so knock yourself out.

PS: Beginning with 10.2018 I will be absent til next year, I may respond to emergencies but that is not guaranteed and depends on availability.

[0] https://sourceforge.net/p/semediawiki/mailman/message/36414343/
[1] https://github.com/SemanticMediaWiki/SemanticMediaWiki/milestone/6

[kghbln]

any release planning can happen hereafter, so knock yourself out.

That's the moment I have always dreaded. I am up and dunno when ... Apart form that having features rather than documented features is ok with the community which relieves a lot of pressure.

[kghbln]

I am planning on doing the 3.0.0-rc.1 on 2018-09-20

[JeroenDeDauw]

This two-year huge big-bang release approach is not working for me as I outlined before. I have decided to not volunteer my time helping with the fallout this time and will not be doing so until there is a better process.

[kghbln]

This two-year huge big-bang release approach is not working for me as I outlined before. I have decided to not volunteer my time helping with the fallout this time and will not be doing so until there is a better process.

I understand that this long release process was exceptional for reasons explained along the way at various spots. At least this is the impression I got. You probably have an even better understanding of the work that was done. Looking back all other big releases were done every four to eight months which I believe is a good schedule both from a developer and a user perspective. This time span is a good compromise between the two existing extremely radical approaches we are seeing here.

[kghbln]

I will not be able to do the release candidate today. I will do this either this weekend or during the next week.

[jamesmontalvo3]

Anything I can do to help?

[kghbln]

Anything I can do to help?

Actually yes. You could probably add the missing descriptions to the items down in the RELEASE-NOTES or even describe some of the numerous new features at the top. This has not to be fully polished but somehow readable.

Today several factors basically broke my neck which were unpredictable at the beginning of the week. :(

[jamesmontalvo3]

So I'm clear: you're saying add descriptions to these:

[kghbln]

So I'm clear: you're saying add descriptions to these:

Yes, this would already help a lot.

[jamesmontalvo3]

Okay. I'm on it. Anything else you can think of?

[mwjames]

So I'm clear: you're saying add descriptions to these:

... yes and you might read through them (I only added some quick notes
while keeping the PR list fairly updated otherwise that would have
been a nightmare by now to finalize the list) and propose some changes
where the description is unclear or misleading.

If your unsure about the content (or context) then just ask I'll be
around during the weekend since I have to make some minor adjustments
I found during the last week.

Cheers

On 9/21/18, James Montalvo [e-mail] wrote:

So I'm clear: you're saying add descriptions to these:

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1365#issuecomment-423559192

[kghbln]

Okay. I'm on it. Anything else you can think of?

Not, when it directly gets down to tagging the "rc.1". This would already be doing a bunch of good.

Apart from than the biggest issue is that most of the docu is missing on the wiki. I guess updating the info on the "list" format including https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/3135 will be nice.

In the end field tests of "rc.1" will be nice. I guess you already have a wiki in mind for this. :)

[mwjames]

PS: If you send a PR and this only touches .md files then commit it
with [skip ci] so we avoid running the CI concert on those changes.

On 9/21/18, James Montalvo [e-mail] wrote:

Okay. I'm on it. Anything else you can think of?

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1365#issuecomment-423559704

[jamesmontalvo3]

@mwjames Understood: I'm looking at the code changes in addition to the text of the PRs. I'll add [skip ci].

@kghbln Okay I'll take a look at the list format issue afterwards. Briefly skimming it I'm not sure I understand how to "use CSS instead" but I'll figure it out.

I guess you already have a wiki in mind for this.

I have SMW master running on a copy of our production server with people testing it. No issues thus far. We're likely going to pull the RC into production right away since we haven't seen any issues, and we're running MW 1.31 so SMW 3.0 is a better choice than 2.5...so you'll get testers right away!

[kghbln]

Okay I'll take a look at the list format issue afterwards. Briefly skimming it I'm not sure I understand how to "use CSS instead" but I'll figure it out.

This is exactly what I would have to do.

I have SMW master running on a copy of our production server with people testing it. No issues thus far. We're likely going to pull the RC into production right away since we haven't seen any issues, and we're running MW 1.31 so SMW 3.0 is a better choice than 2.5...so you'll get testers right away!

A W E S O M E, this is great and at the same time very encouraging

[jamesmontalvo3]

I'm progressing slowly through RELEASE-NOTES because trying out the new features I'm learning about. Thanks for all the great work!

[kghbln]

So I am now going to do the "rc.1" Sit tight and stay tuned. @mwjames By looking at the open pulls and the progress of the automated testing I believe that these may soon be merged to still be part of it.

[mwjames]

By looking at the open pulls and the progress of the automated testing I believe that these may soon be merged to still be part of it.

As soon as their turn green, I'll push the merge button.

[kghbln]

@SemanticMediaWiki/testers SMW 3.0.0-rc.1 was just tagged. You may now do intense field testing. Thanks a lot!

[kghbln]

While I was at it I not only upgraded on "s-mw.o" SMW from 2.5.8 to 3.0.0-rc.1 but also MW from 1.27.5 to 1.31.1. The upgrades took about 15 minutes and went smooth without noticeable issues. Well the custom skin needs to be fluffed but this was expected.

[kghbln]

I am thinking of doing a "rc.2" on 2018-10-02.

[kghbln]

@Nikerabbit We have not seen translation updates in a while. This may be related to this. I would like to do a "rc.2" and need to figure out if I should wait or not.

[JeroenDeDauw]

Why would you need to wait on translation updates for a RC?

[kghbln]

Why would you need to wait on translation updates for a RC?

I wanted to avoid doing a tag at 10 and get translations at 11. Anyways this is indeed not a holdup. The main thing for me/us to know is if we are also affected by this. Since it is mandatory and not debatable for the final release to have translation updates I could right away postpone the schedule by a month minimum hence I do not necessarily need to do a second rc right now.

[Nikerabbit]

As a precautionary measure we have skipped all translation updates since last week. SMW is not directly affected though, so if you want I can do an update you right away. In any case we are restoring regular translation updates later this week.

[kghbln]

Thanks a lot for your insight.

In any case we are restoring regular translation updates later this week.

In this case it is not necessary to do an immediate update and schedules may stay as they are.

[kghbln]

@SemanticMediaWiki/testers SMW 3.0.0-rc.2 was just tagged. You may now continue to do intense field testing. Thanks a lot!