Sphinx: Sphinx-1.7 release plan

[tk0miya]

Hi maintainers,

Sphinx-1.6 was released at May 16. According to our time based release policy, this November is time to release. But I feel we could not work so much for Sphinx in this summer because of busyness, absent and so on. So I'd propose you to release next major version in this Winter, December, January or February. What do you think about the schedule?

I know we still have a lot of issues and requests.
Refs: https://github.com/sphinx-doc/sphinx/milestone/32

Personally, I could not work about 3months. So it would be nice to postpone the release 3 months. At present, 1.7 doesn't have big new feature from 1.6 yet. So it's not good for major release now.

[shimizukawa]

Thanks for the suggestion. I agree to postpone for about 3 months.

BTW, under the time-based release policy, I think that we don't have to worry about the presence of big new features.

[stephenfin]

I'd like to add two additional items that I think should form part of the discussion for this release: considering switching to trunk-based development and deprecating sphinx-contrib.

Switching to trunk-based development

Sphinx is currently developed using a something similar to the git-flow model. My understanding of this is as below: please correct me if I have anything wrong.

• We have two branches, master and stable (git-flow uses develop and master, but that doesn't matter).
• We consider master the unstable, development branch for the next release, while stable is for the latest stable release.
• Patches can be submitted against either master or stable. If the former, they will not appear until the next MAJOR/MINOR release. If the latter, they will appear in the next PATCH release and will appear on master after a merge from stable (which happens at regular intervals)
• A new MAJOR or MINOR release is cut by merging the other way from master to stable. A new PATCH release is cut by tagging a given commit on stable.

This leads to something similar to the below.

I'd like to propose that we switch to a trunk-based model. This model differs from the current model in the following ways:

• We now have N+1 branches, with a master branch and N stable branches named stable-VERSION or similar (examples: stable-1.8, branch-1.8, sphinx-1.8...)
• We still consider master the unstable, development branch and stable-VERSION branches the production-ready ones
• Patches should always be pushed to master and then backported (via git cherry-pick -x) to any version of stable-VERSION branches that we still support where the bug applies. The only exception to this is fixes that would no longer apply on master (for example, a feature was removed)
• A new MAJOR or MINOR release is cut by branching off from master when we consider a release _ready_. A new PATCH release is cut by tagging a given commit on stable-VERSION.

This leads to something similar to the below:

This model is used by projects like the Linux kernel, node.js, every OpenStack project out there, and many more. There are a couple of reasons I think this would be a better model for Sphinx:

• We can continue to fix older releases after we've moved onto the next one

  Sphinx is packaged by distros which maintain a given version for many years. We can benefit from their experiences as bugs are discovered NOTE: We don't need to support every release - perhaps only the last N releases

• We have a stable reference to all releases in a given MAJOR.MINOR version

  At the moment, it is not possible to find the documentation for Sphinx X.Y - only X.Y.Z. This means that effectively the only useful docs are those for master and stable. This is annoying if I want to find documentation for a slightly older version.

• Much less confusing history and easier contributions

  I have personally submitted doc fixes to Sphinx and wondered why they didn't show up at sphinx-doc.org. Having a policy that everyone submits patches to master and we backport these makes things much clearer. We also have less merge conflicts to deal with.

The only disadvantage I see is the additional effort required by maintainers to backport these patches if contributors choose not to do so. However, this is probably offset by avoiding merge conflicts found when merging from master to stable and vice versa.

For more information on the differences between the models, see here

Deprecating sphinx-contrib

An easy one. The effort to migrate things from BitBucket to GitHub is now complete and the new organization has been growing steadily. This has been discussed at length here and, as noted there, I'd appreciate your help in closing this out and removing the repo before Sphinx 1.7 releases. I have a pull request open to do this. I can answer any question on the mailing list or here.

[jfbu]

@stephenfin

At the moment, it is not possible to find the documentation for Sphinx X.Y - only X.Y.Z. This means that effectively the only useful docs are those for master and stable. This is annoying if I want to find documentation for a slightly older version.

You seem to be referreing to sphinx-doc.org, of course by git cloning the repo we can theoretically build doc for any tagged version. Theoretically as there may be difficulties from local environment (Python, Docutils). But having a gigantic pop-up at sphinx-doc.org will all past tagged version inclusive of intermediate releases does not look like improvement. Bug-fix releases try hard to not add new features or delete ones, so in each X.Y.z series, the latest one counts. Currently they are quite a few available via the pop-up at sphinx-doc.org. It is definitely not only master and stable.

You can always go to the rst sources of the documentation at any given point in commit history.

I have personally submitted doc fixes to Sphinx and wondered why they didn't show up at sphinx-doc.org. Having a policy that everyone submits patches to master and we backport these makes things much clearer. We also have less merge conflicts to deal with.

? I don't understand that. If your patch has been merged, either in stable or master, it will show on sphinx-doc.org. You may need though to manually trigger the doc update, by opening the pop-up and clicking on Builds. You can trigger a build there. Sometimes it mysteriously fails for whatever RTD reason, but at some point it should succeed.

I find the Sphinx git model quite simple and efficient. When time to major release becomes nearer a release-branch is created. The only incident is when some patch is then accidentally merged to master and not to the release-branch, meaning it will miss next major release. Anyway, near final release, no feature changing patches should be merged into release-branch, because the testing will have been done already by the people subscribing to user or dev-groups.

Backporting patches is asking for quite a few headaches on the maintainer side.

[tk0miya]

I don't think so. The advantages you said is not much better.

We can continue to fix older releases after we've moved onto the next one

From shortage of maintainers, we don't support older release. We always support latest release only. So this is not advantage for us.

We have a stable reference to all releases in a given MAJOR.MINOR version

What do you mean? We can refer all documents for each minor release. For example, docs for 1.5.x is placed at here: http://www.sphinx-doc.org/en/1.5.6/
I know its URL contains revision. But I believe this might not confuse users.

Much less confusing history and easier contributions

We usually add big changes into master branch. It means we can't cherry-pick the commits into stable-x.y branch sometimes. For example, sphinx.ext.autodoc is much different between master and stable.
AFAIK, contributors usually post new PRs for master branch. On review, we request to rebase them to stable branch if it is a bug fix. I think current model does not confuse whole of contributors.

The only disadvantage I see is the additional effort required by maintainers to backport these patches if contributors choose not to do so. However, this is probably offset by avoiding merge conflicts found when merging from master to stable and vice versa.

I feel time of maintainers is most important for this project. So I object to give new operations for them (including me).

Note: I don't opposite to create stable-x.y branch. But I don't know that is valuable or not.

@shimizukawa could you give your comment for us? I think changing of branch model effects to our release, CI and translation process (and anything else?).

[tk0miya]

Thanks for the suggestion. I agree to postpone for about 3 months.

@shimizukawa Thank you for comment.

It seems nobody opposite to my proposal, so now I propose all of you the rough schedule:

• Mid of Jan: Feature freeze, Call for translation and release 1.7b1
• Mid of Feb: release 1.7 final

Please let me know your opinion.

Thanks,

[stephenfin]

I meant to reply to this sooner but forgot to. Easy reply though...

Note: I don't opposite to create stable-x.y branch. But I don't know that is valuable or not.

OK, can we do this? At the end of this cycle, we can rename stable to stable-1.7 and keep that. We don't need to support patches to this branch, but it gives us a good reference containing the history for the release.

[jfbu]

@stephenfin don't we have already such references via tags?

[stephenfin]

@stephenfin don't we have already such references via tags?

We do, but as noted above it's not really suitable for docs. I'd like to be able to refer to the docs for 1.6 at http://www.sphinx-doc.org/en/1.6/. At the moment, I can only do this with a PATCH version.

[jfbu]

and https://github.com/sphinx-doc/sphinx/tree/1.6/doc isn't enough? (tongue in cheek)

[tk0miya]

We already provides the final document for 1.6.x series at here:
http://www.sphinx-doc.org/en/1.6.5/

(Unfortunately, Sphinx gives a version formed x.y to the first major release, not x.y.z. So "docs for 1.6" is a bit ambiguous.)

[jfbu]

I think before release of 1.7 we should adopt definitive policy regarding smart quotes (issue #3967). The change at 1.6 meant that smart quotes were applied by default not only for html and latex, but all output formats. This surprises users of text or man builders (e.g. #4357). Perhaps a config setting could be added, possibly a list whose elements would be the builders allowing smart quotes per default, thus html and html like targets and latex which I think were those obeying html_use_smartypants in 1.5.x series (not checked). Arguably 1.6.6 should backport that too, as I think the 1.6 change was a bug.

Unfortunately I don't think Docutils 0.14 offers an easy docutils.conf way to customize the "qDe" default regarding what kind of action smart quotes do (here, quotes, dashes, ellipsis). We could possibly provide this at Sphinx, but this is secondary issue.

[mitya57]

Perhaps a config setting could be added, possibly a list whose elements would be the builders allowing smart quotes per default

Good idea. How about naming this setting smartypants_builders?

Maybe we should also extend the deprecation of html_use_smartypants for one more release cycle and treat html_use_smartypants = False as smartypants_builders = []?

[jfbu]

@mitya57 My comment was a bit of wishful thinking, although I had well in mind the core problem that the transform is applied at early Docutils core action.

To illustrate, naive attempt fails (mypatch.txt applied to current stable branch): make clean html text applies smart quotes to both and make clean text html applies smart quotes to none, because (#3967) the transform affects the doctree (at pub.publish() which executes core Docutils code once and for all.) and the patch modifies smart_quotes setting depending on first builder name.

[stephenfin]

We already provides the final document for 1.6.x series at here:
http://www.sphinx-doc.org/en/1.6.5/

(Unfortunately, Sphinx gives a version formed x.y to the first major release, not x.y.z. So "docs for 1.6" is a bit ambiguous.)

Could we change this behaviour going forward? To be honest, I don't see much value in having http://sphinx-doc.org/en/MAJOR.MINOR.PATCH/ and would much rather http://sphinx-doc.org/en/MAJOR.MINOR/ links. The reason for this is that, from the perspective of documentation, there is no case where I'll want the documentation for 1.6.N-1 instead of 1.6.N. Newer versions of the documentation will invariably include bugfixes and I'll want to see these, regardless of what PATCH release they're using (you can use admonitions if something changed between patch releases). The reason I'm specifically suggesting feature branches is because I assume we are using readthedocs.org to host our docs and I think the only way to get release series (MAJOR.MINOR) URLs for this was to use release series branches. I could be wrong though.

If there's anything I could do to make this happen, please let me know.

[jfbu]

@stephenfin current behaviour appears coherent with PEP 440 and rtd policy. The www.sphinx-doc.org site shows last tagged releases for each major.minor.x series.

[tk0miya]

Could we change this behaviour going forward?

I don't know that. Its tag-branch model is strongly related with our documentation (includes translations). I agree with using MAJOR.MINOR to our documentation, but I don't know how to do that. I guess only @shimizukawa knows that.

I can say okay to make stable-1.x branch. It's very easy. But I feel changing document management model is not. TBH, I feel it is better to keep as is even if we make stable-1.x branch.
(In addition, we drop support for old stables immediately, so I feel old docs are not important.)

Note: We have to use 1.6 and 1.7 branch to manage our branches instead of stable-1.6 and stable-1.7. Because readthedocs uses name of branches or tags for its URL.

[tk0miya]

Hello maintainers,

Mid of Jan: Feature freeze, Call for translation and release 1.7b1

It's time to feature freeze. Are you ready? If no objection, I'll call feature freeze in next weekend (Jan 14th) and release Sphinx-1.7b1.
Basically, any new feature is not allowed for 1.7 after that.
So please let me know if you are in working. We can adjust schedule :-)

Of course any bug fixes are always accepted, and changes for 1.8 is also okay.

[Zac-HD]

It's time to feature freeze. Are you ready? If no objection, I'll call feature freeze in next weekend (Jan 14th) and release Sphinx-1.7b1. Basically, any new feature is not allowed for 1.7 after that.

I was hoping to get #4243 merged - it's been done for months and just needs review. I've even rebased the merge conflicts away recently!

[tk0miya]

Hi maintainers,

Today is X-DAY. Are you ready? I'll call feature freeze and ship 1.7b1 within 24 hours. More accurate, I will do that at Jan 16th 12:00 pm (UTC).
If you are still working, please let me know. I can postpone the schedule if it is important.

Note: Here is an issue list for 1.7
https://github.com/sphinx-doc/sphinx/milestone/32

At feature freeze, I will make 1.7-release branch. After that, only bug fixes (and minor changes) are allowed to the branch. And master branch is used for next major release; v1.8.

[tk0miya]

I propose you to merge #4271 after feature freeze. It is almost reviewed, but the value of option is not determined yet. I think this is minor fix and it does not break our code.

[jfbu]

I am not working on features at this time; #4430 adds a new confval, but it may to some extent be considered as bugfix, hence merge decision can be postponed.

[tk0miya]

Okay, the effect of #4430 is only limited for LaTeX writer. Go with 1.7b2.

[tk0miya]

I'm starting release work.

[tk0miya]

I just released 1.7.0b1 now.
https://pypi.python.org/pypi/Sphinx/1.7.0b1

[shimizukawa]

Thank you for the great work!

On Mon, Jan 15, 2018 at 10:34 PM Takeshi KOMIYA [e-mail]
wrote:

I just released 1.7.0b1 now.
https://pypi.python.org/pypi/Sphinx/1.7.0b1

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sphinx-doc/sphinx/issues/4211#issuecomment-357684341,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJQR5c74MObovmzPq94UcSlq-GNMXT0ks5tK1QCgaJpZM4QOcL2
.

[jfbu]

4431 is a blocker see https://github.com/sphinx-doc/sphinx/issues/4431#issuecomment-358588311

(should be fixed at #4452)
(has been merged to stable at d3058f462de8abc514e662270d3df62b08f953f9)

[stephenfin]

As is #4446, IMO

[tk0miya]

Ah, I'm very sorry. I'd forgotten to call for translation.

[tk0miya]

We have some updates on 1.7-release branch. So I'll ship a next beta release in a few days (maybe tomorrow).
And I'll ship final release on this weekend (Feb 3-4th) or next one (Feb 10-11th).

Please let me know your opinion.
Thank you always for your help.

[tk0miya]

So I'll ship a next beta release in a few days (maybe tomorrow).

I'd worked for #4230 yesterday and today. So I don't release 1.7.0b2 yet.
But I almost finished work about it on #4523. so I can ship it tomorrow.

And I'll ship final release on this weekend (Feb 3-4th) or next one (Feb 10-11th).

Because of the late of beta2, I'd like to release the 1.7.0 final on next weekend (Feb 10-11th).
In Japan, I and @shimizukawa will have hack-day in 11th. It's good chance to work to me.

[tk0miya]

Hi maintainers,

Do you have any remaining tasks or worries for 1.7 release?
This weekend is X-DAY of the release. I will work on the release in 12th Feb in JST. So please finish your work until then.
As usual, we can adjust the date if you still have any jobs. So please let me know about it.

Note: This is remaining issues for 1.7 milestone. I will work on some of them as possible, but I would postpone remaining issues to 1.7.1 (or 1.8) if not finished.
https://github.com/sphinx-doc/sphinx/milestone/32

[jfbu]

@tk0miya sorry for late response, please do go ahead. I apologize for not moving forward issues resolution regarding latex, but at this time I can not at all work on Sphinx and can't tell exactly when I will have some time. Good luck for the hard work of release.

[tk0miya]

@jfbu don't mind. Let's do them on latter release. I always appreciate your work!

[tk0miya]

From now on, I'm starting a new release. Wait a moment :-)

[tk0miya]

released!
https://pypi.python.org/pypi/Sphinx/1.7.0

Thank you for all your work!

[shimizukawa]

🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉