Cylc-flow: Remove mkdir_p.py when moving to Python 3

[kinow]

mkdir_p.py is almost simply an alias to os.makedirs, with the only difference being that it suppress the OSError: File exists error.

In Python 3.2+ os.makedirs takes an extra exist_ok=True argument to ignore that error. Which means that we can remove mkdir_p.py with no issues as soon as we start working on the Python 3 branch :+1:

On a side note, I wonder if we could have a label like "Python 3" perhaps? And start documenting what else we have to do, so that we don't forget by the time we start working on it?

[hjoliver]

Yep, do it. Not sure if I was unaware of 'os.makedirs' 8 yrs ago, or just hallucinating! Feel free to create useful labels too.

[matthewrmshin]

On a side note, I wonder if we could have a label like "Python 3" perhaps? And start documenting what else we have to do, so that we don't forget by the time we start working on it?

We should probably create some milestones. E.g.:

• Cylc 8alpha (Packaging solution done. GTK code removed. Python 2to 3 done.)
• Cylc 8beta (Web Architecture Ready: Reverse Proxy + Authentication service ready, and 1 or more new web service ready for demo.)
• Cylc 8. (Web UI done.)

[matthewrmshin]

@hjoliver As @kinow said, exists_ok as in os.makedirs(d, exist_ok=True) was only introduced in Python 3.2, so you were not hallucinating 8 years ago. 💊 💊 💊

Similar logic in Rose can be upgraded too:
https://github.com/metomi/rose/blob/d5e0db15c0b4d6979cf4db46b7d85b5fa567dd0e/lib/python/rose/fs_util.py#L131

[kinow]

@hjoliver could have suggested that change for Python!

[hjoliver]

Yay, glad I was not hallucinating :grin: (and I should have read @kinow 's post more carefully ... am on my phone at the conference).

[sadielbartholomew]

We should probably create some milestones

@matthewrmshin agreed.

We would of course need an agreed & rigorous definition of 'done' & 'ready' in each case, especially where it is not as clear-cut what a finished feature would be (i.e. for the Web UI we will want it to be functional & users to be happy.)

I advocate we brush up on our Greek alphabet & break milestones down further than your suggestions, though I realise they were just for illustrative purposes & not a concrete proposal. The breakdown between minor & major milestones is something we should consider carefully, because I don't think we can estimate even crudely relatively how much time & effort these various roadmap goals will take.

[oliver-sanders]

We should probably create some milestones

@matthewrmshin You know I'm keen on naming releases up-front i.e. 7.8.0 rather than next-release. Better for issue assignments and enforces the difference between bugfix and major releases.

I expect with different people working on different aspects of Cylc it will be very difficult to have fine-grained milestones for the intermediate stages of cylc8.

[hjoliver]

@oliver-sanders - we have "next-release" because it often wasn't clear in the past if the next release would be a maintenance release, before the the next minor or major.

I suppose an alternative strategy would be always anticipate the next minor release, and then cherrypick bug fixes to an un-advertised maintenance release if necessary. Is that what you'd prefer?

[hjoliver]

[note I just corrected "minor" -> "maintenance" in the second sentence above, without which it must have seemed rather nonsensical - sorry!]