Espeasy: V2.0 and mega branches too far from each other

Created on 3 Mar 2018 · 50Comments · Source: letscontrolit/ESPEasy

Dear maintainer(s)

While I'm mainly using the v2.0 (aka develop) branch to develop and push PR to you, I've once switched to the mega branch just to discover features that are still not currently in the v2.0 branch. This should be the opposite. 🌵

So I've made a diff between v2.0 and Mega branch and I've seen a LOT of differences!

As I can understand that ReadMe, Releases Note and versions differ, and that lastest (not merged) commits to V2.0 differs, I can not figure out the large amount of changes made into the mega that are still NOT in the v2.0

So either I still not understand the differences between the 2 branches 👶 (so please clarify), either you really should merge the mega branch INTO v2.0 branch, so that there are no such large divergence as of today.

_BTW I saw that there is a conflict to merge between both, but you could at least cherry-pick most of the mega-only commit into the v2.0._

🍻 🍻 🍻

Build Needs Info Discussion

Source

soif

👍2

Most helpful comment

Opinions about branching strategies can be very strong. There have been battles about less ;)
But I totally agree there should be some change in branching. Currently there are developers coding against "mega" and some (including me) against v2.0.
Regardless what's considered to be the best strategy, the current workflow is just wrong.

This one resembles your suggestion I guess:

I guess we should continue to use the "normal"/"dev"/"testing" defines to mark different levels of faith in stability. But that would probably only be used on plugins and controllers.
The core should be the same between releases.
This way we can prevent errors in storage compatibility and forgotten merges.

Last night, I merged about 5 plugins with a major issue, which was fixed in Mega and not in v2.0.
That should not happen and those are very hard to find.

For an "official" release, there should be a window of at most a few weeks between split an release.
But I think we could somehow adopt a continuous release cycle more or less what we're doing the last few months. Although it would be nice to have an official release indicating the big improvements. (and meet @Grovkillen, @psy0rz etc. :) )

TD-er on 3 Mar 2018

👍2 🎉1

All 50 comments

https://www.letscontrolit.com/wiki/index.php/ESPEasy#Version_names.2C_numbers.2C_and_branches

Grovkillen on 3 Mar 2018

I agree there is too much uncertainty, bugs fixed in either branch and not
in the other.
Some overview would be great.

On Mar 3, 2018 08:45, "soif" notifications@github.com wrote:

Dear maintainer(s)

While I'm mainly using the v2.0 (aka develop) branch to develop and push PR
to you, I've once switched to the mega branch just to discover features
that are still not currently in the v2.0 branch. This should be the
opposite. 🌵

So I've made a diff between v2.0 and Mega branch and I've seen a LOT of
differences!

As I can understand that ReadMe, Releases Note and versions differ, and
that lastest (not merged) commits to V2.0 differs, I can not figure out the
large amount of changes made into the mega that are still NOT in the v2.0

So either I still not understand the differences between the 2 branches 👶
(so please clarify), either you really should merge the mega branch INTO
v2.0 branch, so that there are no such large divergence as of today.

BTW I saw that there is a conflict to merge between both, but you could at
least cherry-pick most of the mega-only commit into the v2.0.

🍻 🍻 🍻

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/letscontrolit/ESPEasy/issues/997, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADk9lmn0V6kjJYGa9Rl-R8wVnDWjLCUkks5takofgaJpZM4Sa0mV
.

TD-er on 3 Mar 2018

Why do not simply adopt the convention widely adopted in MOST open source project all over the world?
Two master and develop branches. PR goes to _develop_ ONLY, then _develop_ is regulary merged into _master_ when commits are considered stable enough.

This way developers work on the _develop_ branch, while regular users test the more mature _master_ branch: simple, standard , easy and no conflict between branch

In this goal you would certainly need to:

rename _master_ to something like _v1.4-stable_
rename _v2.0_ to _develop_
rename _mega_ to _master_
finally merge all difference from _master_ into _develop_, to restart clean.
from now on, make PR and new commits ONLY to _develop_, then regularly merge it into _master_
and insert tags in _master_ and _develop_ like _"v2-alpha20180302"_.

That would certainly be easier for everyone, including beta testers.

_This would even allow us to also use the convenient _git flow_ workflow (read this or this if you dont know it already)._

What you think ? :heart_eyes:

soif on 3 Mar 2018

👍1

This one resembles your suggestion I guess:

Last night, I merged about 5 plugins with a major issue, which was fixed in Mega and not in v2.0.
That should not happen and those are very hard to find.

TD-er on 3 Mar 2018

👍2 🎉1

Yep, the "git-flow" workflow resembles to your scheme, I find it more clear on the graph at:
https://danielkummer.github.io/git-flow-cheatsheet/

But anyway using "git-flow" is not really my point, it would just be an option (that you'd choose to use or NOT) made possible simply by using the standard [ _develop_ and _master_ ] branch naming practice.

You could just stick to a _master_ + _develop_ strategy (PR to dev, merge to Master), and may-be add a _develop-plugins_ branch for the testing ones. You should also definitively rename the current master to something like 'v1.4-current-stable'.

But whatever the naming you finally choose, IMHO _mega_ should be merged into _v2.0_ in a way or another.

I'm only a small committer, and you are the guys who will decide how you want to organize yourselves, but I've just liked to give you some feedback on how messy and confusing it looks for external guys whoever want to contribute or just test latest firmware released (and so report issues). And also as noted by @TD-er , the current branches just add unneeded work and headache 👶 .

I'm pretty sure it would be a huge improvement for everyone, if you make all of this simplier, whatever the strategy you finally choose 🍺

@TD-er : What you mean by :

and meet @Grovkillen, @psy0rz etc.

soif on 3 Mar 2018

@soif At the slack channel, @Grovkillen mentioned it would be nice if we could meet face-2-face.
And @psy0rz and I live quite close to eachother (less than 100 km). So that was once mentioned as an excuse for a "release party" :)

And I think it is great feedback, especially since I spent more than halve the time yesterday on merging + comparing branches, it is exactly the right time to start a discussion about it.

TD-er on 3 Mar 2018

It's my main focus right now, the release party! :wink:

Grovkillen on 3 Mar 2018

You're both from Netherland, right? Greeting from South of France 😃

soif on 3 Mar 2018

In my opinion it is better to push into development branch and then after a while of testing, cherry-pick it into stable, but it is sometimes more than one commit to pick .... how to track what is missed?
On the other hand have some problems with git rebase, so I use only git pull, not sure what is wrong, need to try one more time.

Greetings from Poland :)

uzi18 on 3 Mar 2018

@uzi18 I guess it would make things easier for you.
You only have to focus on one base level and thus less issues with Git for the normal devs.

@soif @psy0rz and I both live in the northern part of the Netherlands indeed. @Grovkillen lives in .se :)

TD-er on 3 Mar 2018

What a truly European 🏤 dev team 😈 ! Not any Yankee around ?

@uzi18 Cherry-Picking only a few commits, would in mid term returns to the exact same situation : the two branches will end up diverging .

The common practice is that everyone always work and then push to the develop branch; But, then, the maintainer merge the develop branch into the master branch. Often he would choose to merge only commit that are old enough so that he can be confident that they were stable enough whenever no developer have encountered any issue since. Also ANY commit (ie a HotFix) pushed directly to the master branch should immediately be merged into the dev branch at the same time.

Another more sophisticated workflow is to use the "Git Flow" method cited above, but that needs a bit more training for everyone involved.

The opinion of our beloved @Grovkillen ⭐️ and @psy0rz 🌟 would certainly be helpful.

soif on 3 Mar 2018

Yes we have a big misunderstanding regarding the branching so a reboot is needed. But not until after the release of 2.0 I vote.

Grovkillen on 3 Mar 2018

For now I just trying to give for rules more power and intuitive coding way for newbies.
As You can see every step further we need to fix strange or even never working things (time comparisions).

@soif 1/2 of Europe ;)

uzi18 on 3 Mar 2018

But not until after the release of 2.0

But what will be the real stable v2.0? 😱

According to the wiki, you says it is the v2.0 branch (ie the master) and that "Mega" is the development version (aka develop), but we most PR to _V2.0_ while @psy0rz always merge if after some time into _mega_, while some fixes, are in the meantime committed ONLY to mega.

This the exact mess I'm talking about 😸 😸 😸 😸

soif on 3 Mar 2018

BTW both branches are not that far at this time, so it should not be too difficult to "reboot", while every commit we are submiting enlarge the gap between the branches, and will certainly make it more difficult to merge in the future..

This would certainly help us altogether to be more efficient, and thus fastly release the awaited v2.0 sooner, if the branching method were more crystal clear for everyone. 👅

soif on 3 Mar 2018

👍1

@soif I agree about the mess.
Regardless of what we will choose, any consistent push/pull direction is better than the current situation.

Also there is a number of issues in v2.0 that are working in Mega. So this will take a whole lot of time to fix it.
I'm not saying that we should do it right now, but we should consider it and decide in at most the next 2 or 3 weeks.
like I also wrote to @Grovkillen in the chat:

Last night was about 70% merging/searching and 30% coding.
And I am holding off certain bigger things because of the merge issues. (e.g. settings to/from JSON)
Also changing settings is a bit hard due to the 2 different branches. You'll have to take both into account when adding things to the settings.

TD-er on 3 Mar 2018

@soif Regardless of what we will choose, any consistent push/pull direction is better than the current situation.

This exactly resumes what I think! 👏 👏 👏 👏 👏 👏

soif on 3 Mar 2018

I'll be on a business trip in the Netherlands in may or so. Near Twente... How far would that be from where @TD-er and @psy0rz live ?

s0170071 on 3 Mar 2018

@s0170071
Probably less than 2h drive from my house.

TD-er on 3 Mar 2018

My idea thusfar was this: mega is the development branch, v2.0 is the next release/stable branch.

Bugfix PR's go into the next stable (v2.0) and are merged back to the development branch (mega) regularly. All development is going on in the development branch only.

This way we only have to fix stuff once and can do a git merge v2.0 to get those fixes into the development branch.

This looked like less hassle in theory: we dont need to cherry pick anymore. But the problem arises when someone fixes something in the development branch and we have to cherrypick it back to the stable branch.

So indeed a more standard approach might be better: do ALL development in dev and cherry pick those fixes that should go in stable as well.

Should we do this? How and when should we do it?

psy0rz on 4 Mar 2018

Merging back and forth between branches is prone to error. Sometimes there are dependencies that are not obvious and its a lot of work. Do we really need two branches ?

We could have just one. Add new stuff at first, when the milestone comes- pull in bugfixes only. After the milestone add new stuff...

s0170071 on 5 Mar 2018

If could, i would vote for one ... and a half branch! 😆
I mean:

everybody works on develop. When the milestone comes or when he find that the current state is really stable, the maintainer merges it into master and tag a release. 🍻 🎉
then everyone continue on develop
if there are "hotfixes" , they are directly pushed into master, and always cherry-picked into develop, so that branches never diverge (as of today). _develop_ should ALWAYS be ahead!

So the master is just the photography of a state considered very stable. Then only rare, small and urgent hotfixes are posted directly into it. The real branch always stays _develop_.

Anyway, you the masters need to decide something ASAP! 🐞

soif on 5 Mar 2018

👍2

Fully agree on the ASAP :) (and based on very quick reading, I agree with your view)

TD-er on 5 Mar 2018

Just to get a bit of insight. Is it the "release isolation" the approach @soif is suggesting?

https://docs.microsoft.com/en-us/vsts/tfvc/branching-strategies-with-tfvc#release-isolation

Grovkillen on 5 Mar 2018

This is rather the "development-isolation" just above in your page. With only really rare hotfix to the _master_. Even Never if possible.
_develop_ is THE branch to always use, then _master_ is just a sort of "snapshot" of stable (released) versions.

--> Very simple for everyone, commonly used in a lot of OpenSource projects

soif on 5 Mar 2018

I agree with @soif. I oppose however hotfixing a release. It its released it should remain untouched. I would rather increment the version index. e.g. 2.0 --> 2.0.1

s0170071 on 5 Mar 2018

Absolutely, i just meant than when (on very rare occasions), a fix must be urgently posted to lets say v2.1.0 , and while the _develop_ branch is not considered as stable enough to be merged into _master_ at the current time , the hot fix would be posted directly into _master_, while merging immediately back to _develop_ , and obviously tagging the master as v2.1.1

soif on 5 Mar 2018

👍1

Ok, so what are the options here?

Development isolation
More?

Grovkillen on 5 Mar 2018

Discussions regarding beaching is also found here: #714

Grovkillen on 5 Mar 2018

BTW regarding version naming pinned in your #714 , I personally use and inject (using my EspBuddy, best ESPEasy dev friend) a Build tag which is automatically build using a formula like : ="GitBranch,GitTag,#GitCommit(onlyWhenNotEqualToLatestTag)_DayMonth-hh:mm)", ie:

mega,mega-20180304,#6193553_04Mar-10.46 always shown on the Main web GUI page. really more helpful than the current .... EMPTY string 😛

[SPAM]rant, rant.. on Linux (or VM), or OSX : try EspBuddy : it'll certainly make your dev time faster..[/SPAM]

soif on 5 Mar 2018

@soif add some informations on EspBuddy repo how it works exactly :)

uzi18 on 5 Mar 2018

@soif All that date and build information is already on the info page. Just not on the main page.

s0170071 on 5 Mar 2018

About the branching...
I really think we should go to a single branch containing all the new features/functions/plugins/etc.
That's the only way we can prevent bugs caused by merging, simply not merge them between branches that grow further apart.

Also the simple question, "which branch to use for development?" is already a difficult one.
Mega is a superset of v2.0, or at least that was the intention. So adding new features should be done on Mega, which makes merging a nightmare. Fixing should be done on v2.0 to make sure you're not using stuff only present in Mega.

Adding things to store in the settings has to be done on both, or else you will break compatibility. But that means you have to port things from Mega to v2.0. And that may lead to an avalanche of merges.
What to do with fixes on Mega? They should also be fixed in v2.0, but may not always be that easy, will be forgotten, or perhaps not present there, or manifesting as different errors or fixes will cause other issues.

So please let's change that, since it takes a lot of time and a lot of (negative) energy.

What do you think about these "guidelines"?

There should be only 1 "dev" branch.
Changes to "core" parts (e.g. rules, storage, webserver, controllers, refactoring) should be coordinated and be split in smaller parts.
New controllers, plugins should be marked "unstable" or "testing", but may be incorporated into builds quite fast, as long as users know things may still be in flux or not stable yet.
There must be a "dev" branch to which no-one will commit directly. All should be done via branches and/or pull requests.
A new feature or fix branch should start at the current head of the "dev" branch.
Not sure about an extra "master" or "mainline" branch. It sounds more logical to make a "release" branch from a certain point with a "release candidate" or two, followed by a release.
Release branches should have a proper name like "release/v2.0" and should not be deleted after a release.
The difference between the last RC and a release should only be a file stating the version number.
Each release (and RC) should have a tag. Maybe also for daily builds, but have to look at how practical that will be. Tags should have a "folder"-like structure like "release/v2.0-RC1" or "daily/2018/03/05" Folder like structures can be folded in most Git tools.
The duration of an active "release branch" should be a few weeks at most.
Changes on RC's in a release branch should be done only by cherry pick, or via a PR based on a temporary branch of the release branch.
A hotfix should be done in a sub-branch starting at the release point of a release branch.

That's a lot of text, so please shoot :) (not at the messenger, at the suggestion)

TD-er on 5 Mar 2018

👍1

@s0170071

currently, I see this in Info :

Build | 20100 - Mega (core 2_3_0) (😆 😆 😆 😆 nearuseless at this time)
GIT version | small,mega-20180304,#6193553_04Mar-10.46 _(MY injection !!!!!!)_
Build time | Mar 4 2018 10:47:08 _(OK a litle helpful)_
Binary filename | ThisIsTheDummyPlaceHolderForTheBinaryFilename...

😈 😈 😈 😈

soif on 5 Mar 2018

The core label is far from useless. Or do you mean the "20100 - Mega" part?

TD-er on 5 Mar 2018

[ESPEasy-v2.0]$ git describe --tags --long --dirty
v2.0-20180228-41-g7318e87

uzi18 on 5 Mar 2018

[ESPEasy-mega]$ git describe --tags --long --dirty
mega-20180304-4-g1335dca

uzi18 on 5 Mar 2018

What about priorities ?

Is there a way to "vote" for new features and bugfixes?
Do bugfixes always have priority ? If so, are there categories related to urgency?
e.g. stability = 1, safety related=2, plugin =3 feature =5?

Just asking cause I got stuck in a really nasty config caused reset loop just yesterday. I would love to see the xml configuration way sooner than https for example.

s0170071 on 6 Mar 2018

XML configuration?

But I agree on a proper view of priority.
There is a system of 'thumbs-up' votes on which you can sort issues.

TD-er on 6 Mar 2018

@TD-er With XML configuration I mean storing configuration, i.e. SettingsStruct, ExtraTaskSettungs etc. as XML so that it is scalable.
The thumbs up is ok, but there should be a "sticky" issue in which owner of the issue updates the Top10 issues so you can place your thumbs.

s0170071 on 6 Mar 2018

JSON would be better IMHO : lighter , easier to parse, easier to read, well implemented in most languages.

soif on 6 Mar 2018

@soif: fine with me.

s0170071 on 6 Mar 2018

🎉1

@soif about
Binary filename | ThisIsTheDummyPlaceHolderForTheBinaryFilename...
this happens when you don't invoke crc2.py. It injects the MD5 checksum and the filename of the binary. See https://github.com/s0170071/CRC4ESP for installation instructions.

s0170071 on 6 Mar 2018

@s0170071 , to not completely hijack this thread, i've just opened an issue about this in your own repo 🎉

soif on 6 Mar 2018

I see people talking about why we dont just have one branch.

We had one branch before, but the problem then is we stay in infinte beta: New features are added all the time and new bugs will also be created all the time. There never will be a point in time the one branch can be considered stable.

Thats why a "stable" branch was created (v2.0): The branch is not stable YET and has to become the next stable release. We only add fixes there until its really stable. Then we can release v2.0.0.

In the mean time new features keep getting added to the dev branch. (mega)

So i hope the reasoning behind having 2 branches is clear now. Maybe we should do it another way? Suggestions are welcome.

psy0rz on 7 Mar 2018

F34R ! :-)
looking at the mess it might be easier to keep it even simpler
rename master to legacy
rename mega to dev
remove v2.0 branch
use tags.
you'll have plenty of time to rethink/rebranch when you manage to get a stable v2 out :)