Dietpi: General | Methods to potentially improve DietPi release cycle stability

[MichaIng]

Currently we relatively often push new releases with one or more serious bugs, affecting network connectivity, time sync => SSL errors, permissions and such. This is due to a quite fast development with many new features, core improvements and such.

Results are hotfix releases, which are not always separated from other new features (currently multiple WiFi SSIDs e.g.) possibly leading to other new bugs and unhappy users.

Reason is clearly that we have on the one hand many great ideas from users and devs about new features and improvements but lack the devs/time and in my case possibilities to test those changes on every device and possible setup.

My suggestion is, even that it is an additional effort and slows down development a bid, to implement an open beta/RC branch.
E.g. we do a moreless monthly release with new features and improvements. But we do a beta/rc release one week earlier, that hopefully many interested users can test, and concentrate our efforts then on fixing bugs that may have been introduced, before merging it into final release.
This would mean e.g. that we develop always in a new "development"/"dev" branch, but backport only bug fixes to the RC branch (could be the testing branch) in regular RCs during the testing week, using subsubversion strings to allow updates for the beta users.

Additional effort is the backports that need to be done, but at least bug fixes are usually small code snippets. And it should clearly increase our release quality, at least if we find enough users that are willing to join the beta/testing branch.

Another topic is bugs, that break APT, time sync and such and by this break the ability to update without manual pre-fixes. Perhaps we can move current emergency update steps into an own small pre-update file/script, that is applied as very first update step, at best available via non-SSL source? But maybe there is an easier solution, as I think APT and SSL are the only reason why this might be needed. In the past we did ATP upgrade after applying the incremental patch file, but that brings other possible issues when we expect current packages for certain steps.

These are just suggestions and the current ideas and solutions I have in mind. My most important intention is to start discussion/idea collection about how to make DietPi more robust for production systems.

[Fourdee]

@MichaIng

Reason is clearly that we have on the one hand many great ideas from users and devs about new features and improvements but lack the devs/time and in my case possibilities to test those changes on every device and possible setup.
My suggestion is, even that it is an additional effort and slows down development a bid, to implement an open beta/RC branch.

Great idea, agree 👍

• Master = Stable
• Beta = Pre-release testing, open to all users
• Testing = In dev, potentially unstable

The only issue I see, is after we roll out Testing > Beta and need beta users to update, we'd essentially need to increase the subversion to notify update. Which results in master branch skipping a few versions per release?

Perhaps we can move current emergency update steps into an own small pre-update file/script, that is applied as very first update step, at best available via non-SSL source?

Yep, i've been thinking about this for a while, moving EMR patches to a patch_prefile, which runs:

• After 1st initial internet connection test
• Before systemD timesync check
• Before APT

It can then be used for pre-timesync emergency patches.

at best available via non-SSL source?

We can use SSL source with simply the --no-check-certificate option to prevent non-timesync'd system failing. Essentially, we could use Github for patch_prefile I believe.

My most important intention is to start discussion/idea collection about how to make DietPi more robust for production systems.

Yep I agree. However, this will only improve with these new systems, if we have users who are willing to help test.

[MichaIng]

@Fourdee

Beta = Pre-release testing, open to all users
Testing = In dev, potentially unstable

Beta = testing and testing = dev is perhaps a bid confusing? This is why I thought to use the current "testing" branch for testing/beta and implement a new "development"/"dev" branch for what we currently do on "testing"? But this means that we need to switch e.g. local git branches and such, so when this is too much effort, your suggestion would work as well of course 😉.

The only issue I see, is after we roll out Testing > Beta and need beta users to update, we'd essentially need to increase the subversion to notify update. Which results in master branch skipping a few versions per release?

I would add sub-subversion for this, e.g. v6.17.0 for initial RC/beta, then v6.17.1 for next etc.

We can use SSL source with simply the --no-check-certificate option to prevent non-timesync'd system failing. Essentially, we could use Github for patch_prefile I believe.

Ah that's true, the wget download does also work with ignoring untrusted SSL cert, right?

Yep I agree. However, this will only improve with these new systems, if we have users who are willing to help test.

Jep, I hope we find enough that it is worth it, making some advertising for it on twitter and forum e.g. 😄.

[Fourdee]

@MichaIng

the wget download does also work with ignoring untrusted SSL cert, right?

Yep, however, using http:// has the same effect. Both are the only option we have pre time sync.

Jep, I hope we find enough that it is worth it, making some advertising for it on twitter and forum e.g.

Yep 👍

This is why I thought to use the current "testing" branch for testing/beta and implement a new "development"/"dev" branch for what we currently do on "testing"?

Yep, probably a better idea, we should rename the testing branch to 'dev'.

Then with the beta branch, we can roll out dev to beta, when everything is completed. This will then allow users to update and test.

I would add sub-subversion for this, e.g. v6.17.0 for initial RC/beta, then v6.17.1 for next etc.

Yep, this would allow for beta updates from 'dev' branch once we fix issues.

---

Ok so?

• Rename 'testing' to 'dev'
• Create 'beta' branch, which is a copy of 'dev' once we are ready for release
• Allow 'beta' branch to test for X days, before we merge to 'master'
• Use an additional 'beta' version to allow for 'beta' branch to update when 'dev' updates are applied, during the RC stage
• 'beta' version ignored on 'master' branch.
• Release 'beta' to 'master' when stability testing is completed after X days
• Patch notes:
  -- Switch 'testing' to 'dev'
• Web notes:
  -- Create BETA_BRANCH.md
  -- rename 'testing' to 'dev'

[MichaIng]

@Fourdee

Yep, however, using http:// has the same effect. Both are the only option we have pre time sync.

Ah this is great. So we can pre-patch everything besides broken network itself 👍.

Ok so?

• Rename 'testing' to 'dev'
• Create 'beta' branch, which is a copy of 'dev' once we are ready for release
• Allow 'beta' branch to test for X days, before we merge to 'master'
• Use an additional 'beta' version to allow for 'beta' branch to update when 'dev' updates are applied, during the RC stage
• 'beta' version ignored on 'master' branch.
• Release 'beta' to 'master' when stability testing is completed after X days
• Patch notes:
  -- Switch 'testing' to 'dev'
• Web notes:
  -- Create BETA_BRANCH.md
  -- rename 'testing' to 'dev'

Sounds great. I am thinking, if we should use sub-sub-versions for beta updates only and simply ignore them on master/release update check, or as well implement them for master releases (for smaller, properly still needed hotfix releases). As long as every system just checks it's own GitHub branch for current server version string, both should work fine, but has impact on patch_file complexity.

Also about sub-sub-version updates:

• If sub-version is not up-to-date, update via patch_file sub-version patches only.
• If sub-version is up-to-date, check for sub-sub-version update and use separate patch file. We then merge this sub-sub-version patches into the regular sub-version patch, so there always only need to be a single sub-sub-version patch_file for the current beta?

As we already touch the updating system:

• Is your plan to never allow updating to new main version (DietPi v7)?
• Personally I would always try to allow updates, since flashing a new image is always a large effort for users.
• Reason is, that I would love to announce the above via DietPi v7.0 to highlight the step to more stability/reliability and hopefully get more attention from potential beta testers. This would mean already a first pre-patch task to allow patching main versions.

[MichaIng]

Further thinking about pre-patch, downloaded via HTTP. Actually I am not too happy with this for security reasons. I know there are many other cases (3rd party software, non-HTTPS APT sources, ...) where simple HTTP is used, but I would like to use HTTPS where we have control.

• Theoretically someone evil could otherwise implement harmful code and end user has no chance to verify it is from us. Maybe cons < pros but just to think about.
• Thought about some checksum solution, but we cannot know in current version (where it would be needed) how the checksum if future pre-patch would be.

Sadly this case HTTPS would break one of the main reasons for pre-patching: time sync or other HTTPS related issues. Maybe we can instead further improve our G_CHECK_URL to check output for HTTPS related errors. If check fails, but --no-check-certificate does not, e.g. this would be a clear indication after which we can print some usual checks+solutions.

Also I thought about some kind of (optional) news/announcement inside dietpi-banner, or a "New announcement available, call "dietpi-announcement" print, which only appears until the script is called once (and not unchecked by user, dietpi.txt entry, allowed to toggle within the script itself?).
There we could publish some current issues, important bugs and such, possibly based on distro, device and/or installed software of end user. Or simply inform about other interesting things. Stay closer in touch with users, who are open to it.

[Fourdee]

@MichaIng

Sadly this case HTTPS would break one of the main reasons for pre-patching: time sync or other HTTPS related issues.

Yep, its a catch 22. We need one to do the other.

Would be nice to have, but yep, "security" and "risk" of http for a pre-patch isn't worth it.

Maybe we can instead further improve our G_CHECK_URL to check output for HTTPS related errors. If check fails, but --no-check-certificate does not, e.g. this would be a clear indication after which we can print some usual checks+solutions.

Nice idea 👍

Also I thought about some kind of (optional) news/announcement inside dietpi-banner, or a "New announcement available, call "dietpi-announcement" print, which only appears until the script is called once (and not unchecked by user, dietpi.txt entry, allowed to toggle within the script itself?).
There we could publish some current issues, important bugs and such, possibly based on distro, device and/or installed software of end user. Or simply inform about other interesting things. Stay closer in touch with users, who are open to it.

Yep, nice idea 👍

Although we could simplify this and have a link to https://github.com/Fourdee/DietPi/issues/2093 in the banner, then simply keep the ticket open and nudge v6.16 down after v6.17 release?

[Fourdee]

Ok in regards to changes:

testing > dev

• v6.17 patch , change GITBRANCH=testing to GITBRANCH=dev
• Clone testing to dev
• Now work on dev

[Fourdee]

@MichaIng

I've made a start on this: https://github.com/Fourdee/DietPi/issues/2083#issuecomment-426842537

Please do any work on the new 'dev' branch:
https://github.com/Fourdee/DietPi/tree/dev

[Fourdee]

Note to self:

• Code done
• Needs further testing
• RC version only used for 'beta' 'dev' branches. Master branch is always X.X.0

When we want to update 'beta' branch with new updates:

• Change .version 3rd line to ++ number
• Push 'dev' to 'beta'
• Users on 'beta' will then see an update
• Current subversion patch will be rerun

When we want to merge from 'beta'/'dev' to 'master

• Change .version 3rd line to 0 (reset for next release), as subversion takes priority.

[MichaIng]

@Fourdee

RC version only used for 'beta' 'dev' branches. Master branch is always X.X.0

Why not keep RC version for release? This would also allow beta users to update to most current stable, when switching branch setting. E.g. they just test 1st RC, then for some reason miss 2. and 3. and release. Then they want to switch to release. But update would now be shown if server version is X.X.0.

Ah but let me test and revisit the code first before I can give qualified feedback 🤣.

[Fourdee]

@MichaIng

Why not keep RC version for release?

To allow us to reset RC versions for each subversion release. As the RC version is used to allow updates for 'beta' branch.

Ah I see what you mean, just let the RC version roll upwards and never reset? Should work fine aswell.
Might confused our users though:

• 6.17.0
• vs
• 6.17.29403

[MichaIng]

@Fourdee
I though to reset rc version to 0 on next subversion raise in dev branch.

Possible that dietpi-update needs little fix for this, although can't find a reason. But need to have a fresh look into it tomorrow. Idea:

• Check for available subversion update and apply in case.
• If no subversion is available, check for rc version update and reapply current subversion patch in case.
• After every kind of update, simply store the rc version integer gathered from server.
• We will reset rc version to 0 on dev branch after release, when rising subversion as well.
• So regardless branch, if rc server version < rc local version, the current subversion will be offered/reapplied as update. But if a new subversion is available, the local rc version is not even checked, just the one from server stored as local one after update.

---

Btw:

• Is it possible to add branch specific info to the header/description (at the top of repo main page)? To describe the purpose of the branch? We could add this info the README.md top line as well, but then would need to always change it when merging dev => beta or beta => master. I hope there is a possibility the prominently add branch specific info, that do not affect the download.

[MichaIng]

@Fourdee
Do we want to add branch info to dietpi-banner, e.g. into the version string line?
DietPi v6.17.3 (beta)/(release)/(MichaIng/custom)

Fourdee/master could be replaced by "release" and Fourdee/beta by "Beta" and "Fourdee/dev" by "development", but all custom owner/branches leave as "/". So users clearly know which DietPi version they use.

Only issue is, that we need to store the git owner+branch separately on update (4. line of .version?) instead of reading from dietpi.txt, so the info is always correct, even if users changed dietpi.txt but did not yet run update? On the other hand, why would someone change git branch/user, if not want to do an update afterwards...

---

Although we could simplify this and have a link to #2093 in the banner, then simply keep the ticket open and nudge v6.16 down after v6.17 release?

I think a link would not do. Users know where to ask/search, if they face an issue. But some issues might occur with the update (although with pre-patch the risk is minimized), or it is not directly visible to users. To prevent users from running into it and searching the forum for workarounds, it would be most helpful if we directly warn/prompt about them in the header. But yeah, one might argue, that everyone should be able to and do the effort of having a look into forums/github oneself, when facing an issue 😉. This is really taking users by the hand. Although at least until certain degree this is anyway to idea behind DietPi, allowing simple and automated installs/configs and such.

It could further be used to assist forum/twitter announcements by also placing an info there. Bein closer in touch with users, besides bugs/issues.

Being more concrete, we could place a script into .meta, which similar to pre-patch contains some checks, and fills an announcement/notification string (flag file), based on checks results. dietpi-banner, on execution (or regularly via cron) downloads this script (silently), runs it and, if anything was added to the string (at least one check passed), shows a colourized hint, that a new notification/announcement is available and user can run dietpi-banner 2 (?) to see it.
Only issue is, that we also need to track, which notification was already read, so somehow need to store notification ID and read/unread or something 🤔. Hmm becomes more complex to more I go into detail 🤣. Yeah just an idea, perhaps not a good one...

[Fourdee]

@MichaIng

Do we want to add branch info to dietpi-banner, e.g. into the version string line?
DietPi v6.17.3 (beta)/(release)/(MichaIng/custom)

I believe for dev/beta we can add additional info. But with master branch, leave as is to keep it clean:

 ───────────────────────────────────────
 DietPi     | Update available
 ───────────────────────────────────────
 v6.17.3    | Odroid C2 (aarch64)
 ───────────────────────────────────────
 eth0       | 192.168.0.113
 ───────────────────────────────────────

On the other hand, why would someone change git branch/user, if not want to do an update afterwards...

Exactly 👍. I added a global for branch G_GITBRANCH, we could probably do the same for G_GITFORK.

Performance wise, should be faster to store/read the above info in .version as dedicated line. Might be worth doing.

Actually, we should update .version to be sourceable. We have the global vars now.

[Fourdee]

🈯️ Updated .version to be source-able.

[MichaIng]

@Fourdee
Great commit! Looks way more consistent like this. Tiny remark:

• Having this branch selection renders adding at least manual G_GITBRANCH export to PREP useless, besides one can cancel the whip menu.
• Furthermore we cannot assure that on every G_GITOWNER repo these branches are available.
• Easiest would be properly to only ask for this, if G_GITBRANCH and G_GITOWNER were not exported to script, thus as well only when G_GITOWNER default=Fourdee. Otherwise assume, users wants it the way provided by variables.
• More elegant would be a selection menu based on available owners + branches:
  
  
  
  • Scan https://github.com/Fourdee/DietPi/network/members for available owners. Puhh no way too much and most, if not all of them are just left-overs from a PR. Then better: Fourdee (official) as owner selection, Custom (unsupported) as second option to allow textbox entry.
  
  
  • Scan available branches, with, on Fourdee, master, beta and dev at the top: https://github.com/Fourdee/DietPi/branches/all
  
  
  • Puhh, but quite complex, perhaps stay with: "Only prompt branch selection, if both, G_GITBRANCH and G_GITOWNER are not yet available (not exported/added to script).
  
  

Found another inconsistency:

• dietpi-globals read G_GITOWNER and G_GITBRANCH from new .version now, which is the currently installed version.
• dietpi-update however use this values now as well, instead of the ones from dietpi.txt which is the desired/target version. Thus effectively branch cannot be changed any more.
• We should separate this: Use the above globals as "current" info, but let dietpi-update read dietpi.txt for "target" branch/owner to use for server URLs.

[MichaIng]

@Fourdee
Sugar plums!, accidentally committed to dev instead of PR: https://github.com/Fourdee/DietPi/commit/f62675e0ceaf93d4720db22d2c4b29dddecb0888

However will do some testing with this.
€:
🈯️ VM Jessie, from v6.17.3
🈯️ VM Stretch, from v6.17.3, from v6.0
🈯️ VM Buster, from v6.17.3

---

Needed to send some more commits to bring consistency inside and fix some patches that break the new versioning system.
So sorry the commits/changes now have grown outside of a PR, I hope you agree with them.

What I did is:

• Clearly separate $G_GIT* variables, which should be considered as currently applied branch, from dietpi.txt DEV_GIT* settings, which should be considered as target branch for updates.
• It is now possible to not set DEV_GIT* settings at all (comment them), every script in case applies defaults Fourdee/master.
• For above reasons, DEV_GIT* settings are never overwritten. dietpi-update only reads them to access desired branch and updates the "current" values from .version accordingly after update has finished.
  
  
  
  • Only exception is the switch from "testing" branch to "dev" during pre-v6.17 patch, which is of course needed.
  
  
• Removed some obsolete Git settings/values read/writes, since up-to-date dietpi-globals always does. Writes should be done by dietpi-update only.
• Updated check [[ ! $(sed -n 6p /DietPi/dietpi/.version) ]] now assures the most current .version scheme is applied to our current testing/dev systems via dietpi-update -1 and on somehow broken update attempts.
• On patch_file, obsolete pre-v6.10 dietpi-survey call with version adjustments was removed to fix otherwise broken/doubled pre-v6.17 patch. Survey call is anyway guaranteed when new dietpi-update is rerun.

Further tweaks I am thinking about:

    G_VERSIONDB_SAVE(){

        cat << _EOF_ > /DietPi/dietpi/.version
#!/bin/bash
export G_DIETPI_VERSION_CORE=$G_DIETPI_VERSION_CORE
export G_DIETPI_VERSION_SUB=$G_DIETPI_VERSION_SUB
export G_DIETPI_VERSION_RC=$G_DIETPI_VERSION_RC
export G_GITBRANCH=$G_GITBRANCH
export G_GITOWNER=$G_GITOWNER
_EOF_

        chmod +x /DietPi/dietpi/.version

        # - Auto reload
        . /DietPi/dietpi/.version

    }

• Isn't writing and then reassigning the exactly same variables somehow doubled effort? The only difference the re-sourcing has is the export. But actually export for those values should be never needed from within globals, since anyway every script that needs them, runs globals again.
• Personally I would reduce export to the vars, where it's really needed:
  
  
  
  • Those which are used by 3rd party commands: LANG/LC_ALL/TERM
  
  
  • And those, we do not assign in globals, outside of functions, and use for output/check in subscripts: HIERARCHY/G_CHECK_ROOT/G_BACKUP_DISABLED/G_DIETPI_SERVICES_DISABLE/G_USER_INPUTS=0
  
  
  • I don't see any other var that needs to be exported. Also G_WHIP_RETURNED_VALUE does not need to be exported, since we do not use these values in sub scripts without giving them as arguments?
  
  
  • G_PROGRAM_NAME actually also does not need to be exported. It is reset to empty string anyway during dietpi-globals load. Only reason could be, if we call a sub script that does not load globals but uses parent program name somehow. But G_DIETPI-NOTIFY/G_WHIP e.g. is not available then to call it.
  
  

[Fourdee]

@MichaIng

Isn't writing and then reassigning the exactly same variables somehow doubled effort?

Yep, not sure why I did that lol 👍

Personally I would reduce export to the vars, where it's really needed:

Agree, will send commit.

[Fourdee]

Beta system now live, we can mark this as closed.