October: Implement file based build number in dist code

[adrenth]

I have the following situation:

All our OctoberCMS projects are in version control. We update the OctoberCMS instances with composer update or php artisan october:update and commit the changes to the projects' repo. Since we don't want our clients to update the CMS by themselves and cause issues related to that update.

A) My project on production environment has build number 292.
B) My local environment has build number 292 and the latest build is 309.

My update procedure:

• Update local environment using composer and/or artisan command (After installing Core updates the system_parameters table is automatically updated with a new hash and build number).
• Commit changes to repo, test it, build it, tag it, deploy it

When I logon to the production backend and navigate to the Updates section, there's still Build number 292. The files have changed and it is build 309. Of course the system_parameters table hasn't changed.

I don't think OctoberCMS has a solution yet for this issue. Or did I miss something?

What to do? Just manually update the system_parameters table?

What could be a solution for this issue? What's your experience with this?

[shina]

Doesn't the october:up update it?

[adrenth]

No. It does'nt.

It happens only in the UpdateManager after te core package is extracted and installed.

[daftspunk]

Interesting! We may need to include a file based build number in future.

[adrenth]

Exactly! Perfect, thanks for making it a feature request!

[petehalverson]

+1

I'd like our docker image tags to line up with official build numbers. I am willing to help with this issue if I can get me some info on how build numbers relate to this repo.

Thanks!

[adrenth]

Recently each build is tagged in the october repo. But how about this one?

• Where to place this Build number file? Should we put a file called BUILD in root path? Which contains the build number and build hash?
• october:up will check/validate this file and update these system parameters in the database.

What are your thoughts about this @daftspunk / @alekseybobkov ? I'm happy to contribute on this one.

[daftspunk]

Can the build now be extracted from the composer.lock file perhaps?

[adrenth]

That might be a solution, but...

We (and I can imagine somebody else) exclude this kind of files from our builds. Since composer.lock is not required to put on the server after our project was build. So the system cannot determine what the latest build is.

Also composer.lock is a relatively large file to parse.

That may also be true for the ./BUILD file... Or maybe we should create a system.yaml (or whatever name; like the /plugins/foo/bar/plugin.yaml file) which can contain some more (user specific) information:

````
build:
number: 390
hash: 305de721d2f8e9580ef858a799dea70b
time: 2016-12-16 23:16:02

user specific data

version: x.y.z
myVariable: myValue
````

Suggestions, ideas?

[adrenth]

Also an issue (related to the build number) could be solved by implementing a file based build number.

A website is running on a old build 318 and is upgraded to the latest version (composer update) and is deployed to the server. Scripts and stylesheets are suffixed with ...storm.css?v318. But the build number in the database has not been changed. So the suffix stays at ?v318 even if the build is 392.

As a result of this the styling changes to the backend (and/or even frontend) will not change on the clients browser due to good caching mechanisms.

[daftspunk]

This can be solved by introducing a flag to the october:update command

php artisan october:update --build-only

If the API server is "pinged" it will return the latest build number:

Http::post('http://gateway.octobercms.com/api/ping');

This command can be called as part of the post update scripts in composer and should set it in the database, or fail gracefully (if database is not available).

[adrenth]

I think that is a very elegant solution to this "problem"! And what about the hash, is that field related to the build number?

[daftspunk]

Good question, I think the hash needs to be stored as well otherwise the latest build number could still prompt an update.

[petehalverson]

I'd advocate for a file-based reference. Seems like it'd avoid corner case issues when using the CMS with flat files or instances where a database isn't available on install.

[petehalverson]

Either way, I'd be happy to tackle this. Should I use http://gateway.octobercms.com/api/core/update or is there another way I can get the hash?

I'm already doing something similar: https://github.com/aspendigital/docker-octobercms/blob/master/update.sh#L18-L31

[daftspunk]

We may need to improve the ping API to include the hash as well as the build number. If we are going down the path of a file-based reference, what would that look like? Where would it be stored and what do the contents look like?

Feel free to make a proposal, @petehalverson

[LukeTowers]

@petehalverson Still interested in working on this?

[petehalverson]

Since tags were introduced for the repository, we automated our docker image build process to follow, which mitigated the issue for us.

That said, I'm willing help.

I like @adrenth's suggestion of a system.yaml file storing the build and hash. Would that work for you?

Once introduced, should the system::core.build parameter be deprecated?

[adrenth]

The current modules/system module has a Parameter model which is used for storing and fetching system parameters. Which is OK.

This is just some brainfarting:

$file = base_path() . '/system.yaml';

system: core: build: 419

We could introduce a SystemManager singleton which is responsible for storing/fetching values from this specific file and exposes some methods which can be used throughout the whole application.

Uses the October\Rain\Support\Traits\KeyParsers.
Uses the Yaml facade.
Uses the Parameter model for storing it into the database (if available).

$manager = SystemManager::instance(); $manager->getCoreBuild(); // fetches system.core.build from system.yaml file $manager->updateDbParameters(); // updates the database Parameter::set('system::core.build', $this-getCoreBuild());

This manager can update the build number to the database if a database is available...

[petehalverson]

Sorry @adrenth, I should've been more specific. Should we ditch the parameter in the database if it's always going to exist in the codebase moving forward?

[adrenth]

If you'd ask me, I'd ditch the whole parameter Eloquent Model thing. I'm not really fond of storing variable system parameters to a database. I'd prefer OctoberCMS' state to be stored in a single (or more) file, like composers' lock file for example. In most PHP applications I develop I store most of this information in files, mainly because this reason: It allows you to track the changes made to the system (codebase) in GIT (or any other VCS). And maybe revert to a previous state of your OctoberCMS instance. The database is not always in sync with your codebase or doesn't even exist (as mentioned before).

After performing a composer update your composer.lock file changes and if any changes to OC's core were done the system.yaml changes also. Commit it to your VCS and you're done. So besides the system::core.build I think it's a great idea to store all such parameters to this $file = base_path() . '/system.yaml'; file. But I'm really curious what the founders of OC think of this idea... So @daftspunk @alekseybobkov what are your thoughts about this?

[daftspunk]

This has been addressed partially in 78d722adf016ae4146a99816097f9e053c981ec7 by setting the build number post composer update. We can look at injecting a file in to /storage/system.json with the build number as part of the build process.

edit: Confirmed. This should be included in the next build (420+) as a file found in storage/system.json, containing the build number.