Vega-strike-engine-source: Packaging

[BenjamenMeyer]

This needs to be done for 0.6.x so that we can make the release; but we'll have to do the same to master for 0.7.x too...

We need to figure out how to package Vega Strike for our various supported platforms. We can add support for each as we bring it up to snuff, so for 0.6.x we only need Linux packages for the initial release since at this point we only have Linux figured out for builds and testing.

For 0.6.x we need:

• [x] DEB packages (source, binary)
• [x] RPM packages (Red Hat; source, binary)
• [x] Bzip'd Tarball (Source)
• [x] Zip file (Source)

Tarballs and Zip files can generally be easily done by:

$ tar -jcf vegastrike-<version>-source.tar.bz2 .

We can exclude the .git directory using the -X parameter. Zip has similar functionality.

[BenjamenMeyer]

We're using cmake to build the binaries; their tooling also provides a tool called cpack to build installers. It'll build a lot of various things that hit all our target platforms, so it may make for an easy win to get everything. We'll need to investigate it more.

That said - we may still want to maintain our own NSIS or WiX installers for Windows so we can have sufficient control.

https://cmake.org/cmake/help/latest/module/CPack.html#cpack

[Loki1950]

We could also set up a ppa for Ubuntu and derivatives which is essentially a private repo we maintain adding a source Deb to it would endear us to various distro maintainers. Actually three packages the executable, data. music

[pyramid3d]

klauss once had an opensuse build account that can build for different distros, if I recall correctly.

[BenjamenMeyer]

@pyramid3d so we probably should add a version of OpenSuSE to our list of target systems. Do note that SuSe RPMs are not compatible with Red Hat RPMs (different dependency chains among other things; the format itself is the same) so we'd have to have both versions of RPMs.

I only see OpenSuSE targets in that link. Am I missing something?

[pyramid3d]

I only see OpenSuSE targets in that link. Am I missing something?

Probably not. Other distros were maybe removed...

[BenjamenMeyer]

135 gives us a lot of the packaging support we need. I can try to figure out dependencies for Ubuntu and detecting Ubuntu vs Debian in that PR; but we'll need to follow-up with other PRs for RPM-based distros. MacOS X and Windows packages are also started there, and would need significant work to bring them up-to-par, but as I said - it's a start.

[BenjamenMeyer]

Updated #135 with several aspects:

• Added a script to build the packages, at least on Unix where we can use Shell scripts.
• Switched the default build to Release with Debug if not build type is specified; IIRC @ermo noted that distros prefer that build.

The PR does give us the ability to script out the package builds, but the questions remain:

• [ ] how often do we want to build the packages? Every build? On release? etc.
• [ ] where and how do we store packages build by the CI system?
• [ ] do we want to and how would we finish this with a deploy method?

Those questions shouldn't be resolved in #135. We can answer them in discussion here, and then add additional PRs as required to finish it out.

[nabaco]

I'll answer all there questions with this: https://docs.travis-ci.com/user/deployment/

Deploy on releases, into Github releases, using Travis Deploy. As was mentioned in #104, we need to figure it out, as it requires some tinkering.

[nabaco]

I don't think our development is fast enough right now for nightly builds

[BenjamenMeyer]

135 accomplishes quite a bit of what we need; though some things may need to get delayed until the deploy portion. I've taken it as far as I can for the moment. So would like to get it merged. We can figure out next steps as part of peer review, etc.

One thing we'll need to consider with the deploy (CD) phase is how to distinguish between Debian/Ubuntu releases (f.e Ubuntu 20.04, Debian 8, Debian 9, etc) as the package names will be the same from the build system but the dependencies will not be.

[evertvorster]

I just spent a few hours trying to figure out where this program expects all it's files to go. Most distro maintainers will do the packaging for you, if it is not too much of a pain.
I love space sims in general, and played VegaStrike some years ago, so I was overjoyed to see some action on the sources again.

I constructed a couple of AUR packages for Arch Linux, so you guys might get a few more users.

Unfortunately, the make or cmake did not have an install target, so I had to reverse engineer the install scripts to figure out what goes where, and quite frankly there are still quite a number of files that the supposed location is unclear.

[BenjamenMeyer]

@evertvorster can you file an issue outlining what you found? I'm just getting into the packaging of VS for the upcoming 0.6.x release. (Master is 0.7.x).

There's some work to do for sure; working to separate out our data and engine in a useful manner so the engine can be truly used as an engine.

Also, we're on gitter.im if you want to live chat

[BenjamenMeyer]

@evertvorster FYI - created https://gitter.im/vegastrike/vegastrike-packaging?utm_source=share-link&utm_medium=link&utm_campaign=share-link so you and others don't have to wade through all the conversations and can focus on packaging details.

[BenjamenMeyer]

Added #147 for packaging for 0.6.x and #148 to fix a bug in the package version on master.

It'd probably be good to add a means that the Packaging could extract the version from the binaries or another source. We should get to where we only have to update one place and have it be injected everywhere else required.

[nabaco]

@evertvorster for the AUR package, I think you can clone the repos with --depth=1 and --single-branch, so the download size will be much smaller.
I think all the AUR users will appreciate the faster download :)

[evertvorster]

Hey nabaco, some more info on how software makes it into Arch.

The intent of the -git packages in AUR is that each developer ends up with the entire git repo. There might be a way to do shallow clones, but it causes issues further down the line. This is the very initial phase of getting new software into Arch. (There be dragons) Git is intended for developers that want to test new features or fixes as the software is developed. Luckily, the repo is cloned only once, then just updated whenever the user wants to re-build the package.

Of course, once VS does do a tarball release, then a separate packages will be made for them in the AUR, dropping the -git part, but borrowing the build instructions from the git packages. While the user still has to download & build the software, the tarballs are much smaller downloads, and not intended for developers, as they don't change between releases.

Finally, if enough users like the package, someone (not me) will pre-build and host packages on Arch Linux servers.

Other distributions have a range of ways to accomplish the same thing.

My initial point was just that the engine repo is suspiciously large, like there was data in there at some point.
TL;DR The packages I created only makes the software easily available, the trade-off is that it is a large download, and may be unstable.

[BenjamenMeyer]

@evertvorster there may have been data in there long ago. @Loki1950 or @danielrh could answer that question.

Thanks for the info on getting stuff into Arch.

[Loki1950]

The large size may be an artefact of close to 15 years of history on svn and some changes in the svn branches I never had commit access to svn so my memory on this is fuzzy a best

[evertvorster]

Update:
Adding an install target for the binaries of the engine was simple. I have made small patch files for the Arch package, and tested it. Now I am looking at a good way to do an install of the data files to have them all in the right places. Making a makefile that has just the install target might be a way to go. I'll test on my system and come back to you guys. Is there any interest in the patch files from you guys?

[BenjamenMeyer]

@evertvorster certainly...you can email a patch file to one of us or submit a PR. preferably a PR, but either will do.

[evertvorster]

Many thanks for your patience as I get up to speed here. I don't know how to do a PR, if there is some documentation on it, a link would be appreciated. For emails, I would need an address to send it to.

However, this fix is almost comically small.
in the CMakelists.txt file for vegastrike binary, it needs the line:
INSTALL(TARGETS vegastrike DESTINATION bin)

For vegasettings:
INSTALL(TARGETS vegasettings DESTINATION bin)

And for the mesh tool:
INSTALL(TARGETS mesh_tool DESTINATION bin)

The effect of this is that if you were to run "make install" It would install those files in your systems' preferred bin directory. On Arch this is /usr/bin.

Since make install did not work at all before, the impact on other users of the source should be non-existent

[BenjamenMeyer]

@evertvorster GitHub has documentation on making PRs see https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request

We follow a fork model. The basics are:

1. Fork the repository to your github account
2. Clone your fork locally (git clone ...)
3. Create a branch (git checkout -b <branch name>)
4. Make the desired changes
5. Push the changes back to github (git push origin <branch name>)
6. In GitHub, go to our repository and it'll prompt you about creating a PR.

I'll make sure that's in our README/Contributors docs too.

[BenjamenMeyer]

193 will deploy a release to GitHub when we make a release in GitHub - we can verify that's working (yeah!) however, it seems that the binaries are some how being missed from the Debian files. WIll have to dig more into that.

[BenjamenMeyer]

oh - and thanks @nabaco for getting the CD portion working - great job!

[nabaco]

Hi, I think we should remove the source code packages from CPack as Github does that automatically on releases.
We should also fix the .rpm creation.

[stephengtuggy]

RPM creation is almost done. Fedora and openSUSE are done, and I am working on CentOS 8 as we speak.

[stephengtuggy]

CentOS 8 RPM is done. With that, I think maybe we can close out this ticket.

[Loki1950]

Thx for the grunt work on this @stephengtuggy i will take on the task of closing it ;)