Panel: Better way of installing new SRCDS games

[ryanbentley]

Let's use CSGO as an example.

It's a big game, and in the current docker image, SteamCMD will download the entire game. This obviously isn't ideal as it uses finite bandwidth and resources that may affect running servers performance.

IMO a better way of doing it would be to copy the most up-to-date version from the local file system somewhere and use SteamCMD to do updates only, that way even if I don't have the latest SRCDS app downloaded, SteamCMD is downloading small updates rather than 10GB games.

[DaneEveritt]

:+1:

I'm thinking one way of doing this would be having the daemon store the SRCDS games in a specific directory, say /srv/srcds (or whatever, configurable path). We could then use the docker container building to mount the associated directory into the container as a readonly file system. Then, the start script can check that folder (and it might not exist, which is fine, we can just install). If the folder does exist we can then copy over the files, and continue on our way in the container.

Just an idea, and probably one that would work well. Keeping those things in-sync across multiple nodes might be problematic though. For release 0.5.0 of the panel I am planning on having the panel be a repository for all of the services so that any edits or additions to them can be synced across all nodes when daemons reboot. With this change, I could introduce a central file repository as well so that the panel manages one copy of say CS:GO. When the node is setting up, it contacts the panel, and downloads CS:GO from the panel, thereby improving transfer speeds considerably, and letting all nodes have identical setups. From there it would be back to the mounting into the containers.

[ryanbentley]

I like all of these ideas, but perhaps if it was possible to use a file repo that was separate from the panel as well.

How would updates work? Let's say Valve release an update for CSGO today, and we update the file repo, can we then force all nodes to download these new updates, store them in /srv/srcds, and that way all future docker containers or even the running ones, when they try to update, they wont have to update much?

I'm just trying to avoid for example an entire node of SRCDS servers downloading updates. I'd rather the node download an update, and then the out-of-date containers source their updates from the file system.

If you can imagine when a popular game updates, all GSP customers will be downloading and potentially bottlenecking nodes or hindering performance of running servers.

[jffz]

Was thinking about it on my side, maybe it's possible to do something using steamcache on each node and then use this cache localy ..

https://github.com/steamcache/steamcache-dns

[xyeLz]

I actually really like your suggestion above, Dane. Would love to see this implemented as it's certainly something that I will be finding becoming higher and higher priority as more and more servers are added to a single host. This doesn't really affect me for a small game like CS:GO, but for Rust when you're pulling a 6GB image over the WAN, it can definitely make an impact.

[DaneEveritt]

This is closed in favor of installation scripts for right now. If you want to handle cache issues and such, you can modify the scripts to hit your self-hosted Steam bucket.

[shinebayar-g]

Just downloaded 20GB of CSGO image. It was painful lol
Most importantly when I added second server it downloaded 20GB again. Anyone solved this issue?

[parkervcp]

You can always set up your own steamcache to do what someone suggested.