Linuxgsm: Older versions of curl are unable to download large files using newer SSL/HTTP technologies
User Story
As a [user description], I want [desired action] so that [desired outcome].
https://linuxgsm.com/lgsm/cod4server/
Basic info
- Distro: [Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-22-generic x86_64)]
- Game: [COD4]
- Command: [install]
Further Information
A clear description of what the bug is and any ideas on how to resolve it. Plus any further context that might be relevant to the issue.
To Reproduce
add user follow directions on website and type ./cod4server install
This is where the error occurs.
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
downloading cod4x18_1772_dedrun.tar.bz2...OK
verifying cod4x18_1772_dedrun.tar.bz2 with MD5...FAIL
cod4x18_1772_dedrun.tar.bz2 returned MD5 checksum: 809b16e34396d6ef588d28748962dc3f
expected MD5 checksum: de29f29d79f9cc24574b838daa501e46
fetching core_exit.sh...OK
Here is logs:
./cod4server install
cod4server@games:~$ ./cod4server install
fetching core_functions.sh...OK
fetching core_legacy.sh...OK
fetching core_messages.sh...OK
fetching core_dl.sh...OK
fetching core_trap.sh...OK
fetching _default.cfg...OK
copying _default.cfg...OK
fetching common.cfg...OK
fetching cod4server.cfg...OK
fetching linuxgsm.sh...OK
fetching core_getopt.sh...OK
fetching command_install.sh...OK
fetching check.sh...OK
fetching check_tmuxception.sh...OK
fetching check_permissions.sh...OK
fetching check_glibc.sh...OK
fetching info_distro.sh...OK
fetching check_system_requirements.sh...OK
fetching install_header.sh...OK
mdMMMMbm
mMMMMMMMMMMm
mMMMMMMMMMMMMm
mMMMMMMMMMMMMMMm
hMMMV^VMMV^VMMMh
MMMMM MM MMMMM
hMMs vv sMMh
hMMM: :MMMh
.hMMMh hMMMh.
-dMMMh __ hMMMd-
:mMMMs || sMMMm:
:MMMM+ || _ +NMMN:
.mMMM+ ======== +MMMm.
yMMMy ############## yMMMy
mMMM: ############## :MMMm
mMM nn nn nn nn MMm
o nNNNNNNNn nNNNNNNNn o
nNNNNNNNNNn nNNNNNNNNNn
nNNNNNNNNNNN NNNNNNNNNNNn
+NNNNNNNNN: :NNNNNNNNN+
nNNNNNNN /\ NNNNNNNn
nnnnn db nnnnn
888 d8b .d8888b. .d8888b. 888b d888
888 Y8P d88P Y88b d88P Y88b 8888b d8888
888 888 888 Y88b. 88888b.d88888
888 888 88888b. 888 888 888 888 888 Y888b. 888Y88888P888
888 888 888 88b 888 888 Y8bd8P 888 88888 Y88b. 888 Y888P 888
888 888 888 888 888 888 X88K 888 888 888 888 Y8P 888
888 888 888 888 Y88b 88Y .d8pq8b. Y88b d88P Y88b d88P 888 * 888
LinuxGSM 888 888 888 Y8888Y 888 888 Y2012P88 Y8888P 888 888
=================================
LinuxGSM_
by Daniel Gibbs
Game:Call of Duty 4
Website: https://linuxgsm.com
Contributors: https://linuxgsm.com/contrib
Donate: https://linuxgsm.com/donate
=================================
fetching install_server_dir.sh...OK
Server Directory
=================================
/home/cod4server
Continue? [Y/n] Y
mkdir: created directory '/home/cod4server/serverfiles'
fetching install_logs.sh...OK
Creating log directories
=================================
installing log dir: /home/cod4server/log...OK
installing LinuxGSM log dir: /home/cod4server/log/script...OK
creating LinuxGSM log: /home/cod4server/log/script/cod4server-script.log...OK
installing console log dir: /home/cod4server/log/console...OK
creating console log: /home/cod4server/log/console/cod4server-console.log...OK
installing game log dir: /home/cod4server/serverfiles/Logs...OK
creating symlink to game log dir: /home/cod4server/log/server -> /home/cod4server/serverfiles/Logs...OK
fetching check_deps.sh...OK
Checking Dependencies
=================================
tmux
wget
ca-certificates
file
bsdmainutils
util-linux
python3
tar
bzip2
gzip
unzip
binutils
bc
jq
lib32gcc1
libstdc++6:i386
Information! Required dependencies already installed
fetching install_server_files.sh...OK
Installing Call of Duty 4 Server
=================================
##O=# # ########################## 19.4%
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
downloading cod4x18_1772_dedrun.tar.bz2...OK
verifying cod4x18_1772_dedrun.tar.bz2 with MD5...FAIL
cod4x18_1772_dedrun.tar.bz2 returned MD5 checksum: 809b16e34396d6ef588d28748962dc3f
expected MD5 checksum: de29f29d79f9cc24574b838daa501e46
fetching core_exit.sh...OK
End of Logs
Steps to reproduce the behaviour:
- After fresh user created.
- su - NewUsername
- wget -O linuxgsm.sh https://linuxgsm.sh && chmod +x linuxgsm.sh && bash linuxgsm.sh cod4server
- ./cod4server install
- Error occurs through the installation.
Expected behaviour
Installation of cod4.
lorentedford
All 55 comments
Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.
![issue-label-bot[bot] picture](https://avatars2.githubusercontent.com/in/27079?v=4&s=40)
issue-label-bot[bot]
on 17 Nov 2019
Looks like the download didnt complete so you have a corrupted download. you will need to delete the download and try again
dgibbs64
on 17 Nov 2019
Looks like the download didnt complete so you have a corrupted download. you will need to delete the download and try again
I have done this twice.. Looks like its an installation script issue.
lorentedford
on 17 Nov 2019
Just tested this myself with no issues. The download is large at 4GB in size. The curl error shows the download failed at 19%. The download simply didn't complete.
##O=# # ########################## 19.4%
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
Here are my results.
Installing Call of Duty 4 Server
=================================
################################################################################################################################################################################################################################# 100.0%
downloading cod4x18_1772_dedrun.tar.bz2...OK
verifying cod4x18_1772_dedrun.tar.bz2 with MD5...OK
Trust me this isn't adding up.
verifying cod4x18_1772_dedrun.tar.bz2 with MD5...FAIL
cod4x18_1772_dedrun.tar.bz2 returned MD5 checksum: 809b16e34396d6ef588d28748962dc3f
expected MD5 checksum: de29f29d79f9cc24574b838daa501e46
Your error clearly states the download failed at 19.4%. The md5 sum WILL fail with a file that's only 19% downloaded. For whatever reason, the connection dropped and the download failed
Here is your original error see where it stated 19.4%
##########################19.4%
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
downloading cod4x18_1772_dedrun.tar.bz2...OK
verifying cod4x18_1772_dedrun.tar.bz2 with MD5...FAIL
cod4x18_1772_dedrun.tar.bz2 returned MD5 checksum: 809b16e34396d6ef588d28748962dc3f
expected MD5 checksum: de29f29d79f9cc24574b838daa501e46
fetching core_exit.sh...OK
Clearly it has to be the server where its being downloaded possibly?
YouTube
This is in response to a bug i posted with the Linuxgsm community the user didn't believe me that I was working with a fresh user account and installation. I...
lorentedford
on 17 Nov 2019
My monitoring software has reported no outages or downtime for over a week. I host all the files on my dedicated server. I have tested with my own test servers with no issues. It appears that your server connection is dropping the download for some reason. Is the server hosted on a local network or the internet? As it might be a poor connection. Here is the url if you want to download the using a browser https://files.linuxgsm.com/
dgibbs64
on 17 Nov 2019
The server is hosted in a data center environment even I have to connect to it from the outside.. We are doing a full fresh re-installation of this box. At this time I will keep you posted.
lorentedford
on 17 Nov 2019
Even with a fresh installation of Ubuntu 18.04 LTS this fails
############################################# 20.0%
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
downloading cod4x18_1772_dedrun.tar.bz2...OK
verifying cod4x18_1772_dedrun.tar.bz2 with MD5...FAIL
cod4x18_1772_dedrun.tar.bz2 returned MD5 checksum: 4ba4c1d95d5612e0e3c3b3851a2480e8
expected MD5 checksum: de29f29d79f9cc24574b838daa501e46
fetching core_exit.sh...OK
Oddest thing ever.
lorentedford
on 17 Nov 2019
So I have tried installing this on my server at home at my home network which i know is up and working correctly I got this...
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
downloading cod4x18_1772_dedrun.tar.bz2...OK
verifying cod4x18_1772_dedrun.tar.bz2 with MD5...FAIL
cod4x18_1772_dedrun.tar.bz2 returned MD5 checksum: c63d303be76f8aebdba4258865525199
expected MD5 checksum: de29f29d79f9cc24574b838daa501e46
fetching core_exit.sh...OK
cod4server@server3:~$
I have now verified that this is an issue with either your connection where the files are located or an issue with some thing posted on on your cod 4 installation page maybe its downloading an outdated version of linuxgsm when i run the commands idk..
Here is the directions i have followed.
https://linuxgsm.com/lgsm/cod4server/
Here is the video of the dedicated box re installation from a fresh install Its at 2x the speed for you.
LinuxGSM_LinuxGSM is the command-line tool for quick, simple deployment and management of Linux Call of Duty 4 dedicated game servers.
This is a full format reinstall of an Ubuntu 18.04LTs box still failed to install cod4 linuxgsm.. Here is where the posting has taken place. https://github.c...
lorentedford
on 18 Nov 2019
@lorentedford The issue here is rather a network issue than anything else.
What about trying with iptables off?
Also, what server provider are you using and which country is your server located in? Maybe runing some "ping" commands to the file hosting server would provide us some useful info.
UltimateByte
on 18 Nov 2019
Potential issue related to fail2ban blocking Cloudflare IP addresses. I have made a change. Please test again.
dgibbs64
on 18 Nov 2019
@lorentedford The issue here is rather a network issue than anything else.
What about trying with iptables off?
Also, what server provider are you using and which country is your server located in? Maybe runing some "ping" commands to the file hosting server would provide us some useful info.
Well since this happened with my home data center servers I this is not the issue. Besides iptables wouldn't affect downloads by default even on a fresh install.. You would have to program that into iptables..
Now..
Potential issue related to fail2ban blocking Cloudflare IP addresses. I have made a change. Please test again.
No this didn't work on a fresh install again. I have dedicated business class fiber with five static ip's at my data center here in my basement where I actually host web and email servers along with many other gamer servers that run linuxgsm with out any issues. I also play with asterisk phone systems and many other projects in the amateur radio community.. I can assure you this is a download issue from your servers.. Especially since I have verified this on an outside data center that is not at or near my network.
Here are my static ip ranges. 157.185.79.34 to 157.185.79.38. According to pf-sense its pulling from 157.185.79.38 non of my other projects have any issues and my download stream has been solid over the past 24 hrs to our gateway.
Here is what Pfsense is showing on traffic information over the past 8 hours.
From what i can see on the router side is that some thing from the download server side disconnects us.
lorentedford
on 18 Nov 2019
Potential issue related to fail2ban blocking Cloudflare IP addresses. I have made a change. Please test again.
Please read my above posts and i am off to bed will be up anywhere between 6 to 8 hours from now I will check my email through out my sleep to see If i get another notice from github of reply on this.. Chat in the morning
Best Wishes
lorentedford
on 18 Nov 2019
Just to be clear. I have identified a potential issue on my server. It appears fail2ban is blocking Cloudflare IP addresses (instead of the offending bad IP address) This might cause connections to be dropped. I have disabled the fail2ban jail that is causing this so the issue might be resolved. Please test again to see if this has helped.
A potential solution that I will investigate later to allow log files to see original visitor IPs addresses that will allow fail2ban to work correctly and not block Cloudflare IP ranges
dgibbs64
on 18 Nov 2019
Just to be clear. I have identified a potential issue on my server. It appears fail2ban is blocking Cloudflare IP addresses (instead of the offending bad IP address) This might cause connections to be dropped. I have disabled the fail2ban jail that is causing this so the issue might be resolved. Please test again to see if this has helped.
A potential solution that I will investigate later to allow log files to see original visitor IPs addresses that will allow fail2ban to work correctly and not block Cloudflare IP ranges
Nope Its still denying it..
I do have to wipe the directory and start fresh each time when I go to install..
Here is a screen shot..
Cloudflare SupportLearn how to configure mod_cloudflare to log your visitor鈥檚 original IP address based on your origin web server type (including Apache, nginx, Microsoft IIS and others). Cloudflare no longer update...
lorentedford
on 18 Nov 2019
some time ago i tried to do a "hack" to do a auto resume of the download.
https://github.com/h3o66/LinuxGSM/commit/ff48b8dee686a30dea06af7967980e2bb4ec4884
Do you think this could help as addition for lgsm ?
h3o66
on 18 Nov 2019
@h3o66 yeah something ensure that the download retries would be good. A potential solution will need to be investigated. I will be doing maintenance and testing to make sure that the potential solution resolves the problem.
dgibbs64
on 18 Nov 2019
@h3o66 yeah something ensure that the download retries would be good. A potential solution will need to be investigated. I will be doing maintenance and testing to make sure that the potential solution resolves the problem.
A linode server is like 5 dollars a month to test it on.. Just saying.. I wouldn't use it to necessarily host the server on but installation wise it should work and tell you if you found out why the server doesn't like to download and install correctly.
lorentedford
on 18 Nov 2019
@lorentedford Linode already sponsor me with a free account ;-).
dgibbs64
on 18 Nov 2019
@lorentedford Linode already sponsor me with a free account ;-).
Well there ya go :) So what's wrong with the downloading from your server? Do you have any ideas yet?
lorentedford
on 18 Nov 2019
Keep in mind i have tried on several systems here in our network and at the data center network and still no go on downloading and installation its so odd...
lorentedford
on 18 Nov 2019
Are my ip's blacklisted?
lorentedford
on 18 Nov 2019
Yes I'm working on a fix as we speak. I think some Cloudflare's IP are being blocked by my server which users proxy though
dgibbs64
on 18 Nov 2019
Yeah personally I haven't been a fan of cloud flare services yet.. Just my thoughts.
lorentedford
on 18 Nov 2019
Right, I think things might have settled down. I have reconfigured fail2ban and it now correctly sends bans to the cloudflare firewall. This will have to be monitored.
dgibbs64
on 18 Nov 2019
Well this server on my network failed..
Here are my static ip ranges. 157.185.79.34 to 157.185.79.38.
And the one at the data center eventually failed as well at 29%
lorentedford
on 18 Nov 2019
Found the request in the logs and it's not being blocked. Even shows me how far the download got. I no longer think the issue is with fail2ban blocking anything. I'm stumped with this one currently, especially since I can't replicate this.
Whats possibly non-standard with your server is you are using link aggregation across 4 NIC's. Maybe the file transfer has an issue with that...but I'm just guessing as it has been a while since I have used such a configuration. Possibly doesn't like the particular bonding mode as there are 7 different modes. :man_shrugging:
https://www.interserver.net/tips/kb/network-bonding-types-network-bonding/
Interserver TipsNetwork bonding is a process of combing or joining two or more network interfaces together into a single interface. Network bonding offers performance improvements and redundancy by increasing the network throughput and bandwidth. If one interface is down or unplugged the other one will work. It can be used in situations where you need fault
dgibbs64
on 18 Nov 2019
Let m boot up a linode really quick and try one from linode and see what happens.. Again if its not working on my personal data center network or the dedicated box and you can see where both ips pulled for the data example my server from my personal data center should have been pulling from 157.185.79.38 and not sure on the data center side.. Can you tell me if you see the 157.185.79.38? Why does the server drop this connection?
lorentedford
on 18 Nov 2019
I created a linode vps really quickly to try out your theory that it might be on both of my networks this was spun up literally like 10 minutes ago as stated and it failed.. Here is the ip of my linode server.
198.58.123.110 and 2600:3c00::f03c:92ff:feb0:3d0e
This re affirms my theory that its on your network or your server connections.
See if there is any logs of this ip trying to connect up to your server.
lorentedford
on 18 Nov 2019
Possible lead. Tested on 7 different distros and the download only worked on newer versions of curl, researching found a possible issue with curl using certain types https encryption. Bed for me now but I will continue investigating this. I think I will have to tweak https settings for compatibility.
- CentOS 8 - curl 7.61.1 - PASS
- CentOS 7 - curl 7.29.0 - FAIL
- Debian 10 - curl 7.64.0 - PASS
- Debian 9 - curl 7.52.1 - FAIL
- Debian 8 - curl 7.38.0 - FAIL
- Ubuntu 18.04 - curl 7.58.0 - FAIL
- Ubuntu 16.04 - curl 7.47.0 - FAIL
dgibbs64
on 18 Nov 2019
probobly has outdated Certificate Authorities in the older versions of cURL......try it with a newer version of cURL from backport repos.
markie83
on 19 Nov 2019
ok turns out there is not a backport for cURL on 18.04 so I manually added the ppa for 18.10
deb http://ppa.launchpad.net/xapienz/curl34/ubuntu cosmic main
deb-src http://ppa.launchpad.net/xapienz/curl34/ubuntu cosmic main
and then installed curl from there and then it worked.........not something I really wanted to do in a production environment. Ubuntu REALLY needs to do a backport for 18.04 LTS
markie83
on 19 Nov 2019
ok turns out there is not a backport for cURL on 18.04 so I manually added the ppa for 18.10
deb http://ppa.launchpad.net/xapienz/curl34/ubuntu cosmic main
deb-src http://ppa.launchpad.net/xapienz/curl34/ubuntu cosmic mainand then installed curl from there and then it worked.........not something I really wanted to do in a production environment. Ubuntu REALLY needs to do a backport for 18.04 LTS
Yes this actually worked
Waiting for @dgibbs64 to respond before i do this i don't want to mess this up on my friends server that I am installing for him..
lorentedford
on 19 Nov 2019
Yup updating curl would be a good fix for this. I will, however, look at making changes to my webserver to ensure compatibility with older versions of curl.
dgibbs64
on 19 Nov 2019
Yup updating curl would be a good fix for this. I will, however, look at making changes to my webserver to ensure compatibility with older versions of curl.
Me and Markie talked about this last night this should have been back ported because this is going to cause issues down the line.. Markie is my boyfriend I brought him over to my case because I wanted to make sure I wasn't missing anything..
lorentedford
on 19 Nov 2019
I agree curl should be backported. Especially with the speed that new web technologies and certificate authorities are changing. Looking through the settings I think the easiest and most reliable option is to revert to HTTP. I would prefer to use https but since there is no sensitive data being transferred I think it makes sense in this case.
So, in summary, the quick fix is to upgrade curl to a newer version. On my end, I will make changes to move to HTTP, which I will make ready for the next release.
dgibbs64
on 19 Nov 2019
Well now we are back to this again...
lorentedford
on 19 Nov 2019
@dgibbs64 What did you do from yesterday?? Because now I can't download the files either way..
Just trying to figure out what happened...
lorentedford
on 19 Nov 2019
Now this is the same server that installed everything just fine yesterday after we modified with the latest curl.
Here is what is happening now after creating a new user and running the commands.
wget -O linuxgsm.sh https://linuxgsm.sh && chmod +x linuxgsm.sh && bash linuxgsm.sh cod4server
./cod4server install
lorentedford
on 19 Nov 2019
Ok so where does that leave us?
lorentedford
on 21 Nov 2019
i think its related to cloudflare's compatibility with older software, try turn off http2 and tls 1.3 in CF's website config
Frisasky
on 22 Nov 2019
Hey this problem still exist 12-1-2019
Why is this closed????
lorentedford
on 1 Dec 2019
did u updated to latest version of lgsm?
Frisasky
on 1 Dec 2019
Yes
lorentedford
on 1 Dec 2019
did u updated to latest version of lgsm?
Did you update your website??
Here is my reinstall script since i got tired of making user accounts..
#!/bin/sh
cd ~
rm -R ./cod4server ./lgsm/ ./linuxgsm.sh ./log/ ./serverfiles/
sleep 2s
wget -O linuxgsm.sh https://linuxgsm.sh && chmod +x linuxgsm.sh && bash linuxgsm.sh cod4server
sleep 4s
./cod4server install
I don't appreciate being treated like i don't know shit.. I get it working one time give you guys the information then you all break it.. Oh then close the ticket after I have been waiting patiently for over a week..
Here is what i am going off of... https://linuxgsm.com/lgsm/cod4server/
It would have been nice if you guys tagged me in my own ticket and asked me if the issue was fully resolved instead of just assuming that the problem is fixed and moving on..
lorentedford
on 1 Dec 2019
kindly asking can you join our discord server so we could perform better diagnostic
https://linuxgsm.com/discord
Discord
Check out the LinuxGSM_ community on Discord - hang out with 1,674 other members and enjoy free voice and text chat.
Frisasky
on 1 Dec 2019
kindly asking can you join our discord server so we could perform better diagnostic
https://linuxgsm.com/discord
I am in the general chat and waiting on a video to finish rendering with my fresh install
https://youtu.be/re3enYFImjc
Check out the LinuxGSM_ community on Discord - hang out with 1,674 other members and enjoy free voice and text chat.
So I have been battling it out with linuxgsm at this point my ticket has been open for over 13 days and I had a solution until they broke some thing back end...
lorentedford
on 1 Dec 2019
Further testing is ongoing with this issue. Using http has helped improve reliability but is not currently working all the time. I have found a way for curl to continue if the connection drops for any reason so I will be adding that feature to the curl command. Testing is ongoing to confirm if cloud flare is causing a connection issue or not but is very hard to confirm either way.
dgibbs64
on 1 Dec 2019
This problem was resolved as of Mon Dec 2 19:33:03 UTC 2019 Thank you so much for all your hard work! I do still notice a performance issue with the download but at least it works. Thanks again! However I do notice that it disconnects and reconnects to finish downloading but its a slow process. I still think there is some thing underlining that is going on maybe even server side..
lorentedford
on 2 Dec 2019
Removal of cloudflare caching and reverting to http has resolved the issue. Methods to improve downloads for non-steam games will need to be investigated moving forward
dgibbs64
on 6 Dec 2019
theres some other CDN providers around, maybe have a try on them?
Frisasky
on 6 Dec 2019
@Frisasky KeyCDN offer a free account for open source. They might be what we are after.
dgibbs64
on 6 Dec 2019
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
![github-actions[bot] picture](https://avatars2.githubusercontent.com/in/15368?v=4&s=40)
github-actions[bot]
on 5 Jan 2021
Related issues
Septembers
路
3Comments
Khelgar1
路
3Comments
UltimateByte
路
4Comments
n0x-io
路
3Comments
Audrik
路
3Comments