Gitea: Pushing to empty repository is not reflected in web UI
- Gitea version (or commit ref): 1.10.2
- Git version: 2.25.0
- Operating system: ArchLinuxARM
- Database (use
[x]):
- [ ] PostgreSQL
- [ ] MySQL
- [ ] MSSQL
- [x] SQLite
- Can you reproduce the bug at https://try.gitea.io:
- [ ] Yes (provide example URL)
- [x] No: https://try.gitea.io/mxt92822/empty_test
- [ ] Not relevant
- Log gist: https://gist.github.com/kauron/136587e6abf7611481d7461000eb8c4d
Description
Exactly the same as in #9365, but the filesystem is mounted with exec. I think the localhost API calls are the hooks, but I'm not sure. The hooks in the repo run properly when executed manually through the terminal.
(description copied from the previously mentioned issue)
I created a new repo and pushed a local repository. I tried it both with --all (it has two branches) and without (pushed only master). Either way the repository is not changed to non-empty. I did it manually in the DB to set the is_empty column to 0 which fixed the problem.
kauron
All 84 comments
I can't see any calls to pre-receive or post-receive so they're clearly not running.
My suspicion is that your Gitea repositories hooks have lost their executable bit. That means for example not just the files inside pre-receive.d but pre-receive itself
The other possibility is that they're out of date.
What do you mean by saying that they run find when executed from the terminal? Have you managed to properly spoof a pre-receive command?
After a very long time and plenty of scratching of heads it appears the likely solution is:
https://github.com/go-gitea/gitea/issues/9792#issuecomment-691291999
zeripath
on 16 Jan 2020
My suspicion is that your Gitea repositories hooks have lost their executable bit. That means for example not just the files inside pre-receive.d but pre-receive itself
This is the output of ls -lh hooks/ inside one of the gitea repos:
total 76K
-rwxr-xr-x 1 gitea gitea 478 Jan 15 22:44 applypatch-msg.sample
-rwxr-xr-x 1 gitea gitea 896 Jan 15 22:44 commit-msg.sample
-rwxr-xr-x 1 gitea gitea 3.1K Jan 15 22:44 fsmonitor-watchman.sample
-rwxr-xr-x 1 gitea gitea 303 Jan 15 22:44 post-receive
drwxr-xr-x 2 gitea gitea 4.0K Jan 15 22:44 post-receive.d
-rwxr-xr-x 1 gitea gitea 189 Jan 15 22:44 post-update.sample
-rwxr-xr-x 1 gitea gitea 424 Jan 15 22:44 pre-applypatch.sample
-rwxr-xr-x 1 gitea gitea 1.6K Jan 15 22:44 pre-commit.sample
-rwxr-xr-x 1 gitea gitea 416 Jan 15 22:44 pre-merge-commit.sample
-rwxr-xr-x 1 gitea gitea 1.5K Jan 15 22:44 prepare-commit-msg.sample
-rwxr-xr-x 1 gitea gitea 1.4K Jan 15 22:44 pre-push.sample
-rwxr-xr-x 1 gitea gitea 4.8K Jan 15 22:44 pre-rebase.sample
-rwxr-xr-x 1 gitea gitea 303 Jan 15 22:44 pre-receive
drwxr-xr-x 2 gitea gitea 4.0K Jan 15 22:44 pre-receive.d
-rwxr-xr-x 1 gitea gitea 544 Jan 15 22:44 pre-receive.sample
-rwxr-xr-x 1 gitea gitea 283 Jan 15 22:44 update
drwxr-xr-x 2 gitea gitea 4.0K Jan 15 22:44 update.d
-rwxr-xr-x 1 gitea gitea 3.6K Jan 15 22:44 update.sample
The other possibility is that they're out of date.
How could I check that/update them?
What do you mean by saying that they run find when executed from the terminal? Have you managed to properly spoof a pre-receive command?
In the other issue, the ability of the server's user to execute the hooks was disabled, so I logged in as the gitea user and manually executed one of the hooks (successfully). I'd already checked that they were executable, but wanted to make sure I was reading the permissions properly.
kauron
on 16 Jan 2020
OK so those are all appropriately executable, next could you check the contents and mode of pre-receive.d/gitea
To update/correct these you need to use the administrator task "Update and resynchronize hooks"
zeripath
on 16 Jan 2020
Contents and mode:
[gitea@host pre-receive.d]$ ls -lh
total 4.0K
-rwxr-xr-x 1 gitea gitea 84 ene 15 22:44 gitea
[gitea@host pre-receive.d]$ cat gitea
#!/usr/bin/env bash
"/usr/bin/gitea" hook --config='/etc/gitea/app.ini' pre-receive
After running the corresponding administrator task, the only thing that changes is the time attribute of the file.
The pre-receive hook seems to be the default:
#!/usr/bin/env bash
data=$(cat)
exitcodes=""
hookname=$(basename $0)
GIT_DIR=${GIT_DIR:-$(dirname $0)}
for hook in ${GIT_DIR}/hooks/${hookname}.d/*; do
test -x "${hook}" || continue
echo "${data}" | "${hook}"
exitcodes="${exitcodes} $?"
done
for i in ${exitcodes}; do
[ ${i} -eq 0 ] || exit ${i}
done
This issue has been automatically marked as stale because it has not had recent activity. I am here to help clear issues left open even if solved or waiting for more insight. This issue will be closed if no further activity occurs during the next 2 weeks. If the issue is still valid just add a comment to keep it alive. Thank you for your contributions.
![stale[bot] picture](https://avatars3.githubusercontent.com/in/1724?v=4&s=40)
stale[bot]
on 16 Mar 2020
This issue is still valid.
kauron
on 17 Mar 2020
This issue has been automatically marked as stale because it has not had recent activity. I am here to help clear issues left open even if solved or waiting for more insight. This issue will be closed if no further activity occurs during the next 2 weeks. If the issue is still valid just add a comment to keep it alive. Thank you for your contributions.
stale[bot]
on 16 May 2020
This issue is still valid. @zeripath, what additional info can I provide to help debug this?
kauron
on 16 May 2020
Hmm... I'm a bit stumped but, one thing I've noticed elsewhere: make sure your LFS_CONTENT_PATH is an absolute path. (I wonder if you're affected by the thing that #11362 fixes)
zeripath
on 16 May 2020
It was a relative path. I've changed it to an absolute path, restarted gitea and tried again, but the problem's still there.
kauron
on 16 May 2020
What version of gitea are you using? - Would it be possible to update to master (preferably) or 1.11.5?
zeripath
on 16 May 2020
I'm using 1.11.5, but I'll try to get the master branch up and running.
kauron
on 20 May 2020
I have tried executing v1.11.5 in a x86_64 PC. Then I tried in the same ARM machine where it fails, running a separate instance under a different user with a blank db and the existing db. I could not reproduce the issue in either case.
While checking the logs, I've found the following:
may 26 16:31:08 HOSTNAME gitea[10611]: 2020/05/26 16:31:08 ...eue/queue_wrapped.go:68:setInternal() [W] [Attempt: 7] Failed to create queue: level for cfg: [123 34 65 100 100 114 101 115 115 101 115 34 58 34 49 50 55 46 48 46 48 46 49 58 54 51 55 57 34 44 34 66 97 116 99 104 76 101 110 103 116 104 34 58 50 48 44 34 66 108 111 99 107 84 105 109 101 111 117 116 34 58 49 48 48 48 48 48 48 48 48 48 44 34 66 111 111 115 116 84 105 109 101 111 117 116 34 58 51 48 48 48 48 48 48 48 48 48 48 48 44 34 66 111 111 115 116 87 111 114 107 101 114 115 34 58 53 44 34 68 66 73 110 100 101 120 34 58 48 44 34 68 97 116 97 68 105 114 34 58 34 100 97 116 97 47 100 97 116 97 47 113 117 101 117 101 115 47 116 97 115 107 34 44 34 77 97 120 87 111 114 107 101 114 115 34 58 49 48 44 34 78 97 109 101 34 58 34 116 97 115 107 34 44 34 78 101 116 119 111 114 107 34 58 34 34 44 34 80 97 115 115 119 111 114 100 34 58 34 34 44 34 81 117 101 117 101 76 101 110 103 116 104 34 58 50 48 44 34 81 117 101 117 101 78 97 109 101 34 58 34 116 97 115 107 95 113 117 101 117 101 34 44 34 87 111 114 107 101 114 115 34 58 49 125] error: leveldb: manifest corrupted (field 'comparer'): missing [file=MANIFEST-708535]
may 26 16:31:08 HOSTNAME gitea[10611]: 2020/05/26 16:31:08 ...eue/queue_wrapped.go:68:setInternal() [W] [Attempt: 7] Failed to create queue: level for cfg: [123 34 65 100 100 114 101 115 115 101 115 34 58 34 49 50 55 46 48 46 48 46 49 58 54 51 55 57 34 44 34 66 97 116 99 104 76 101 110 103 116 104 34 58 50 48 44 34 66 108 111 99 107 84 105 109 101 111 117 116 34 58 49 48 48 48 48 48 48 48 48 48 44 34 66 111 111 115 116 84 105 109 101 111 117 116 34 58 51 48 48 48 48 48 48 48 48 48 48 48 44 34 66 111 111 115 116 87 111 114 107 101 114 115 34 58 53 44 34 68 66 73 110 100 101 120 34 58 48 44 34 68 97 116 97 68 105 114 34 58 34 100 97 116 97 47 100 97 116 97 47 113 117 101 117 101 115 47 105 115 115 117 101 95 105 110 100 101 120 101 114 34 44 34 77 97 120 87 111 114 107 101 114 115 34 58 49 48 44 34 78 97 109 101 34 58 34 105 115 115 117 101 95 105 110 100 101 120 101 114 34 44 34 78 101 116 119 111 114 107 34 58 34 34 44 34 80 97 115 115 119 111 114 100 34 58 34 34 44 34 81 117 101 117 101 76 101 110 103 116 104 34 58 50 48 44 34 81 117 101 117 101 78 97 109 101 34 58 34 105 115 115 117 101 95 105 110 100 101 120 101 114 95 113 117 101 117 101 34 44 34 87 111 114 107 101 114 115 34 58 49 125] error: leveldb: manifest corrupted (field 'comparer'): missing [file=MANIFEST-708536]
This message is repeated periodically, every 90 seconds approx.
Another message that I'm not sure whether or not is related is the following line when pushing the repo:
remote: Processed 0 references in total
a similar line appeard in the cases when it worked properly, but instead of 0 references, it was 1 reference, or it didn't appear at all.
kauron
on 26 May 2020
OK so that's a problem with the queues - they have been corrupted due to a crash or a kill -9 at some point. You will need to remove them.
zeripath
on 26 May 2020
So do I need to delete the two folders called task and issue_indexer in $GITEA_HOME/data/data/queues/?
I did so, and the log message stopped appearing, but the issue still occurs.
kauron
on 26 May 2020
I have walked through this issue and I'm in the exact same boat (and have been for months), except for the queue issue of https://github.com/go-gitea/gitea/issues/9792#issuecomment-634064659.
I'm giving some extra info, may not be relevant, but perhaps something pops up.
- Raspberry Pi 4
- Arch Linux for ARM
- Gitea 1.11.5
- Only using this in my local network, so I haven't (yet) set up an SSL certificate, i.e. communication is unencrypted and directly to the Pi's IP.
- System root on SSD, repos are on separate mount:
[repository]
ROOT = /srv/git
(where /srv is not a directory but a mounted partition)
/etc/systemd/system/gitea.service.d/override.conf:
[Service]
ReadWritePaths=/etc/gitea/app.ini /srv/git
ltGuillaume
on 29 May 2020
This issue has been automatically marked as stale because it has not had recent activity. I am here to help clear issues left open even if solved or waiting for more insight. This issue will be closed if no further activity occurs during the next 2 weeks. If the issue is still valid just add a comment to keep it alive. Thank you for your contributions.
stale[bot]
on 29 Jul 2020
This issue still hasn't been addressed and basically means I have to do a manual SQL queryy every time I create a new repo.
ltGuillaume
on 29 Jul 2020
This problem is always due to users not pushing with a key managed by Gitea.
The only way Gitea knows about commits is if it is told about them during post-receive - to do that there is a post receive hook in the repository and that requires an environment which is set up by Gitea serv or the web server.
I will not go in to how to set the environment correctly because that is unsupported - but it is not difficult to find what the environment should be. Only push to Gitea's repositories using a Gitea managed key or over http.
zeripath
on 1 Aug 2020
@ltGuillaume Ok, let's look at your case. You're definitely pushing to gitea's http url? Are you post-receive hooks present and executable? Are they up-to-date?
zeripath
on 2 Aug 2020
I'm always using http://myserver:3000/ltGuillaume/Repo-Name.git, yes, but I probably should be doing something more. I'm not sure what post-receive hooks I should create. I usually do the following, after creating an empty repo from the Gitea web interface:
git init
git add .
git commit
git remote add origin http://myserver:3000/ltGuillaume/Repo-Name.git
git push -u origin master
In your app.ini there is a [repository] section. That has a setting ROOT which says where the raw git repositories that Gitea uses are kept.
Your repositories are stored in directories under that ROOT called owner/reponame.git
In each, there is a hooks/ directory. In particular there are 4 special scripts in these pre-receive,pre-receive.d/gitea, post-receive, post-receive.d/gitea.
These need to be executable and run each time a push is made.
Gitea will create these "delegate" hooks for you. There is a function on the http://localhost:3000/admin page: Resynchronize pre-receive, update and post-receive hooks of all repositories - that will update the content for these. However, if your repositories are on a non-executable partition that would cause problems.
It is worth checking the values of these files.
zeripath
on 2 Aug 2020
The partition containing my repos is mounted using defaults,noatime,nodiratime,discard, so there's no noexec there.
The pre- and post-receive.d hook folders and the gitea files within them are present and have the permissions (d)rwxr-xr-x.
I just created a new repo via the web interface without initializing it and went into its post-receive.d and and pre-receive.d folders: both had a gitea file with:
#!/usr/bin/env bash
"/usr/bin/gitea" hook --config='/etc/gitea/app.ini' [pre/post]-receive
The files pre-receive and post-receive were also present, here the contents of one:
#!/usr/bin/env bash
data=$(cat)
exitcodes=""
hookname=$(basename $0)
GIT_DIR=${GIT_DIR:-$(dirname $0)}
for hook in ${GIT_DIR}/hooks/${hookname}.d/*; do
test -x "${hook}" && test -f "${hook}" || continue
echo "${data}" | "${hook}"
exitcodes="${exitcodes} $?"
done
for i in ${exitcodes}; do
[ ${i} -eq 0 ] || exit ${i}
done
Ok those files look right to me on a quick glance.
What version are you using?
zeripath
on 2 Aug 2020
1.11.6 on Arch ARM (1.12.3 is not yet available).
ltGuillaume
on 2 Aug 2020
Hmm... I'm not sure how Arch ARM packages their builds as it seems they have 1.12.3 https://github.com/archlinuxarm/PKGBUILDs/tree/master/community/gitea
techknowlogick
on 3 Aug 2020
So I think in 1.11 there was an issue for at least one user that required the --work-path command line option to be set out may be worth manually adjusting the hooks to see if that is the problem?
I fixed it for 1.12
zeripath
on 3 Aug 2020
I'll try adding --work-path='/var/lib/gitea/' to the hooks.
__Edit:__ After pushing a new commit with the changed pre-/post-receive.d/gitea's, the repo is still seen as empty by Gitea's web interface.
It's a bit strange: https://archlinuxarm.org/packages shows that v1.11.6 is currently the latest for the Raspberry Pi 4, as aarch64 is not (officially) supported yet, but I can't find anywhere if there's a reason why it's not updated yet.
ltGuillaume
on 7 Aug 2020
it would be helpful to add an echo to those hooks to check that they're actually being run.
zeripath
on 7 Aug 2020
I'd have to understand first where the echo'ed text would turn up.
I can stop gitea as a service and run it from bash under user gitea again if that helps, but it looks like these hooks run another process anyway, so I'm not sure if their output will turn up in the console then. Will they be in /var/log/gitea/gitea.log without any changes (or do I need to change to debug logging for example)?
ltGuillaume
on 7 Aug 2020
No an echo in pre-receive or post-receive hook would appear in your git stream.
I wouldn't be surprised if when you run Gitea under user if the hooks worked.
zeripath
on 8 Aug 2020
I updated to Gitea 1.12.3 and created a new empty repo today. After several commits to it, Gitea still hasn't picked up on its not being empty anymore.
No an echo in pre-receive or post-receive hook would appear in your git stream.
I wouldn't be surprised if when you run Gitea under user if the hooks worked.
Could you perhaps rephrase?
Am I to add echo The pre-receive hook was run into pre-receive.d/gitea script (and in post), then push a commit and then should be able to find it where?
ltGuillaume
on 13 Aug 2020
I also see that the sizes of all my local repositories are showing 0B (the others are mirrors) and that none of the activity in those local repositories show up on my dashboard.
ltGuillaume
on 6 Sep 2020
Yes this will keep happening until you fix whatever is happening with the hooks.
Gitea will not be informed of changes happening to its repositories unless the hooks are running correctly.
Have you placed an echo in the pre- or post-receive hooks yet?
zeripath
on 6 Sep 2020
Hi, I've finally had time to test this issue more in depth, by adding echo statements to the hooks (update, pre-receive and post-receive). It appears that the post-receive does not receive the data, as it indicates 0 references processed.
I'm using version: 1.12.4, installed from this [package][1].
The complete output of a git push is the following:
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 870 bytes | 870.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Start pre-receive
remote: End pre-receive
remote: Start update
remote: End update
remote: Start post-receive
remote: Processed 0 references in total
remote: End post-receive
To ssh://****:22/user/test.git
* [new branch] master -> master
Branch 'master' set up to track remote branch 'master' from 'origin'.
This is my output. I duplicated the command from /*.d/gitea and put echo in front in order to see the actual commands that are or should be run:
E:\Projects\Websites\MyProject>git push
Enumerating objects: 11, done.
Counting objects: 100% (11/11), done.
Delta compression using up to 8 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 669 bytes | 669.00 KiB/s, done.
Total 6 (delta 4), reused 0 (delta 0), pack-reused 0
remote: /usr/bin/gitea hook --config=/etc/gitea/app.ini pre-receive
remote: /usr/bin/gitea hook --config=/etc/gitea/app.ini update refs/heads/master bfd503996671dd9b4546fcb1e4ee6ee2409dfeed 0eca4bdb74e4df71788292effba14aa58c47813f
remote: /usr/bin/gitea hook --config=/etc/gitea/app.ini post-receive
remote: Processed 0 references in total
To http://address:3000/ltGuillaume/MyProject.git
bfd5039..0eca4bd master -> master
Processed 0 references in total...
That is interesting
zeripath
on 7 Sep 2020
The only ways that a hook can declare no references are:
https://github.com/go-gitea/gitea/blob/d257485bc0026c9717fe7bf4c9953ad1b7a1a9ae/cmd/hook.go#L344-L346
I.e. it's a wiki in the db or GITEA_REPO_IS_WIKI has been set in the environment - please check!
Or:
https://github.com/go-gitea/gitea/blob/d257485bc0026c9717fe7bf4c9953ad1b7a1a9ae/cmd/hook.go#L348-L352
It's being passed a spurious extra space. In which case it would be helpful to see what is being received by the hook process but I'm too tired to write the bash right now. The basics would be to prefix the call to Gitea serv hook with cat - | tee but it would be good to pass it through xxd to prove if there are sneaky carriage returns in there.
The git post receive docs are very clear that there cannot be more than 2 spaces on a line on stdin for its receive scripts so if something is adding that then that's the problem.
So two options:
A) somehow the environment flag is being set as a wiki. Could add an echo for the environment variable to see if it has been set.
B) something is screwing with the stdin for the receive hooks and adding a space where there should not be. That could be that your Windows git client is sending a \r at the end of the each line in contravention with its own docs.
If you can show that it's not the environment variable - then it's very likely that and it should be very easy to adjust the code to cope with that.
zeripath
on 7 Sep 2020
Thanks for looking into the code.
A) I have no idea why and how this could happen (automatically) with _every_ new repo I create via Gitea's web. I added an echo to pre-receive.d/gitea, result was: remote: GITEA_REPO_IS_WIKI = false (see below for full output).
B) I'm not sure if this is what you mean, but I do have core.autocrlf set to false in .gitconfig on my Windows git client, but changing this explicitly to true and pushing another commit did not change anything. Output:
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 299 bytes | 299.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
remote: GITEA_REPO_IS_WIKI = false
remote: /usr/bin/gitea hook --config=/etc/gitea/app.ini pre-receive
remote: /usr/bin/gitea hook --config=/etc/gitea/app.ini update refs/heads/master c300b3e004ab5f9edcec153eb13bbcb5c5f5bbe4 41cf8dc7ed0e8198be29c34e623384861d84386f
remote: /usr/bin/gitea hook --config=/etc/gitea/app.ini post-receive
remote: Processed 0 references in total
To http://address:3000/ltGuillaume/MyProject.git
c300b3e..41cf8dc master -> master
I also created a new repo with this option set to true, but I got the same result as with the option set to false.
ltGuillaume
on 8 Sep 2020
OK are you able to compile gitea?
zeripath
on 8 Sep 2020
One other thing - are you always pushing from Windows?
zeripath
on 8 Sep 2020
If you could apply the patch in #12772 to gitea and build it I wonder if this would finally solve yours and a few others problems.
zeripath
on 8 Sep 2020
OK are you able to compile gitea?
That's a bit beyond me, I'm just running packages from the Arch default repo on a Pi 4.
One other thing - are you always pushing from Windows?
Currently, yes.
ltGuillaume
on 9 Sep 2020
I just opened up a VirtualBox machine with Manjaro and created a new repo on my Gitea instance. I then followed the exact instructions again as shown by Gitea, but this time using git on Manjaro Linux. The result is the same:
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 223 bytes | 223.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Processed 0 references in total
To http://address:3000/ltGuillaume/MyProject.git
* [new branch] master -> master
Branch 'master' set up to track remote branch 'master' from 'origin'.
It's not particularly difficult to compile gitea - but fine.
I've taken the latest origin/release/v1.12 and compiled it as per Arch's pkgbuild (https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=gitea-git) with the below patches.
The file will be present until 19:00 BST today at which point I will delete.
PATCHES
From b26a160de7cfef480d7edea697e39c016f84747d Mon Sep 17 00:00:00 2001
From: Andrew Thornton <[email protected]>
Date: Tue, 8 Sep 2020 22:11:40 +0100
Subject: [PATCH 1/2] Tolerate a terminal CR on a pre/post-receive line
My suspicion is that occasionally windows users will send a terminal
CRLF on the post-receive lines in violation of the git spec. Currently
gitea enforces that there have to be only 3 fields - but the terminal CR
will form an additional field.
This PR will allow a 4th field iff it is simply a carriage return.
It will also log when it is not allowing lines.
Fix #9792
Signed-off-by: Andrew Thornton <[email protected]>
---
cmd/hook.go | 19 +++++++++++++++++--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/cmd/hook.go b/cmd/hook.go
index fa932087f..b345800bf 100644
--- a/cmd/hook.go
+++ b/cmd/hook.go
@@ -17,6 +17,7 @@ import (
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/git"
+ "code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/private"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/util"
@@ -195,8 +196,15 @@ Gitea or set your environment appropriately.`, "")
}
fields := bytes.Fields(scanner.Bytes())
- if len(fields) != 3 {
+ if len(fields) < 3 {
+ fmt.Fprintf(out, "Ignoring bad pre-receive line with less than 3 fields: %s", scanner.Text())
continue
+ } else if len(fields) > 3 {
+ if string(fields[4]) != "\r" && len(fields) == 4 {
+ fmt.Fprintf(out, "Ignoring bad post-receive line with greater than 4 fields: %s", scanner.Text())
+ continue
+ }
+ log.Debug("Tolerating terminal CR in violation of specification")
}
oldCommitID := string(fields[0])
@@ -329,8 +337,15 @@ Gitea or set your environment appropriately.`, "")
}
fields := bytes.Fields(scanner.Bytes())
- if len(fields) != 3 {
+ if len(fields) < 3 {
+ fmt.Fprintf(out, "Ignoring bad post-receive line with less than 3 fields: %s", scanner.Text())
continue
+ } else if len(fields) > 3 {
+ if string(fields[4]) != "\r" && len(fields) == 4 {
+ fmt.Fprintf(out, "Ignoring bad post-receive line with greater than 4 fields: %s", scanner.Text())
+ continue
+ }
+ log.Debug("Tolerating a final CR on pre-receive lines in violation of spec")
}
fmt.Fprintf(out, ".")
--
2.25.1
From fc277c73033cdf7f27005064a2da4da75505bbfa Mon Sep 17 00:00:00 2001
From: Andrew Thornton <[email protected]>
Date: Wed, 9 Sep 2020 14:54:07 +0100
Subject: [PATCH 2/2] forcibly debug
Signed-off-by: Andrew Thornton <[email protected]>
---
cmd/hook.go | 4 ++--
cmd/serv.go | 5 +++--
2 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/cmd/hook.go b/cmd/hook.go
index b345800bf..781e15199 100644
--- a/cmd/hook.go
+++ b/cmd/hook.go
@@ -139,7 +139,7 @@ func runHookPreReceive(c *cli.Context) error {
return nil
}
- setup("hooks/pre-receive.log", false)
+ setup("hooks/pre-receive.log", true)
if len(os.Getenv("SSH_ORIGINAL_COMMAND")) == 0 {
if setting.OnlyAllowPushIfGiteaEnvironmentSet {
@@ -281,7 +281,7 @@ func runHookPostReceive(c *cli.Context) error {
return nil
}
- setup("hooks/post-receive.log", false)
+ setup("hooks/post-receive.log", true)
if len(os.Getenv("SSH_ORIGINAL_COMMAND")) == 0 {
if setting.OnlyAllowPushIfGiteaEnvironmentSet {
diff --git a/cmd/serv.go b/cmd/serv.go
index 7c2be5157..5ba257d07 100644
--- a/cmd/serv.go
+++ b/cmd/serv.go
@@ -50,8 +50,9 @@ var CmdServ = cli.Command{
}
func setup(logPath string, debug bool) {
- if !debug {
- _ = log.DelLogger("console")
+ _ = log.DelLogger("console")
+ if debug {
+ _ = log.NewLogger(1000, "console", "console", `{"level":"trace","stacktracelevel":"NONE","stderr":true}`)
}
setting.NewContext()
if debug {
--
2.25.1
Thanks for going through all this. Unfortunately this seems to be an x64 build, whereas ArchLinux for ARM (on the Raspberry Pi) is still x86.
Edit: This has been changed very recently, but for stability's sake, I'm still on the ARMv7 x86 release.
Considering I get the exact same results when using git exclusively from a Linux distro (and without any files in the repo which were created within Windows, either), I doubt this patch would solve my issue, don't you think? See https://github.com/go-gitea/gitea/issues/9792#issuecomment-689196515.
ltGuillaume
on 9 Sep 2020
there's more to the patch than just the suggested PR.
zeripath
on 9 Sep 2020
It's going to be very difficult to replicate the exact version that Arch is using to build gitea on a non-x86 architecture.
For example I can't build with "pam".
The patch does more than just the proposed change. Extra logging information should be provided.
zeripath
on 9 Sep 2020
Edit: I'm guessing the missing file(s) are bundled in the gitea build here? https://archlinuxarm.org/packages/armv7h/gitea/files/PKGBUILD
locale_en-US.ini is not on my system.
ARM-6:
Sep 09 18:24:10 pi audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:24:10 pi kernel: audit: type=1130 audit(1599668650.876:477): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:24:10 pi kernel: audit: type=1131 audit(1599668650.876:478): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:24:10 pi kernel: audit: type=1130 audit(1599668650.886:479): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:24:10 pi audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:24:11 pi gitea[24634]: 2020/09/09 18:24:11 cmd/web.go:107:runWeb() [I] Starting Gitea on PID: 24634
Sep 09 18:24:11 pi gitea[24634]: 2020/09/09 18:24:11 ...dules/setting/git.go:93:newGit() [I] Git Version: 2.28.0, Wire Protocol Version 2 Enabled
Sep 09 18:24:11 pi gitea[24634]: 2020/09/09 18:24:11 ...es/setting/markup.go:61:newMarkupSanitizer() [W] Skipping empty section: markup.sanitizer.1.
Sep 09 18:24:11 pi gitea[24634]: 2020/09/09 18:24:11 routers/init.go:118:GlobalInit() [T] AppPath: /usr/bin/gitea
Sep 09 18:24:11 pi gitea[24634]: 2020/09/09 18:24:11 routers/init.go:119:GlobalInit() [T] AppWorkPath: /var/lib/gitea
Sep 09 18:24:11 pi gitea[24634]: 2020/09/09 18:24:11 routers/init.go:120:GlobalInit() [T] Custom path: /var/lib/gitea/custom
Sep 09 18:24:11 pi gitea[24634]: 2020/09/09 18:24:11 routers/init.go:121:GlobalInit() [T] Log path: /var/log/gitea/
Sep 09 18:24:11 pi gitea[24634]: panic: fail to set message file(en-US): open conf/locale/locale_en-US.ini: no such file or directory
Sep 09 18:24:11 pi gitea[24634]: goroutine 1 [running]:
Sep 09 18:24:11 pi gitea[24634]: gitea.com/macaron/i18n.initLocales(0x5bafbee, 0x0, 0x192368b, 0xb, 0x5219560, 0x199231a, 0x12, 0x563e000, 0x17, 0x20, ...)
Sep 09 18:24:11 pi gitea[24634]: /source/vendor/gitea.com/macaron/i18n/i18n.go:57 +0x4e8
Sep 09 18:24:11 pi gitea[24634]: gitea.com/macaron/i18n.I18n(0x5f1fa48, 0x1, 0x1, 0x0, 0x0)
Sep 09 18:24:11 pi gitea[24634]: /source/vendor/gitea.com/macaron/i18n/i18n.go:160 +0x4c
Sep 09 18:24:11 pi gitea[24634]: code.gitea.io/gitea/routers.InitLocales()
Sep 09 18:24:11 pi gitea[24634]: /source/routers/init.go:100 +0x314
Sep 09 18:24:11 pi gitea[24634]: code.gitea.io/gitea/routers.GlobalInit(0x1e377e8, 0x56135d0)
Sep 09 18:24:11 pi gitea[24634]: /source/routers/init.go:124 +0x2b4
Sep 09 18:24:11 pi gitea[24634]: code.gitea.io/gitea/cmd.runWeb(0x5f7c580, 0x0, 0x0)
Sep 09 18:24:11 pi gitea[24634]: /source/cmd/web.go:116 +0x25c
Sep 09 18:24:11 pi gitea[24634]: github.com/urfave/cli.HandleAction(0x1622290, 0x1a7467c, 0x5f7c580, 0x560d800, 0x0)
Sep 09 18:24:11 pi gitea[24634]: /source/vendor/github.com/urfave/cli/app.go:490 +0xac
Sep 09 18:24:11 pi gitea[24634]: github.com/urfave/cli.Command.Run(0x18a9367, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x19bc8d0, 0x16, 0x0, ...)
Sep 09 18:24:11 pi gitea[24634]: /source/vendor/github.com/urfave/cli/command.go:210 +0x7a4
Sep 09 18:24:11 pi gitea[24634]: github.com/urfave/cli.(*App).Run(0x5082460, 0x500c140, 0x4, 0x4, 0x0, 0x0)
Sep 09 18:24:11 pi gitea[24634]: /source/vendor/github.com/urfave/cli/app.go:255 +0x514
Sep 09 18:24:11 pi gitea[24634]: main.main()
Sep 09 18:24:11 pi gitea[24634]: /source/main.go:109 +0x59c
Sep 09 18:24:11 pi systemd[1]: gitea.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
ARM-5:
Sep 09 18:27:24 pi audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:27:24 pi kernel: audit: type=1130 audit(1599668844.373:789): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:27:24 pi kernel: audit: type=1131 audit(1599668844.373:790): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:27:24 pi kernel: audit: type=1130 audit(1599668844.383:791): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:27:24 pi audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=gitea comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 09 18:27:24 pi gitea[25952]: 2020/09/09 18:27:24 cmd/web.go:107:runWeb() [I] Starting Gitea on PID: 25952
Sep 09 18:27:24 pi gitea[25952]: 2020/09/09 18:27:24 ...dules/setting/git.go:93:newGit() [I] Git Version: 2.28.0, Wire Protocol Version 2 Enabled
Sep 09 18:27:24 pi gitea[25952]: 2020/09/09 18:27:24 ...es/setting/markup.go:61:newMarkupSanitizer() [W] Skipping empty section: markup.sanitizer.1.
Sep 09 18:27:24 pi gitea[25952]: 2020/09/09 18:27:24 routers/init.go:118:GlobalInit() [T] AppPath: /usr/bin/gitea
Sep 09 18:27:24 pi gitea[25952]: 2020/09/09 18:27:24 routers/init.go:119:GlobalInit() [T] AppWorkPath: /var/lib/gitea
Sep 09 18:27:24 pi gitea[25952]: 2020/09/09 18:27:24 routers/init.go:120:GlobalInit() [T] Custom path: /var/lib/gitea/custom
Sep 09 18:27:24 pi gitea[25952]: 2020/09/09 18:27:24 routers/init.go:121:GlobalInit() [T] Log path: /var/log/gitea/
Sep 09 18:27:24 pi gitea[25952]: panic: fail to set message file(en-US): open conf/locale/locale_en-US.ini: no such file or directory
Sep 09 18:27:24 pi gitea[25952]: goroutine 1 [running]:
Sep 09 18:27:24 pi gitea[25952]: gitea.com/macaron/i18n.initLocales(0x47f786e, 0x0, 0x194657f, 0xb, 0x47000e0, 0x19b522f, 0x12, 0x458ee00, 0x17, 0x20, ...)
Sep 09 18:27:24 pi gitea[25952]: /source/vendor/gitea.com/macaron/i18n/i18n.go:57 +0x4e8
Sep 09 18:27:24 pi gitea[25952]: gitea.com/macaron/i18n.I18n(0x4a1fa48, 0x1, 0x1, 0x0, 0x0)
Sep 09 18:27:24 pi gitea[25952]: /source/vendor/gitea.com/macaron/i18n/i18n.go:160 +0x4c
Sep 09 18:27:24 pi gitea[25952]: code.gitea.io/gitea/routers.InitLocales()
Sep 09 18:27:24 pi gitea[25952]: /source/routers/init.go:100 +0x314
Sep 09 18:27:24 pi gitea[25952]: code.gitea.io/gitea/routers.GlobalInit(0x1e58a90, 0x47f2190)
Sep 09 18:27:24 pi gitea[25952]: /source/routers/init.go:124 +0x2b4
Sep 09 18:27:24 pi gitea[25952]: code.gitea.io/gitea/cmd.runWeb(0x3d30d10, 0x0, 0x0)
Sep 09 18:27:24 pi gitea[25952]: /source/cmd/web.go:116 +0x25c
Sep 09 18:27:24 pi gitea[25952]: github.com/urfave/cli.HandleAction(0x1645170, 0x1a97590, 0x3d30d10, 0x3e6e800, 0x0)
Sep 09 18:27:24 pi gitea[25952]: /source/vendor/github.com/urfave/cli/app.go:490 +0xac
Sep 09 18:27:24 pi gitea[25952]: github.com/urfave/cli.Command.Run(0x18cc247, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x19df7e5, 0x16, 0x0, ...)
Sep 09 18:27:24 pi gitea[25952]: /source/vendor/github.com/urfave/cli/command.go:210 +0x7a4
Sep 09 18:27:24 pi gitea[25952]: github.com/urfave/cli.(*App).Run(0x3edd420, 0x3c0c140, 0x4, 0x4, 0x0, 0x0)
Sep 09 18:27:24 pi gitea[25952]: /source/vendor/github.com/urfave/cli/app.go:255 +0x514
Sep 09 18:27:24 pi gitea[25952]: main.main()
Sep 09 18:27:24 pi gitea[25952]: /source/main.go:109 +0x59c
Sep 09 18:27:24 pi systemd[1]: gitea.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
So you need bindata builds?
zeripath
on 9 Sep 2020
I'm aware I'm asking a lot, truly sorry about that 馃槦 The default ARMv7 releases from the Arch repo are indeed bindata builds then, yeah.
ltGuillaume
on 9 Sep 2020
re-uploaded
zeripath
on 9 Sep 2020
Another lang file now fails: Sep 09 20:35:54 pi gitea[10272]: panic: fail to set message file(pt-PT): open conf/locale/locale_pt-PT.ini: no such file or directory
ltGuillaume
on 9 Sep 2020
Just edit your app.ini and stick
[i18n]
LANGS = en-US
NAMES = English
.gitconfig: autocrlf = false
touch README.md
git init
git add README.md
git commit -m "First commit"
git remote add origin http://address:3000/ltGuillaume/MyProject.git
git push -u origin master
Output:
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 8 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 302 bytes | 302.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Pre-Receive hook echo...
remote: Update hook echo...
remote: Post-Receive hook echo...
remote: Ignoring bad post-receive line with less than 3 fields: Processed 0 references in total
To http://address:3000/ltGuillaume/MyProject.git
* [new branch] master -> master
Branch 'master' set up to track remote branch 'master' from 'origin'.
That's very interesting the post receive script is not receiving any refs to update or even anything over stdin...
This is very weird. Something must be eating stdin on the way in to post-receive
zeripath
on 10 Sep 2020
The exact same thing happens when I clone this repo in Manjaro Linux, then push a new commit:
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 287 bytes | 287.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Pre-Receive hook...
remote: Update hook...
remote: Post-Receive hook...
remote: Ignoring bad post-receive line with less than 3 fields: Processed 0 references in total
To http://address:3000/ltGuillaume/MyProject.git
3c0f4dc..ce74771 master -> master
Could you show me your post-receive.d/Gitea script and post-receive script again?
In the post-receive one it could be useful to add:
echo "Received"
echo "${data}"
echo "End"
remote: Pre-Receive hook...
remote: Update hook...
remote: Post-Receive hook...
remote: Received
remote:
remote: End
remote: Ignoring bad post-receive line with less than 3 fields: Processed 0 references in total
Yup that proves it something is eating or preventing your post-receive messages from reaching the hook. Try adding the same echos to the pre-receive variant too to see if they're suffering the same problem.
I'm a bit stuck on how to proceed - it doesn't look like it's really a Gitea issue anymore. All hooks would fail in this situation.
What I don't completely understand is how git's objects are being passed in - those go over stdin too.
It could be bash, cat, git, or SSHD that are doing this but it's not Gitea (except maybe one of the options it puts in the authorized_keys file).
Weird as hell.
Maybe the post-receive script needs to read:
#!/usr/bin/env bash
data=$(</dev/stdin)
exitcodes=""
hookname=$(basename $0)
GIT_DIR=${GIT_DIR:-$(dirname $0)}
echo "RECEIVED"
echo "${data}"
echo "END"
for hook in ${GIT_DIR}/hooks/${hookname}.d/*; do
test -x "${hook}" && test -f "${hook}" || continue
echo "${data}" | "${hook}"
exitcodes="${exitcodes} $?"
done
for i in ${exitcodes}; do
[ ${i} -eq 0 ] || exit ${i}
done
Actually make that:
data=$(
zeripath
on 10 Sep 2020
All three now have echo "${data}".
Script above (with data=$(</dev/stdin)) in post-receive.d/gitea.
remote: Pre-Receive hook...
remote:
remote: End
remote: Update hook...
remote:
remote: End
remote: Post-Receive hook...
remote: RECEIVED
remote:
remote: END
remote: Processed 0 references in total
To http://address:3000/ltGuillaume/MyProject.git
b8e3f18..76a202c master -> master
Going to bed now, hope we'll figure this out somehow...
ltGuillaume
on 10 Sep 2020
Sorry I haven't replied today. Your findings above definitely indicate that the messages git should be sending are not getting through to the bash hooks.
Now that is weird. So the question is what is to blame.
Well one thing you could do - just to rule out one last way Gitea could be to blame - is to simplify and remove most of the restrictions on the authorized key line in .authorized_keys but I don't think that will fix things.
My suspicion is that it is bash on the server or it is a the version of git that is on the server.
But how do you figure out?
One option is to try to upgrade these to see if that fixes it. Another is to seek help on arch. Explain that hooks are receiving no content on stdin on pushing. They might know what the problem is.
zeripath
on 11 Sep 2020
Well one thing you could do - just to rule out one last way Gitea could be to blame - is to simplify and remove most of the restrictions on the authorized key line in .authorized_keys but I don't think that will fix things.
I'm using HTTP to push my commits, so will authorized_keys even have any influence?
My suspicion is that it is bash on the server or it is a the version of git that is on the server.
But how do you figure out?
One option is to try to upgrade these to see if that fixes it. Another is to seek help on arch. Explain that hooks are receiving no content on stdin on pushing. They might know what the problem is.
Everything is up-to-date, and I'm using a rolling release distro (Arch), so it's more up-to-date than most other distros already.
git: 2.28.0
bash: 5.0.018 (also had these isssues with bash 4.x)
I just tried cloning, then committing some changes to an affected repo by using Git on the server itself. Result's the same:
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 299 bytes | 299.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Pre-Receive hook...
remote:
remote: End
remote: Update hook...
remote:
remote: End
remote: Post-Receive hook...
remote: RECEIVED
remote:
remote: END
remote: End
remote: Processed 0 references in total
To http://localhost:3000/ltGuillaume/MyProject.git
76a202c..e8533d5 master -> master
I'll create a topic on https://archlinuxarm.org/forum/ and I'll try Gitea on my Manjaro virtual machine, just to be sure it's not a cross-platform Arch issue (I highly doubt it).
Edit: Ok, in Manjaro, the correct behavior with the script from https://github.com/go-gitea/gitea/issues/9792#issuecomment-689856986 resulted in
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Writing objects: 100% (3/3), 259 bytes | 259.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote: RECEIVED
remote: d55828af5c0b4f64966368def180496530ac6603 c1ca110e76cc2a4f27b9909e926802e60a66cc0b refs/heads/master
remote: END
remote: Processed 0 references in total
To http://localhost:3000/ltGuillaume/MyProject.git
d55828a..c1ca110 master -> master
I wonder what's happening on Arch then. How weird.
zeripath
on 11 Sep 2020
FWIW I'm using manjaro on a pinebook pro (aarch64) and I'm unable to replicate.
techknowlogick
on 11 Sep 2020
I guess I got 1 step closer with the following (this must be a recent addition, I've checked the Wiki multiple times, obviously).
Edit: user Simplefied has added this workaround to the Arch Wiki on June 6th (https://wiki.archlinux.org/index.php?title=Gitea&type=revision&diff=618524&oldid=615095).
Process dumped core on ArchLinux ARM
A problem can appear after pushing new repo. In web UI repo stay uninitialized, and the following error is present in the system journal:
Jun 06 14:34:20 rpi-srv systemd-coredump[5724]: Process 5722 (cat) of user 986 dumped core. Stack trace of thread 5722: #0 0x0000000076f06e58 posix_fadvise64 (libc.so.6 + 0xd2e58)To fix this problem, override gitea.service:
# systemctl edit gitea[Service] SystemCallArchitectures= SystemCallFilter=
This has changed the push output from:
remote: Pre-Receive hook...
remote:
remote: End
remote: Update hook...
remote:
remote: End
remote: Post-Receive hook...
remote: RECEIVED
remote:
remote: END
remote: End
remote: Processed 0 references in total
to
remote: Pre-Receive hook...
remote:
remote: End
remote: Update hook...
remote:
remote: End
remote: Post-Receive hook...
remote: RECEIVED
remote: e8533d5f096957a7d6298510aec9af8f89fb4e33 28f42c6f8275198abc5856307bd3d024e3759436 refs/heads/master
remote: END
remote: End
remote: Processed 0 references in total
It looks like this mean there are no issues with hook not getting the stdin contents anymore, but there are still no references processed.
ltGuillaume
on 11 Sep 2020
After recreating the repo (and thus removing the scripts), I finally got
remote: . Processing 1 references
remote: Processed 1 references in total
Every test since has consistently produced a working, fully initialized repo.
Thanks @zeripath for all your help. Sorry I didn't find the core dump myself. As you can see the dump is linked to cat, I didn't think to look for anything like that, I guess I should have after your script.
ltGuillaume
on 11 Sep 2020
Sorry it took so long. Maybe we need to drop the bash scripts and write some go to do it.
zeripath
on 11 Sep 2020
I don't know. Now I found the solution, I'm seeing:
https://github.com/go-gitea/gitea/issues/6620#issuecomment-519912187 Definitely related, not officially resolved (closed by stalebot).
https://github.com/go-gitea/gitea/issues/11231 Seems like it's not yet resolved (closed by stalebot) and may have the same solution.
ltGuillaume
on 11 Sep 2020
OK so if you can write a clear solution in a comment here we can close this issue and add links to it on those issues.
zeripath
on 11 Sep 2020
The following addition to the Arch Wiki has solved the following issues for me:
- Newly created repos remain as if empty in web UI after pushing commits
- Repo sizes remain 0B after manually setting repos those as non-empty
- No repo updates on the Dashboard (https://github.com/go-gitea/gitea/issues/6620)
User Simplefied has added this workaround on June 6th (https://wiki.archlinux.org/index.php?title=Gitea&type=revision&diff=618524&oldid=615095).
Process dumped core on ArchLinux ARM
A problem can appear after pushing new repo. In web UI repo stay uninitialized, and the following error is present in the system journal:
Jun 06 14:34:20 rpi-srv systemd-coredump[5724]: Process 5722 (cat) of user 986 dumped core. Stack trace of thread 5722: #0 0x0000000076f06e58 posix_fadvise64 (libc.so.6 + 0xd2e58)To fix this problem, override gitea.service:
# systemctl edit gitea[Service] SystemCallArchitectures= SystemCallFilter=
ltGuillaume
on 11 Sep 2020
Thanks @ItGuillaume really sorry this took so long
zeripath
on 11 Sep 2020
@ArchangeGabriel would you be able to add:
[Service]
SystemCallArchitectures=
SystemCallFilter=
to the systemd service file?
zeripath
on 11 Sep 2020
@zeripath I would prefer not, this is a security feature. First of all, it seems to only fail under ArchLinux ARM, so this should rather be changed there than in the standard Arch (ARM variant is a separate distro that sync most of our PKGBUILD but carry specfic changes). Then, it would be better to determine what should be allowed (see https://www.freedesktop.org/software/systemd/man/systemd.exec.html#System Call Filtering for documentation) rather than allowing everything.
ArchangeGabriel
on 11 Sep 2020
Fair enough.
Presumably the arch arm systemd has some default set for these values which is relatively restrictive - I guess we would need an arch user to play the whack-a-mole game to figure out what calls are actually needed.
I'll have a think about whether we can actually avoid these bash calls - I think if we restructure things slightly we should be able to do that. I'm not sure that would necessarily reduce the syscall problem - it might do and getting rid of the bash would likely be helpful anyway
zeripath
on 11 Sep 2020
- As far as I can tell, including
SystemCallArchitectures=seems to be unnecessary. I don't know if it is recommended for other reasons, but it's not necessary for the workaround to do its job. - I already tried
SystemCallFilter=+@setfor all specified sets in the https://www.freedesktop.org/software/systemd/man/systemd.exec.html documentation, but that didn't do anything. - Then I compared the
systemd-analyze syscall-filteroutput between Manjaro and Arch ARM and saw no differences of consequence:
Manjaro showed
# Unlisted System Calls (supported by the local kernel, but not included in any of the groups listed above):
# mincore
# seccomp
# syslog
where ARM showed an extra call faccessat2 in the set @file-system and an extra call cacheflush in the set @default. So I doubt there's anything to gain by specifically setting SystemCallFilter=+some_call, it might just be a bug.
ltGuillaume
on 12 Sep 2020
Is it possible that the cat binary is a different arm architecture binary so the issue is that systemd is preventing the call?
zeripath
on 12 Sep 2020
Is it possible that the cat binary is a different arm architecture binary so the issue is that systemd is preventing the call?
I would think that in that case SystemCallArchitectures= would be necessary for the workaround to function. As stated before, it's not.
I copied over cat, here it is. It says ARM HF ABI:5, just like the gitea executable.
cat.zip
ltGuillaume
on 12 Sep 2020
Here's a very similar issue with havegd: https://archlinuxarm.org/forum/viewtopic.php?f=15&t=14549
ltGuillaume
on 12 Sep 2020
If I _only_ add the following line (discarding SystemCallArchitecture= and SystemCallFilter=), pushing commits also works:
[Service]
SystemCallErrorNumber=EPERM
@ArchangeGabriel I suppose this would be preferable, right? Or does this imply the otherwise explicitly added SystemCallFilter= somehow? I couldn't find that in the documentation, so I'm guessing it doesn't.
------- Edit -------
This is what's in /usr/lib/systemd/system/gitea.service
[Unit]
Description=Gitea (Git with a cup of tea)
After=syslog.target
After=network.target
After=mysqld.service
After=postgresql.service
After=memcached.service
After=redis.service
[Service]
User=gitea
Group=gitea
Type=simple
WorkingDirectory=~
RuntimeDirectory=gitea
LogsDirectory=gitea
StateDirectory=gitea
Environment=USER=gitea HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea
ExecStart=/usr/bin/gitea web -c /etc/gitea/app.ini
Restart=always
RestartSec=2s
CapabilityBoundingSet=
NoNewPrivileges=True
#SecureBits=noroot-locked
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/etc/gitea/app.ini
PrivateTmp=true
PrivateDevices=true
PrivateUsers=true
ProtectHostname=true
ProtectClock=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectKernelLogs=true
ProtectControlGroups=true
LockPersonality=true
MemoryDenyWriteExecute=true
RestrictRealtime=true
RestrictSUIDSGID=true
SystemCallArchitectures=native
SystemCallFilter=@system-service
[Install]
WantedBy=multi-user.target
So, no SystemCallErrorNumber was specified.
I added SystemCallErrorNumber=EPERM to the file, removed all the overrides and all worked fine.
So it seems we just need to get SystemCallErrorNumber=EPERM into gitea.service by default.
ltGuillaume
on 12 Sep 2020
@ltGuillaume good thinking.
I think that could be added to our documentation too.
zeripath
on 12 Sep 2020
I'm hoping this will get picked up https://bugs.archlinux.org/task/60669#comment192661 (it's assigned to @ArchangeGabriel)
@ArchangeGabriel Also, this bug indeed mentions that the current hardening isn't very restrictive at all: it uses the set @system-service, which is a rather large superset. If you/we could restrict it more specifically to gitea's tasks, the syscall hardening would be way more useful.
My thinking is, though, that on Arch ARM currently something is already not permitted around cat with the current @system-service syscall hardening, otherwise the SystemCallErrorNumber wouldn't have mattered.
ltGuillaume
on 12 Sep 2020
Great, looks like everything's implemented. The Wiki's been updated, too 馃憤
ltGuillaume
on 14 Sep 2020
Related issues
kifirkin
路
3Comments
jonasfranz
路
3Comments
lunny
路
3Comments
ghost
路
3Comments
BNolet
路
3Comments