Caddy: http 502 - caddy proxies to svn server

[mkrensing]

1. What version of Caddy are you using (caddy -version)?

0.10.10
I can reproduce the error with linux and windows builds.

2. What are you trying to do?

Using Caddy as a proxy (HTTP and HTTPS) to a apache httpd with subversion module enabled

3. What is your entire Caddyfile?

http://svn.myhome.net {
  log stdout
  errors stderr
  proxy / http://127.0.0.1:8088 {
  }
}

4. How did you run Caddy (give the full command and describe the execution environment)?

caddy --conf Caddyfile

5. Please paste any relevant HTTP request(s) here.

6. What did you expect to see?

No error while "svn up"

7. What did you see instead (give full error messages and/or log)?

In TortoiseSVN:
Error GET request on
Error /svn/test/!svn/ver/1/foo/bar.mp3
Error failed: 502 Bad Gateway

8. How can someone who is starting from scratch reproduce the bug as minimally as possible?

Install apache + svn in docker using this Dockerfile:

FROM httpd:2.4
COPY ./public-html/ /usr/local/apache2/htdocs/
RUN mv /usr/local/apache2/conf/httpd.conf /usr/local/apache2/conf/httpd.conf_default
RUN apt-get update && apt-get install -y \
    subversion \
    libapache2-mod-svn \
 && rm -rf /var/lib/apt/lists/*
RUN mkdir /opt/svn
RUN svnadmin create /opt/svn/public
RUN svnadmin create /opt/svn/media
RUN chown -R daemon:daemon /opt/svn

Building the docker container using

docker run  -dit \
            -v $(pwd)/conf/httpd.conf:/usr/local/apache2/conf/httpd.conf \
            -v $(pwd)/conf/digest.passwd:/opt/svn/digest.passwd \
            -p 8088:8088 \
            --name apache_svn \
            apache_svn

Modify the httpd.conf

LoadModule dav_module           /usr/lib/apache2/modules/mod_dav.so
LoadModule dav_svn_module       /usr/lib/apache2/modules/mod_dav_svn.so
#SSLVerifyClient require

<Location /public>
  DAV svn
  SVNPath /opt/svn/public
  AuthType Digest
  AuthName "svn.myhost.net"
  AuthDigestDomain /public http://svn.myhost.net/public
  AuthDigestProvider file
  AuthUserFile /opt/svn/digest.passwd

  <LimitExcept GET PROPFIND OPTIONS REPORT>
    Require valid-user
  </LimitExcept>
</Location>

Add a valid user to the /opt/svn/digest.passwd file

[mholt]

Thanks for opening an issue with the template. The reproduction instructions are a bit involved, so it'll take some time before I get to this. Someone else is welcome to look into this in the meantime.

[whitestrake]

Does Caddy output any errors on stderr for these 502s?

[mkrensing]

Hi Whitestrake,

here are example log and errors outputs.

stdout:

10.0.2.2 - - [19/Feb/2018:19:14:48 +0000] "OPTIONS /media HTTP/1.1" 401 381
10.0.2.2 - - [19/Feb/2018:19:14:48 +0000] "OPTIONS /media HTTP/1.1" 200 185
10.0.2.2 - - [19/Feb/2018:19:14:48 +0000] "OPTIONS /media HTTP/1.1" 200 97
10.0.2.2 - - [19/Feb/2018:19:14:48 +0000] "PROPFIND /media HTTP/1.1" 207 641
10.0.2.2 - - [19/Feb/2018:19:14:48 +0000] "OPTIONS /media HTTP/1.1" 401 381
10.0.2.2 - - [19/Feb/2018:19:14:48 +0000] "OPTIONS /media HTTP/1.1" 200 185
10.0.2.2 - - [19/Feb/2018:19:14:48 +0000] "OPTIONS /media HTTP/1.1" 200 97
10.0.2.2 - - [19/Feb/2018:19:14:48 +0000] "PROPFIND /media HTTP/1.1" 207 641
10.0.2.2 - - [19/Feb/2018:19:14:49 +0000] "OPTIONS /media HTTP/1.1" 401 381
10.0.2.2 - - [19/Feb/2018:19:14:49 +0000] "OPTIONS /media HTTP/1.1" 200 185
10.0.2.2 - - [19/Feb/2018:19:14:49 +0000] "OPTIONS /media HTTP/1.1" 200 97
10.0.2.2 - - [19/Feb/2018:19:14:49 +0000] "PROPFIND /media HTTP/1.1" 207 641
10.0.2.2 - - [19/Feb/2018:19:14:49 +0000] "PROPFIND /media/!svn/vcc/default HTTP/1.1" 207 393
10.0.2.2 - - [19/Feb/2018:19:14:49 +0000] "PROPFIND /media/!svn/bln/12 HTTP/1.1" 207 446
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 401 381
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 200 185
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 200 97
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "PROPFIND /media HTTP/1.1" 207 641
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 401 381
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 200 185
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 200 97
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "PROPFIND /media HTTP/1.1" 207 641
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 401 381
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 200 185
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 200 97
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "PROPFIND /media HTTP/1.1" 207 641
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "PROPFIND /media/!svn/vcc/default HTTP/1.1" 207 393
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "PROPFIND /media/!svn/bln/12 HTTP/1.1" 207 446
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 401 381
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 200 185
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "OPTIONS /media HTTP/1.1" 200 97
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "PROPFIND /media HTTP/1.1" 207 641
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "GET /media/!svn/ver/3/fooo123123123.pdf HTTP/1.1" 200 1829522
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "GET /media/!svn/ver/3/bar9023423.pdf HTTP/1.1" 200 46793
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "GET /media/!svn/ver/3/fooo09203492034.zip HTTP/1.1" 200 2921627
10.0.2.2 - - [19/Feb/2018:19:14:58 +0000] "GET /media/!svn/ver/3/bar234234234.pdf HTTP/1.1" 200 902450

(... 10 more ...)

10.0.2.2 - - [19/Feb/2018:19:14:59 +0000] "REPORT /media/!svn/vcc/default HTTP/1.1" 200 15187402
10.0.2.2 - - [19/Feb/2018:19:14:59 +0000] "GET /media/!svn/ver/3/foo213423423.JPG HTTP/1.1" 200 1406375

(... 14 more ...)

errors:

19/Feb/2018:19:15:01 +0000 [ERROR 502 /media/!svn/ver/3/foo091834.pdf] context canceled
19/Feb/2018:19:15:01 +0000 [ERROR 502 /media/!svn/ver/3/bar0234902.pdf] context canceled
19/Feb/2018:19:15:01 +0000 [ERROR 502 /media/!svn/ver/3/foo1234234.xlsx] context canceled
19/Feb/2018:19:15:01 +0000 [ERROR 502 /media/!svn/ver/3/Info.txt] context canceled

thanks for your involvement!

[whitestrake]

If possible, can you test your SVN client locally against http://127.0.0.1:8088 directly, rather than http://svn.myhome.net:80, and confirm that the back end isn't issuing 502s?

[mkrensing]

I tested it locally against http://127.0.0.1:8088 directly and it works fine.
I tested it with TortoiseSVN and svn command line (against http://svn.myhome.net) and with different svn servers:

• Apache and mod_dav_svn on linux
• VisualSVN Server on windows

The result was always the same: 502 errors.

[mkrensing]

Every now and then there is another error displayed on the TortoiseSVN Client:

ra_serf: The server sent a truncated HTTP response body.

The strange thing is, there is no error logged by caddy on stderr that time.
The apache logged following errors:

[Wed Feb 21 22:09:27.473317 2018] [dav:error] [pid 90:tid 140079997634304] [client 172.17.0.1:35054] Unable to deliver content.  [500, #0]
[Wed Feb 21 22:09:27.476409 2018] [dav:error] [pid 90:tid 140079997634304] [client 172.17.0.1:35054] Could not write data to filter.  [500, #175002]

[mkrensing]

Funny story: After adding the following directive to the apache httpd.conf the 502 errors are gone!

SVNAdvertiseV2Protocol off

But the "ra_serf: The server sent a truncated HTTP response body." errors are still there.

The svnbook gave me the hint:

SVNAdvertiseV2Protocol On|Off

New to Subversion 1.7, this toggles whether mod_dav_svn advertises its support for the new version of its HTTP protocol also introduced in that version. Most admins will not wish to use this directive (which is On by default), choosing instead to enjoy the performance benefits that the new protocol offers. However, whena configuring a server as a write-through proxy to another server which does not support the new protocol, set this directive's value to Off.

[mkrensing]

to avoid misunderstandings: If I test it locally against http://127.0.0.1:8088 directly there are no errors,

• no 502
• no "ra_serf: The server sent a truncated HTTP response body."

[tikiatua]

Hi everyone,

I have a very similar problem when using caddy as reverse proxy for apache2-subversion. All requests to files in a subfolder will return a 502. Without any additional error information.

[mkrensing]

testing with caddy 0.11 it works well except for svn RENAME commands.

I used wireshare to detect the differences between "caddy proxy" way and the "direct" way.
The http COPY command failed with 502 error.
But after changig the "Destination" http header value it works well.

so I used the https://github.com/mholt/caddy/pull/2144 feature to rewrite the header value:

  proxy / http://127.0.0.1:8088 {
    header_upstream OriginalDestination {>Destination}
    header_upstream Destination "https://mydomainname.net(.*)" "http://127.0.0.1:8088/$1"
  }

👍

[275761919]

My problem is similar with this but the status code is 499 instead of 502 and the version of caddy is 0.11.1 through I also rewrite the header value following above.

[mholt]

@275761919 That is a different issue, and just means the client is cancelling the request before it is finished. (Please open new issues rather than using existing ones -- thanks)

[mholt]

Can anyone confirm if this is still an issue in v2?

[mkrensing]

I'm on it but need help with https://caddy.community/t/caddy-v2-reverse-proxy-rewrite-headers-with-header-up/6705

[mholt]

Pretty sure this is not a problem in v2. Closing unless it's shown to be an issue still.