Caddy: fcgi: invalid header version

Created on 23 Sep 2016 · 14Comments · Source: caddyserver/caddy

1. What version of Caddy are you running (`caddy -version`)?

0.9.2

2. What are you trying to do?

Use Caddy with php-fpm started with startup command.
I am using a lumen framework.

3. What is your entire Caddyfile?

:80 {
  root /app/public
  fastcgi / 127.0.0.1:9000 php
  rewrite {
    to {path} {path}/ /index.php?{query}
  }
  cors
  gzip
  log stdout
  prometheus 0.0.0.0:9180
  startup /usr/sbin/php-fpm7.0 -D -O

4. How did you run Caddy (give the full command and describe the execution environment)?

Caddy is dockerized.
And I send a GET request with curl. It is working half of the time (1 in 2). It works, it doesn't, it works, it doesn't.

curl -v 'localhost/_health'

5. What did you expect to see?

A json repsonse.
And a 200 return code

172.19.0.1 - [23/Sep/2016:11:42:19 +0200] "GET /index.php HTTP/1.1" 200 188

6. What did you see instead (give full error messages and/or log)?

A 502 error and this entry in caddy logfile:

172.19.0.1 - [23/Sep/2016:11:42:21 +0200] "GET /index.php HTTP/1.1" 502 16
23/Sep/2016:11:42:21 +0200 [ERROR 502 /index.php] fcgi: invalid header version

7. How can someone who is starting from scratch reproduce this behavior as minimally as possible?

I can't share my source code.

bug

Source

commarla

👍1

All 14 comments

I'm having the same problem with version 0.9.2. Every second request gives a 23/Sep/2016:13:04:12 +0200 [ERROR 502 /] fcgi: invalid header version

The problem can be reproduced with a new laravel installation if you have php7.0-fpm and composer installed.


cd /tmp
composer create-project --prefer-dist laravel/laravel test

Use the following Caddyfile

localhost:80 {
    tls off
    root /tmp/test/public
    errors visible

    fastcgi / /var/run/php/php7.0-fpm.sock php {
        index index.php
    }

    rewrite {
        to {path} {path}/ /index.php?{query}
    }
}

Then start caddy

➜  /tmp/test  caddy -version                      
Caddy 0.9.2

➜  /tmp/test  caddy -conf /tmp/test/Caddyfile

Then

➜  ~  curl localhost
<!DOCTYPE html>...

➜  ~  curl localhost
23/Sep/2016:13:10:59 +0200 [ERROR 502 /] fcgi: invalid header version

➜  ~  curl localhost
<!DOCTYPE html>...

➜  ~  curl localhost
23/Sep/2016:13:10:59 +0200 [ERROR 502 /] fcgi: invalid header version

tobias-kuendig on 23 Sep 2016

The cause is probably related to this https://github.com/mholt/caddy/issues/1123

abiosoft on 23 Sep 2016

I don't see any tcp connection related logs.

commarla on 23 Sep 2016

FastCGI is currently reusing connections (since 0.9.2); so this can be due to writing the headers on an existing connection.

abiosoft on 23 Sep 2016

👍1

Thanks for the explanation @abiosoft

commarla on 23 Sep 2016

Just started using Caddy and found it very odd that it would drop every second request! Is there a workaround, or just roll back to .9.1 for now?

timothyallan on 24 Sep 2016

@timothyallan The fix for this was just merged in #1129 -- so if you can (or want to) build Caddy from source, you can begin using it immediately. But I plan to roll out a patch release within the week.

Thank you @abiosoft!

mholt on 24 Sep 2016

🎉1

@timothyallan you can roll back to 0.9.1 or build from source.

abiosoft on 24 Sep 2016

I have built the master and everything works fine. Thanks @abiosoft @Echsecutor and @mholt !

commarla on 24 Sep 2016

This bug is indeed due to reusing connections. More precisely, due to my quick and dirty serialization instead of proper fastCGI demuxing, it can happen that an empty header (or body) is received. The former is invalid.

The current master does not use persistent connections any more, hence the bug is gone, but also the feature. ;) I guess 'we are working on it' ;)

Echsecutor on 25 Sep 2016

@Echsecutor But persistent connections can be turned on with the pool parameter now, right? Does the bug still surface when pool 1 is used?

mholt on 25 Sep 2016

The feature is not gone but rather it is now opt-in. So yes, using pool n where n > 0 enables it.

And nope the bug doesn't surface anymore whether or not persistent connection is used.

abiosoft on 25 Sep 2016

jepp, I see that this is how it's intended. I am just saying that also for pool n > 0 on the current master no persistent connections are being used. Hence @commarla 's comment (that things are working fine now) is due to the fact that persistent connections are currently not working. Wether they work as intended is still up to testing once a minor problem in the current master is fixed. https://github.com/mholt/caddy/pull/1129#issuecomment-249391582

Echsecutor on 25 Sep 2016

A I meant this comment: https://github.com/mholt/caddy/pull/1129#pullrequestreview-1443068. Anyway in PR https://github.com/mholt/caddy/pull/1134 pooling is working and I can not reproduce this bug any more (it was also showing up in my setup). At least demuxing errors for the body should now also be caught by the tests.

Echsecutor on 25 Sep 2016

Was this page helpful?

0 / 5 - 0 ratings