Authelia: 500 Internal Server Error on json post

Created on 9 Oct 2017 · 5Comments · Source: authelia/authelia

So I have an nginx proxy set up as a portal that forwards traffic to different web servers based on domain name. On each web server, I've got another nginx proxy that handles https and then forwards traffic onto the actual web server (which may be running on any number of web servers). I have it set up this way primarily to standardize configuration so that I can maintain servers via saltstack.

Anyway, I've only (currently) want authelia to authenticate traffic that comes through my portal (and thus the open web), not internal, so that's where I have authelia set up. But for at least some of my servers (I haven't done a ton of testing yet to see which are affected by this), authelia appears to break json posts. This is from a login attempt to my TeamPass server:

POST https://pass.mydomain.com/sources/identify.php

Request Headers:

Accept  
application/json, text/javascript, */*; q=0.01
Accept-Encoding 
gzip, deflate, br
Accept-Language 
en-US,en;q=0.5
Content-Length  
244
Content-Type    
application/x-www-form-urlencoded; charset=UTF-8
Cookie  
KEY_PHPSESSID=VTSbqvRSAjLK80z8c3r89Es4U5p9aa4YHB%2FYtiGDmWHoVptY8%2Bg73rtXNtQwkXJQGyRAeXo91qnwu228w%2Ff8ZA
%3D%3D; PHPSESSID=g1dg1u0qfgt9jm065t7sobsj8r; 34bc6dbd03c68ab12651a771f86f6f2904d120be5a1ba72f69=5781fce91aa51c314d15117f341f5cc22e57fd2995a59bb343
; connect.sid=s%3A_gFSXcpzwc09m38qsR184o2C0aue83nG.AJjtJUpXhPXeKp9TRaMa0ipbbVIPmxg8%2Bw78R7iO3NI
Host    
pass.mydomain.com
Referer 
https://pass.mydomain.com/index.php?page=items
User-Agent  
Mozilla/5.0 (X11; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0
X-Requested-With    
XMLHttpRequest


Post:

data    WwNNmrOw2ll/CxZl/i6KBpvEg1OuUD8bywrS6RsOgtksQVBizwMJTe229dXQwUiTaHQKKhwif8PN1hgBB4n/NiRJyGgHmK0zo3EX0kFi13hO/Q5xLXoTthlTdeu54AuXVXCXZTKmO7uycCEZfbgH2XsIR7G2OEyzbuDxq1ta+8ok6kunf2K7n5QsL5GW7PyMrEutlHkPAK6pn2U=
type    identify_user


Response:
<html>
<head><title>500 Internal Server Error</title></head>
<body bgcolor="white">
<center><h1>500 Internal Server Error</h1></center>
<hr><center>nginx/1.12.1</center>
</body>
</html>


Nginx Error log:
2017/10/08 22:29:53 [error] 18540#18540: *10 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 71.89.26.224, server: pass.mydomain.com, request: "POST /sources/identify.php HTTP/1.1", subrequest: "/auth_verify", upstream: "https://172.16.41.37:443/verify", host: "pass.mydomain.com", referrer: "https://pass.mydomain.com/index.php?page=items"
2017/10/08 22:29:53 [error] 18540#18540: *10 auth request unexpected status: 504 while sending to client, client: 71.89.26.224, server: pass.mydomain.com, request: "POST /sources/identify.php
HTTP/1.1", host: "pass.mydomain.com", referrer: "https://pass.mydomain.com/index.php?page=items"

I get that result if the authelia configuration is placed on either one of the proxies, but it works fine if I move it to the nginx configuration for the teampass server itself.
Proxy (on teampass server) configuration w/ authelia - doesn't work:

server {
  listen 443;            
  server_name pass.mydomain.com;

  resolver 172.16.40.20;
  set $backend "http://pass.mydomain.com:8080"; 

  ssl on;
  ssl_certificate /etc/nginx/certs/pass.mydomain.com/fullchain.pem;
  ssl_certificate_key /etc/nginx/certs/pass.mydomain.com/privkey.pem;
  ssl_session_cache shared:SSL:10m;
  client_max_body_size 1024m;

  ###authelia
  location /auth_verify {
    internal;
    proxy_set_header  X-Original-URI $request_uri;
    proxy_set_header  X-Real-IP $remote_addr;
    proxy_set_header  Host $http_host;

    proxy_pass        https://authelia.mydomain.com/verify;
  }
  ###

  location / {
    ###authelia
    auth_request /auth_verify;

    auth_request_set $redirect $upstream_http_redirect;
    proxy_set_header Redirect $redirect;

    auth_request_set $user $upstream_http_remote_user;
    proxy_set_header X-Forwarded-User $user;

    auth_request_set $groups $upstream_http_remote_groups;
    proxy_set_header Remote-Groups $groups;

    error_page 401 =302 https://authelia.mydomain.com?redirect=$redirect;
    error_page 403 = https://authelia.mydomain.com/error/403;
    ###

    proxy_pass $backend;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Protocol https;
    proxy_set_header X-Forwarded-Ssl on;


    # re-write redirects to http as to https, example: /home
    proxy_redirect http:// https://;
  }
}
server {
  listen 80 default_server;
  server_name _;
  rewrite ^ https://$host$request_uri? permanent;
}

Teampass nginx configuration - works:

server {
  server_name domain.tld www.domain.tld;
  root /usr/share/webapps/teampass;
  listen 8080;

  ###authelia
  location /auth_verify {
    internal;
    proxy_set_header  X-Original-URI $request_uri;
    proxy_set_header  X-Real-IP $remote_addr;
    proxy_set_header  Host $http_host;

    proxy_pass        https://authelia.mydomain.com/verify;
  }
  ###


  location / {
    ###authelia
    auth_request /auth_verify;

    auth_request_set $redirect $upstream_http_redirect;
    proxy_set_header Redirect $redirect;

    auth_request_set $user $upstream_http_remote_user;
    proxy_set_header X-Forwarded-User $user;

    auth_request_set $groups $upstream_http_remote_groups;
    proxy_set_header Remote-Groups $groups;

    error_page 401 =302 https://authelia.mydomain.com?redirect=$redirect;
    error_page 403 = https://authelia.mydomain.com/error/403;
    ###

    # try to serve file directly, fallback to front controller
    try_files $uri /index.php$is_args$args;
  }

  location ~ \.php$ {
    try_files $uri $document_root$fastcgi_script_name =404;
    fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi.conf;
  }

  error_log /var/log/nginx/teampass_error.log;
  access_log /var/log/nginx/teampass_access.log;
}

I'll keep playing around and see if I get a better idea of what's going on

Bug P1

Source

masau

All 5 comments

Hello @masau , you tried to post the same JSON data from to the two layers and one is working and the other is not, correct? Can it be a conflict between headers (overridden by apps while going through the layers)? Authelia is writing Remote-User and Remote-Groups for example.

I think Authelia's logs might help a bit, please set the level to debug.

clems4ever on 9 Oct 2017

Hi @masau,
Have you tried with the following lines in the /auth_verify Nginx location block?

proxy_pass_request_body off;
proxy_set_header Content-Length "";

n4kre on 9 Oct 2017

👍1

From Nginx documentation:

As the request body is discarded for authentication subrequests, you will need to set the proxy_pass_request_body directive to off and also set the Content-Length header to a null string.

n4kre on 9 Oct 2017

👍1

@clems4ever - It was the same data and it was going through both layers in both cases, the only difference was where I included the authelia configuration. When it was directly in the teampass server block it worked, but when it was in either of the proxies it would not. n4kre's suggestion resolved the problem - do you still want me to set authelia's log level to debug and send you logs?

@n4kre - that worked. I'll add it to my salt configuration and verify that it fixes issues on other servers as well. Thanks

EDIT: After a quick look it appears to have fixed the issues I was having with a few other servers as well. Thanks again

masau on 9 Oct 2017

👍1

@masau, great you solved it! No need to send the logs anymore. Thanks.

And thank you @n4kre.

clems4ever on 9 Oct 2017

Was this page helpful?

0 / 5 - 0 ratings