Envoy: Envoy does not modify `content-length` header after applying BodyFormat to error responses from External Authorization service

[alkov-ibm]

Title: Envoy does not modify content-length header after applying BodyFormat to error responses from External Authorization service.

Description:
When using a localReply Envoy treats responses from External Authorization Service as 'local' (I might argue that external authorization responses should not be treated as 'local' but that may be 'as designed'). However, when applying a body format the content-length header returned for the original body of the external authorization response is not modified after applying the body format. This causes the content-length response header and the actual length of the body to be mismatched.

Repro steps:

• Configure an external http authorization service
• Configure local reply to apply a new Body Format
  For example a JSON format:

            "local_reply_config": {
             "body_format": {
              "json_format": {
               "code": "%RESPONSE_CODE%",
               "error": "%LOCAL_REPLY_BODY%",
               "trace": "%REQ(X-REQUEST-ID)%"
              }
             }
            }

• Make request that fails authorization

For example, response direct from authorization service:

< content-length: 12
< 
Unauthorized

Response returned by Envoy

< content-length: 12
< x-request-id: b87918f4-5b88-4b85-b150-77e8310a4436
< 
* Excess found in a non pipelined read: excess = 71, size = 12, maxdownload = 12, bytecount = 0
{"trace":"b8

Logs:

{"bytes-sent":83,"response-duration":null,"x-forwarded-for":"172.17.0.1","method":"GET","x-global-transaction-id":"1722909b-70e9-46f9-8ebe-23a19cd3b5a5","response-flags":"UAEX","status-details":"ext_authz_denied","response-tx-duration":null,"upstream-cluster":null,"type":"ACCESS","time_date":"2020-08-17T18:21:36.613Z","upstream-host":null,"status":401,"x-aaa-userinfo":null,"host":"test-service.com:32500","upstream-path":"/xxx/ok","bytes-received":0,"path":"/xxx/ok","request-duration":4,"upstream-tls-fail":null,"duration":192,"x-request-id":"1722909b-70e9-46f9-8ebe-23a19cd3b5a5"}

If other information is needed to reproduce/debug I can, but was having issues sanitizing the large amount of data.

[qiwzhang]

Found the problem:

In this sendLocalReply function.

ContentType and ContentLength response headers set by this line

may be overrided by encode_header call in this line

Some encode_header function may have modify_headers function to modify the headers, such as
this line
and this line

[mattklein123]

cc @alyssawilk

[qiwzhang]

Hi @mattklein123 you can assign this one to me. I am working on it now