Ambassador: Custom envoy config for Envoy access log

[Viacheslav-Akimov]

I am trying to reconfigure envoy acceess log pattern and so far the only way to do it in ambassador is to provide a custom envoy configuration. I try to create a configmap using default template as a value for envoy.j2 variable. However ambassador pod fails to start with a following error:

[2018-08-06 07:44:20.058][16][critical][main] source/server/config_validation/server.cc:48] error initializing configuration '/ambassador/ambassador-config-1-envoy.json': JSON supplied is not valid. Error(offset 1, line 1): Invalid value.
2018-08-06 07:44:20 kubewatch 0.36.0 INFO: Invalid envoy config

Looks like ambassador adds yaml header to the final json configuration through calling read_yaml method. See full log attached.

logs-from-ambassador-in-ambassador-558ccc6cb-lv6gb.txt

Versions:

• Ambassador: 0.37.0
• Kubernetes environment: Minikube 0.28.2
• Kubernetes 0.10.0

[alkiko]

Hi,

Is there any news on a fix or workaround on this one? I am trying to avoid 413 Payload Too Large by adding "per_connection_buffer_limit_bytes" with a higher value in envoy.j2.
When I run with the configmap named "ambassador-config" I get the same error with the yaml '---' header being added to the json, resulting in no envoy json configuration being generated:

2018-10-04 23:57:18 kubewatch 0.38.0 ERROR: ERROR ERROR ERROR Starting with configuration errors
2018-10-04 23:57:19 kubewatch 0.38.0 INFO: Scout reports {"latest_version": "0.40.0", "application": "ambassador", "notices": [], "cached": false, "timestamp": 1538697438.52049}
[2018-10-04 23:57:19.037][15][critical][main] source/server/config_validation/server.cc:48] error initializing configuration '/ambassador/ambassador-config-1-envoy.json': JSON supplied is not valid. Error(offset 1, line 1): Invalid value.

2018-10-04 23:57:19 kubewatch 0.38.0 INFO: Invalid envoy config
2018-10-04 23:57:19 kubewatch 0.38.0 INFO: ---
apiVersion: v0.1
kind: Pragma
ambassador_id: default
source: "ambassador-config envoy.j2"
autogenerated: true
---
{
  "listeners": [

    {
      "address": "tcp://0.0.0.0:80",

      "filters": [

(....)

Traceback (most recent call last):
  File "/ambassador/kubewatch.py", line 451, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/ambassador/kubewatch.py", line 431, in main
    sync(restarter)
  File "/ambassador/kubewatch.py", line 328, in sync
    restarter.restart()
  File "/ambassador/kubewatch.py", line 133, in restart
    config = self.generate_config(output)
  File "/ambassador/kubewatch.py", line 190, in generate_config
    raise ValueError("Unable to generate config")
ValueError: Unable to generate config
AMBASSADOR: kubewatch sync exited with status 1
Here's the envoy.json we were trying to run with:
ls: /ambassador/envoy*.json: No such file or directory
No config generated.
AMBASSADOR: shutting down (1)

[dln]

This is the same error as in #771 , no? Just ran into this too.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[hpurmann]

This issue should probably not be closed until the PR lands in master. @kflynn can you re-open please?

[krish512]

Infact, instead of envoy generating its own RequestID, can we use existing TraceID as generated by AWS ALBs so that it is easier to trace requests from cloud resources to applications

748 Comment

[krish512]

I achieved the above using following configuration,

apiVersion: getambassador.io/v2
kind: Module
metadata:
  name: ambassador
  namespace: ambassador
spec:
  config:
    envoy_log_format: >-
      [%START_TIME%] "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%
      %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT%
      %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%"
      "%REQ(USER-AGENT)%" "%REQ(X-AMZN-TRACE-ID)%" "%REQ(:AUTHORITY)%"
      "%UPSTREAM_HOST%"\n