Ambassador: Ambassador overwrites value of x-request-id header.

[eddy-curv]

Describe the bug
Ambassador overwrites value of x-request-id header.

To Reproduce
We use ambassador as our API gateway (with default configuration) in GKE. A Javascript frontend sends an x-request-id header with UUID as a value for tracing capabilities.

Expected behavior
The value of x-request-id header that was sent from the frontend to the backend via Ambassador shouldn't be changed.

Versions (please complete the following information):

• Ambassador: 0.85.0
• GKE 1.14.7

Additional context
Since the default behaviour of envoy is to overwrite x-request-id, there is an option to override this behaviour by setting preserve_external_request_id: true when configuring envoy using config.yaml.

The question is how to do this in Ambassador using ambassador.yaml? I believe that it's a one-liner solution but this option isn't documented anywhere.

[kflynn]

This is probably most easily done by modifying the ambassador Module to support preserve_external_request_id, then modifying the V2Listener to honor the setting.

[keshavmaheshwari]

Any updates on this?

[danielbryantuk]

We've got another +1 on this from the Ambassador OSS Slack: https://datawire-oss.slack.com/archives/CAULN7S76/p1585737550177100

[prakharjoshi]

The changes for the fix have been merged.

[eddy-curv]

Despite using version 1.5.0 of Ambassador OSS and adding preserve_external_request_id to ambassador.yaml, x-request-id is still overridden by Ambassador.
I inspected the aconf.json and ir.json and found out that ir.json lacks of preserve_external_request_id, thus line 805 of v2listener.py will always be false.

[eddy-curv]

Additionally, if I use preserve_external_request_id but have TracingService enabled, which once gets precedence? will x-request-id be overwritten regardless?

[prakharjoshi]

@eddy-curv Thanks for pointing that I am looking into the issue. Will keep posting the updates. Thanks

[eddy-curv]

Thanks @prakharjoshi, can you also address my question about TracingService?
If preserve_external_request_id is true but I also have TracingService enabled, will the value of x-reqeust-id be overridden by TracingService?

I'm asking this because from the following code in v2listener.py it's not clear what is going to happen:

    if 'preserve_external_request_id' in self.config.ir.ambassador_module:
        self.base_http_config["preserve_external_request_id"] = self.config.ir.ambassador_module.preserve_external_request_id


    if self.config.ir.tracing:
        self.base_http_config["generate_request_id"] = True

The documentation states that "When enabled, the TracingService will instruct Ambassador Edge Stack to initiate a trace on requests by generating and populating an x-request-id HTTP header", common sense tells that TracingService shouldn't override the value of x-reqeust-idif it already exists in the request.

[prakharjoshi]

@eddy-curv Envoy document suggests that if preserve_external_request_id is present, tracing should not override that request ID.

[eddy-curv]

Thanks @prakharjoshi, really hope that the IR module flow will be fixed, we would really like to use that preserve_external_request_id ability.

[prakharjoshi]

@eddy-curv Ping!! Can you check with the new ambassador version?

[eddy-curv]

@prakharjoshi The functionality works as expected, many thanks!

[eddy-curv]

After closer inspection, it seems like the 14th character of x-reqeust-id value will be overwritten due to envoy's behaviour.

for example, aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee will be modified to aaaaaaaa-bbbb-9ccc-dddd-eeeeeeeeeeee.

Can something be done about it or it depends on envoy's implementation only?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.