Envoy: DNS filter - enhancement proposal

[janosi]

Issue Template

Title: DNS filter - enhancement proposal

Description:
Problem: the most generic way of service discovery is a DNS query for an application. That is, if an application uses DNS queries to find its peers this behavior ensures the best portability for the app. On the other hand, we are aware of the drawbacks of DNS queries: the intermediate caches may result in stale records in the answers, so the application may have to execute failure handling for a long period of time. Also new upstream peers will "appear" for the application after the cache is invalidated.

How a DNS filter in Envoy could help: We could utilize that Envoy has up to date information about the upstream peers. In a service mesh concept an Envoy sidecar is deployed in every pod in a Kubernetes environment. Envoy is updated about the state of other application pod instances via the Envoy v2 xSD API.

Idea: The internal iptables rules (or BPF) of the pod are configured so, that outgoing DNS queries (both UDP and TCP based) are redirected to the Envoy sidecar. Envoy should support UDP as an additional protocol beside TCP, and there shall be a listener to listen on TCP/UDP:53. Incoming requests coming in on that port are directed to the DNS filter. The DNS filter processes the request: if the request is for a domain/cluster that Envoy knows (based on the xDS API) then Envoy either answers with “localhost” (127.0.0.1 or [::1]) so it "calls the outgoing request to itself", or Envoy answers directly with an upstream host’s IP address. If the request is for another domain, Envoy forwards the request to an external DNS server (i.e. it acts as a DNS proxy). At the same time Envoy may ask for information via its xDS API about the unknown cluster/domain.

Benefits: It is a merge of DNS based service discovery on application side and Envoy xDS API based service discovery on “infrastructure” side. The application receives up to date information in the DNS answers.

[mattklein123]

Interesting idea. Marking design proposal wanted for now. Obviously depends on UDP proxy being implemented (unless TCP DNS were used).

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.

[sumukhs]

I have a slightly related requirement where clusters can potentially mark their endpoints to be addressable by DNS names. With this information, Envoy would know how to resolve dns queries.

Today, we are implementing this logic in a separate DNS server by mapping the dns name to the cluster endpoint, but would be better to integrate it into Envoy.

Is there anyone else working on something related? Is there any push back on Envoy acting as a DNS Server?