Ambassador: Dev Portal attempts to find docs for every mapping by default

[pawanyoda]

Describe the bug
Hi, I am using Ambassador 1.41. and Isito 1.51 on AWS. Istio is working file without mtls. and requests are working through mappings and here using I'm self-signed certificate and its working but additionally every 10, 15 seconds or on every hitting request its getting 404. don't know what is exactly needs output, hitting to .ambassador-internal/openapi-docs

Ambassador pod logs:

2020-04-20T07:08:47.820739878Z ACCESS [2020-04-20T07:08:40.299Z] "GET /test/ghost/.ambassador-internal/openapi-docs HTTP/1.1" 404 - 0 183 1 0 "10.191.243.124" "Go-http-client/1.1" "47376ecf-4352-9420-ba44-5fed720f64f0" "127.0.0.1:8443" "172.20.255.115:9091"
2020-04-20T07:08:47.820742511Z ACCESS [2020-04-20T07:08:40.303Z] "GET /test/ghost2/.ambassador-internal/openapi-docs HTTP/1.1" 404 - 0 181 1 0 "10.191.243.124" "Go-http-client/1.1" "2885effe-5f4d-92c4-a58b-0f39fdd465eb" "127.0.0.1:8443" "172.20.132.127:9092"

Istio proxy logs:

2020-04-20T06:32:41.295495213Z [2020-04-20T06:32:40.287Z] "GET /test/ghost/.ambassador-internal/openapi-docs HTTP/1.1" 404 - "-" "-" 0 181 0 0 "10.191.243.124" "Go-http-client/1.1" "81fd0683-505e-9191-8711-81c39ec074fa" "127.0.0.1:8443" "127.0.0.1:9091" inbound|9091|http|ghost-http-svc.namespace.svc.cluster.local 127.0.0.1:50018 10.191.243.90:9091 10.191.243.124:0 - default
2020-04-20T06:33:41.295919366Z [2020-04-20T06:33:40.300Z] "GET /test/ghost2/.ambassador-internal/openapi-docs HTTP/1.1" 404 - "-" "-" 0 181 0 0 "10.191.243.124" "Go-http-client/1.1" "d5dd610f-67ff-9b0d-add3-a26aacca1faa" "127.0.0.1:8443" "127.0.0.1:9092" inbound|9092|http|ghost2-http-svc.namespace.svc.cluster.local 127.0.0.1:51286 10.191.243.90:9092 10.191.243.124:0 - default

TLS log from ambassador pod

2020-04-20T06:53:33.863263626Z status_dict {'Error check': {'status': True, 'specifics': [(True, 'No errors logged')]}, 'TLS': {'status': True, 'specifics': [(True, '1 TLSContext is active')]}, 'Mappings': {'status': True, 'specifics': [(True, '15 Mappings are active')]}}

Expected behavior
why this 404 getting
Versions (please complete the following information):

• Ambassador: [1.4.1]
• Kubernetes environment [e.g. Minikube, bare metal, Google Kubernetes Engine]
• Version [1.15]

more logs

2020-04-20T07:00:26.322180559Z [2020-04-20 07:00:26.322][262][debug][http] [source/common/http/conn_manager_impl.cc:708] [C184840][S16622113245629541650] request headers complete (end_stream=true):
2020-04-20T07:00:26.322185245Z ':authority', 'elb.us-east-1.elb.amazonaws.com'
2020-04-20T07:00:26.322188905Z ':path', '/test/ghost'
2020-04-20T07:00:26.322192904Z ':method', 'GET'
2020-04-20T07:00:26.322195883Z 'user-agent', 'curl/7.64.1'
2020-04-20T07:00:26.322198933Z 'accept', '/'
2020-04-20T07:00:26.32220149Z 'test', 'dev'
2020-04-20T07:00:26.322204555Z
2020-04-20T07:00:26.322207164Z [2020-04-20 07:00:26.322][262][debug][http] [source/common/http/conn_manager_impl.cc:1257] [C184840][S16622113245629541650] request end stream
2020-04-20T07:00:26.322331695Z [2020-04-20 07:00:26.322][262][debug][router] [source/common/router/router.cc:434] [C0][S17947907241811361392] cluster 'cluster_extauth_127_0_0_1_8500_ambassador' match for URL '/envoy.service.auth.v2alpha.Authorization/Check'
2020-04-20T07:00:26.322337215Z [2020-04-20 07:00:26.322][262][debug][router] [source/common/router/router.cc:549] [C0][S17947907241811361392] router decoding headers:
2020-04-20T07:00:26.3223396Z ':method', 'POST'
2020-04-20T07:00:26.322341612Z ':path', '/envoy.service.auth.v2alpha.Authorization/Check'
2020-04-20T07:00:26.322343627Z ':authority', 'cluster_extauth_127_0_0_1_8500_ambassador'
2020-04-20T07:00:26.322345752Z ':scheme', 'http'
2020-04-20T07:00:26.322347754Z 'te', 'trailers'
2020-04-20T07:00:26.322349516Z 'grpc-timeout', '5000m'
2020-04-20T07:00:26.322351319Z 'content-type', 'application/grpc'
2020-04-20T07:00:26.322353166Z 'x-ot-span-context', 'EhQJrypZ/X31wLQR+wYi4lTiyG4YAQ=='
2020-04-20T07:00:26.322355185Z 'x-envoy-internal', 'true'
2020-04-20T07:00:26.322370444Z 'x-forwarded-for', '10.191.243.124'
2020-04-20T07:00:26.322373357Z 'x-envoy-expected-rq-timeout-ms', '5000'
2020-04-20T07:00:26.322376241Z
2020-04-20T07:00:26.322379341Z [2020-04-20 07:00:26.322][262][debug][pool] [source/common/http/http2/conn_pool.cc:98] [C42] creating stream
2020-04-20T07:00:26.322401258Z [2020-04-20 07:00:26.322][262][debug][router] [source/common/router/router.cc:1618] [C0][S17947907241811361392] pool ready
2020-04-20T07:00:26.322754595Z time="2020-04-20 07:00:26" level=info msg="[gRPC] HTTP/1.1 GET elb.us-east-1.elb.amazonaws.com /test/ghost" MAIN=http REQUEST_ID=16622113245629541650 SUB=http-handler
2020-04-20T07:00:26.322771504Z time="2020-04-20 07:00:26" level=info msg="using default rule" MAIN=http REQUEST_ID=16622113245629541650 SUB=http-handler
2020-04-20T07:00:26.322774557Z time="2020-04-20 07:00:26" level=info msg="selected rule host=\"\", path=\"\", filters=[]" MAIN=http REQUEST_ID=16622113245629541650 SUB=http-handler
2020-04-20T07:00:26.322786238Z time="2020-04-20 07:00:26" level=info msg="[gRPC] *filterapi.HTTPRequestModification : 0 headers (66.904µs)" MAIN=http REQUEST_ID=16622113245629541650 SUB=http-handler
2020-04-20T07:00:26.3229688Z [2020-04-20 07:00:26.322][262][debug][router] [source/common/router/router.cc:1036] [C0][S17947907241811361392] upstream headers complete: end_stream=false
2020-04-20T07:00:26.322975446Z [2020-04-20 07:00:26.322][262][debug][http] [source/common/http/async_client_impl.cc:93] async http request response headers (end_stream=false):
2020-04-20T07:00:26.32297907Z ':status', '200'
2020-04-20T07:00:26.3229824Z 'content-type', 'application/grpc'
2020-04-20T07:00:26.322985791Z 'trailer', 'Grpc-Status'
2020-04-20T07:00:26.322988837Z 'trailer', 'Grpc-Message'
2020-04-20T07:00:26.322991246Z 'trailer', 'Grpc-Status-Details-Bin'
2020-04-20T07:00:26.32301797Z 'x-envoy-upstream-service-time', '0'
2020-04-20T07:00:26.323021968Z

[adamtajti]

Hi @pawanyoda , this is a feature of Ambassador that is documented over here:
https://www.getambassador.io/docs/latest/topics/using/dev-portal/

You basically gain the possibility of exposing your Swagger / OpenAPI documentation over that .ambassador-internal/openapi-docs address and it'll show up on $(ambassador-host)/docs.

If all the services provide documentation like this, then the developers internally / externally can navigate around the provided services more easily.

[christian-roggia]

+1 for this ticket, I would also like to disable this feature as it is generating large numbers of logs making it harder to debug issues with Ambassador.

we are not using this feature but I couldn't find how to disable the requests in the documentation.

p.s. we already disabled in the Helm chart the mappings with createDevPortalMappings: false but we can't disable the developer portal entirely.

[beeant]

Yes, I would like to disable this too.

[e1011215]

this is quite a misfeature. Please make it optional and also make it possible to change the URL.

[manigandham]

+1 for disabling this. It generates a lot of error logs.

It would probably be better to have it disabled by default and enable it in each Mapping (since it requires a custom .ambassador-internal endpoint anyway).

[kflynn]

(I updated the title to make it easier to find.)

[jambonrose]

The inability to disable the documentation-search feature (or to control it on a per-mapping basis) is not only a problem from a logging and noise perspective, but also a cost perspective. Some log/error services charge you by the number of events you generate, so Ambassador's behavior on this front actually costs us money.

Increasing the priority of this issue would be appreciated. If there is a way we can help contribute changes on this front, I'm all ears.

[kflynn]

As of Ambassador 1.5.0, you can set the environment variable POLL_EVERY_SECS to 0 to disable the dev portal (and 1.5.4 logs much, much, much less in general).

[jambonrose]

Thanks for the info @kflynn : that did the trick for us. It's a shame we can't set it on a per-mapping basis, but the reduction of logs and errors is welcome.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tvvignesh]

Not stale. I still see this in the latest version of ambassador even when I have disabled api docs in the helm chart.

[vsimon]

even when I have disabled api docs in the helm chart

How was it defined? I found setting the environment variable POLL_EVERY_SECS to 0 like in the comment https://github.com/datawire/ambassador/issues/2619#issuecomment-656779216 does seem to work to disable these requests (no longer see the requests in the logs).

For the helm chart, defined it in the env section.

    env:
      POLL_EVERY_SECS: 0

[tvvignesh]

@vsimon OOPS! I missed that comment. Just tried it now and it works. Thanks 👍

[khussey]

As of Ambassador 1.9.0, which is now available, the Dev Portal no longer automatically polls .ambassador-internal/openapi-docs. See https://www.getambassador.io/docs/pre-release/topics/using/dev-portal/#developer-portal for more details.

[kivo360]

I have a question about the dev-portal. I'm looking to write a python client that involves Ambassador. I realize openapi-generator-cli can easily be used to codegen yaml files based on a given configuration. I've used it with other APIs. Are there any OpenAPI/Swagger json/yaml configurations for the Ambassador API so I can run codegen on it?