Eshoponcontainers: [Question] How is HTTPS/SSL termination handled?

[PureKrome]

Hi all 👋

I'm trying to figure out how HTTPS / SSL is handled with respect to the Gateway APIs/BFF. What technology handles this? Sure, each Gateway API can be ASP.NET Core or ASP.NET (classic) or Java or Ruby, etc.. So i'm more interested in higher level concepts.

Lets review the existing architecture and using a simple scenario -> the mobile app (only because it's the first Client app in the picture):

So the mobile app communicates with the Mobile-Shopping BFF. This happens to be an ASP.NET Core 2.1 Web App. The Docker file only exposes port 80. There's also no mention of forcing HTTPS in the Startup.cs file.

So to me, this means there's no strict requirement for HTTPS between the Client App and the BFF. Actually, there's no HTTPS communication possible.

So, questions:

• Shouldn't we be enforcing HTTPS between Client Apps <--> Gateway APIs/BFF?
• If yes (to above), then where should this termination be handled?
• Should we be adding reverse proxies in front of Kestrel (for these ASP.NET Core 2.1 Web Api Apps?). Yes, Kestrel has had some hardening, but is this still a good practice to put _something_ in front of that?
• If yes again (to the reverse-proxy question), then should this be inside the Docker container/instance?

This all started out when I made my own Gateway API/BFF and thinking that is my public facing web server, I would need to make sure the communication is over HTTPS ... and then totally struggled/failed to get my Docker instance to work with a development cert.

When asking some other internet-friends for help, a suggestion I got was: "Don't handle the SSL termination in your container/instance, but have something in front, called an Ingress".
e.g.

So i'm now all confused :(

• What would this Ingress be?
• How would this Ingress be apart of the eShopOnContainers project / architecture?

And finally ... just to add one more item into all of this ... we heavily use/leverage Cloudflare for our existing projects and would love to keep using them ... so therefore I would also be adding them between the Client <--> BFF and again not sure if this impacts any of questions above (usually, not). Cloudflare does have the ability to terminate SSL and then they communicate between our public BFF's over HTTP instead.

• Is this the Ingress?
• Can we then IP whitelist the BFF's to Cloudflare (and possibly a few other IP's like out Office static IP or an Azure VNET IP which we have to VPN into)

Anyways, I sincerely hope this question makes sense.

Keep up the awesome work :)

[CESARDELATORRE]

You can handle SSL Termination in the ingress tier in Kubernetes and with NGINX tier in Kubernetes.
In MICROSOFT Azure you could also set SSL termination in “Azure Application Gateway” before getting to your Kubernetes/AKS cluster, so that SSL traffic is not handled by your cluster.

[PureKrome]

@CESARDELATORRE Ah ok .. so this is all new to me still 😊

So is all of this already in the eBook? I thought it might come under the last chapter "Securing .NET Microservices and Web Applications" but it's not. Mind you, i've not completed reading the entire book just yet, but for this problem had a quick scan through future upcoming chapters to see if/where/when i'll get to this.

[eiximenis]

Hi @PureKrome
An ingress is a service exposed outside that acts as a endpoint for other services. Conceptually it is similar to a reverse proxy (and technically is usually implemented using a reverse proxy).

In eShopOnContainers we are using a level 7 ingress (which currently is the only provided oob by Kubernetes). A level 7 ingress means it has knowledge of OCI level 7 and thus can made decisions based on http packages, instead of relying only on tcp/ip packages (as a level 4 ingress do).

So, we have only one service accessible from the outside, which is our ingress controller. It is implemented using nginx. We don't use https but if used, this is the place where you can put it. All traffic goes through ingress, and then ingress re-routes the traffic to specific service (i. e. a api gateway) using internal cluster communication (which can be http).

BTW adding SSL support to k8s is not a hard thing. The hardest part is having a certificate, valid for the DNS used by your cluster. Using a cert provider that supports the ACME protocol (like let's encrypt) it is possible to use cert-manager to issue a SSL certificate at runtime in a automated manner from the cluster. It is basically a configuration :)

So, the key points are:

1. You use ingress to expose all your services through our unique endpoint (in our case this endpoint is one public IP plus a path for each service (like mycluster/ordering-api or mycluster/basket-api. But, although a bit harder you can use subdomains too).
2. SSL should be configured at ingress endpoint. Communication between ingress and all other microservices can be using plain http
3. If using k8s it is possible to auto-issue a SSL certificate for the cluster if the CA accepts the ACME protocol.

Hope this helps :)

[PureKrome]

👋 @eiximenis thank you heaps for your reply! It's really, really helps. I'm really new to this (hence why I'm reading up on eShop, trying to learn all of this) so I still have some questions with respect to your answers. Apologies if they are really noobish :blush:

we have only one service accessible from the outside, which is our ingress controller.

and

All traffic goes through ingress, and then ingress re-routes the traffic to specific service (i. e. a api gateway) using internal cluster communication (which can be http).

Ok, so this means that _all traffic_ will pass through the ingress controller. The only way to access anything is via the Ingress? (My assumption from reading the answers above is: Yep)

You use ingress to expose all your services through our unique endpoint

Hold on .. _all services_ ? Or is this just all Gateways/BFF's? I thought the main idea with these microservices is that access is controlled via the Gateway APIs/BFF's?

SSL should be configured at ingress endpoint. Communication between ingress and all other microservices can be using plain http

Awesome :) I was wondering about this also (internal comms). I usually don't do any super secret govt classified stuff so my assumption here is that if the internal network has been compromised (and NCA or whatever is now snooping all of this traffic) then I should have bigger problems, than encrypted comms. :)

If using k8s it is possible to auto-issue a SSL certificate for the cluster if the CA accepts the ACME protocol.

Oh nice :)

---

Another side question: would information about this topic be useful in the eShop book? Or is this all considered out of scope, etc?

[eiximenis]

Hi again :)

the only way to access anything is via the Ingress?

Yes, but you can have more than on ingress controller listening on different endpoints.

Or is this just all Gateways/BFF's?

All you want to expose sorry. In our case is just the BFFs (and the clients like the webs).

I was wondering about this also (internal comms).

You can still use https internally if you want (although it can be very hard to issue all needed certificates bounded to internal dns used). Also, same applies to security: internal services can be unsecured (anonymous access, no token required) as long as external ones (api gateways) were secured. Or you can have internal services secured using same tokens as external ones (in this case api gateway just forwards the token to internal service. This is the approach used in eShopOnContainers), or you can have internal services secured with a different token than the external ones (so, api gateway also acts as a token issuer for the internal services).

[PureKrome]

Thanks again for all the help @CESARDELATORRE and @eiximenis - really appreciate this.

I'm going to close this issue as I think there's enough information here to get me working on this and testing this out.

One final suggestion - maybe add a section into the eBook about this type of stuff for other's to learn. Even an example would be awesome, too.

Keep up the great work - loving all this stuff! 🍰

[dmpx]

@eiximenis

Using SSL in nginx
ClientRedirectUris Table=>Field RedirectUri=> https://mgt.example.com/oidc/login-callback
Error: Sorry, there was an error : unauthorized_client

Modify: https://mgt.example.com/oidc/login-callback=> http://mgt.example.com/oidc/login-callback
Console input:

identity.api \| fail: IdentityServer4.Validation.AuthorizeRequestValidator[0] identity.api \| Invalid redirect_uri: http://mgt.example.com/oidc/login-callback identity.api \| { identity.api \| "ClientId": "backend", identity.api \| "ClientName": "Web Backend Client", identity.api \| "AllowedRedirectUris": [ identity.api \| "https://mgt.example.com/oidc/login-callback" identity.api \| ], identity.api \| "SubjectId": "anonymous", identity.api \| "RequestedScopes": "", identity.api \| "Raw": { identity.api \| "client_id": "backend", identity.api \| "redirect_uri": "http://mgt.example.com/oidc/login-callback", identity.api \| "response_type": "code id_token", identity.api \| "scope": "openid profile offline_access", identity.api \| "response_mode": "form_post", identity.api \| "nonce": "636822886415027417.MTdlNDViMzAtNWYwMC00Njk1LTljOWItNzYwMGEwYWYyNDU0YWI2MjliZWQtYWY4NS00NTlhLWFiYzktYTY2ZmZkMjFlOGUy", identity.api \| "state": "CfDJ8NAplNreT55NsfSnTsEIdC5YxQJJ-5PCeKaOW5tVVIILKLtJNjB66OKk9a58pI2Ky0kJSSsWxwtuycAZVIOCNjJYa4SbbtzjQFld444PN_POcJVKaFJgKQQEcr0Gwq0td2BLdyl7FlSM29uvVOJI1 l5MqAPM5U7zUjpKA0AVfCniojRUf7-ZcQtH2tdhT19TjEBk-DjSb3rYXVN7VTAoRMtkU9jYEshfqgbqtxayboDjvi3XnsN30AmVUUznWTEAgFG2-VKuJroXW_FOi-4yHH4aM6VVVLTN-43keP0_g_6xFWFFgw4yIVmmzLWYZlSDESKaXuCu1Oud40_n1gxHy tA", identity.api \| "x-client-SKU": "ID_NETSTANDARD2_0", identity.api \| "x-client-ver": "5.3.0.0" identity.api \| } identity.api \| } identity.api \| fail: IdentityServer4.Endpoints.AuthorizeEndpoint[0] identity.api \| Request validation failed

info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | Request starting HTTP/1.1 POST http://mgt.example.com/oidc/login-callback application/x-www-form-urlencoded 0
backend | 2019-01-05 20:49:03.6907|Info|Request starting HTTP/1.1 POST http://mgt.example.com/oidc/login-callback application/x-www-form-urlencoded 0
backend | trce: Microsoft.AspNetCore.HostFiltering.HostFilteringMiddleware[0]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | All hosts are allowed.
backend | 2019-01-05 20:49:03.6913|Debug|The request is insecure. Skipping HSTS header.
backend | dbug: Microsoft.AspNetCore.HttpsPolicy.HstsMiddleware[1]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | The request is insecure. Skipping HSTS header.
backend | dbug: Microsoft.AspNetCore.HttpOverrides.ForwardedHeadersMiddleware[1]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | Unknown proxy: [::ffff:47.75.75.47]:54338
backend | 2019-01-05 20:49:03.6913|Debug|Unknown proxy: [::ffff:47.75.75.47]:54338
backend | trce: Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector[31]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | Performing protect operation to key {0b113743-3113-4f53-bd2f-b01fd81156be} with purposes ('/app', 'SessionMiddleware').
backend | 2019-01-05 20:49:03.6924|Debug|POST requests are not supported
backend | dbug: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[1]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | POST requests are not supported
backend | trce: Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler[9]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | Entering Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler's HandleRemoteAuthenticateAsync.
backend | trce: Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler[24]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | MessageReceived: ''.
backend | warn: Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler[14]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | .AspNetCore.Correlation. state property not found.
backend | 2019-01-05 20:49:03.7033|Warn|.AspNetCore.Correlation. state property not found.
backend | info: Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler[4]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | Error from RemoteAuthentication: Correlation failed..
backend | 2019-01-05 20:49:03.7043|Info|Error from RemoteAuthentication: Correlation failed..
backend | fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | An unhandled exception has occurred while executing the request.
backend | System.Exception: An error was encountered while handling the remote login. ---> System.Exception: Correlation failed.
backend | --- End of inner exception stack trace ---
backend | at Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler`1.HandleRequestAsync()
backend | at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
backend | at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
backend | at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
backend | at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
backend | at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
backend | at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
backend | 2019-01-05 20:49:03.7183|Error|An unhandled exception has occurred while executing the request.
backend | dbug: Microsoft.AspNetCore.HttpsPolicy.HstsMiddleware[1]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | The request is insecure. Skipping HSTS header.
backend | 2019-01-05 20:49:03.7435|Debug|The request is insecure. Skipping HSTS header.
backend | 2019-01-05 20:49:03.7445|Debug|Unknown proxy: [::ffff:47.75.75.47]:54338
backend | dbug: Microsoft.AspNetCore.HttpOverrides.ForwardedHeadersMiddleware[1]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | Unknown proxy: [::ffff:47.75.75.47:54338
backend | trce: Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector[31]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | Performing protect operation to key {0b113743-3113-4f53-bd2f-b01fd81156be} with purposes ('/app', 'SessionMiddleware').
backend | dbug: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[1]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | POST requests are not supported
backend | 2019-01-05 20:49:03.7445|Debug|POST requests are not supported
backend | dbug: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[9]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | AuthenticationScheme: union.authc.cookies was not authenticated.
backend | 2019-01-05 20:49:03.7456|Debug|AuthenticationScheme: union.authc.cookies was not authenticated.
backend | 2019-01-05 20:49:03.7456|Debug|AuthenticationScheme: union.authc.cookies was not authenticated.
backend | dbug: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[9]
backend | => ConnectionId:0HLJJ1GJ5KN5B => RequestId:0HLJJ1GJ5KN5B:00000001 RequestPath:/oidc/login-callback
backend | AuthenticationScheme: union.authc.cookies was not authenticated.