Serving: Why serverless containers instead of pods?

[josephburnett]

Expected Behavior

As a Developer, I may have decomposed my application into several containers. E.g. one for serving HTTP requests and one for processing background tasks.

Actual Behavior

But Knative only accepts a single Container spec. Why?

Additional Info

Container in Revision spec

[josephburnett]

This wasn't originally my question. @kelseyhightower pointed this out during one of our hackathons and it's been rattling around in my head ever since.

Perhaps there's a really good answer (but I don't have it).

[josephburnett]

@mattmoor @vaikas-google @evankanderson

[mattmoor]

cc @cooperneil @sixolet

[sixolet]

I think we were aiming for simplicity. Knative actually may load more into the pod with your application-level container, but those things are usually plumbing (envoy, etc), not application code.

"Processing background tasks" is interesting; my naïve understanding of how you'd arrange that in (the eventual form of) Knative is as a separate Service loosely coupled with the serving Service using the Eventing system. They'd then each scale as appropriate for their own needs. Putting the two in the same pod would seem odd to me.

[josephburnett]

Putting the two in the same pod would seem odd to me.

Suppose that I have an application that reports usage on a best-effort basis. Every time I get an HTTP request, my application generates a number of events to upload. I also want the team that owns the reporting service to own the reporting client, so I give them a separate container in my application and just push the events to localhost:whatever before returning the HTTP call. Their highly-available client (because it's running locally) enqueues the events and uploads them to the service in the background.

[evankanderson]

Some short answers:

1. Simplicity of programming model. Pods add a number of extra layers of
   complexity:

   • Which container receives requests?
   • What happens to other containers when the "primary" container is not
     running a request?
   • What metrics and other components are collected from other containers?
   • What constraints
     exist on the other containers?

2. The primary target for Knative is to simplify application development.
   While it's possible for us to support the full breadth of the underlying
   Kubernetes abstractions, we should make sure that any complexity we add has
   clear practical justification. As another example, we don't support
   user-specified init containers, multiple ports, or volume mounts in the
   user-specified interface, even if we end up scheduling 3 containers in each
   pod (istio, user, and monitoring, IIRC).

[mattmoor]

cc @krancour see @evankanderson's answer above

[evankanderson]

To follow up on https://github.com/knative/serving/issues/1951#issuecomment-416532181, it would be good to call out the components of Knative today which would be useful to make available independently (i.e. is queue a useful sidecar, should it be rolled into Istio/Knative Routing Primitives?).

We're already looking at making both the autoscaler and log collection/monitoring able to work well stand-alone -- we're building those tools because we need them, not because they are core to knative/serving's "stateless triggered container execution". It's worth calling out other components that make sense to separate; the idea is that autoscaler and observability should work fine with native k8s Deployments, for example.

[cppforlife]

shared google doc "Knative Serving: Container vs Pod Spec" with knative-dev@ google group.

[evankanderson]

Doc here

[mattmoor]

Discussed with @evankanderson and going to close until a concrete scenario comes up.