Sanic: Scaling sanic app using Containers and Kubernetes

[jibinmathew69]

What would be a good deployment strategy from sanic application? should it be run behind a web server like Nginx?

And while running it in a kubernetes cluster should it be run behind a Nginx server or a simple load balancer using ingress would be the best solution?

[gbnk0]

Hi,

I use it behind haproxy and with gunicorn on the containers.
I think the best solution for me is:
Loadbalancer -> nginx -> gunicorn -> sanic
Or maybe:
Loadbalancer -> nginx -> sanic

[Garito]

What's the point of gunicorn in this case?
Giving the fact that we are talking about containers, the scenario it make more sense to me will be nginx for loadbalancing and sanic
Would be nice a rationale of why I'm wrong if so
Thanks

[gbnk0]

I'm using gunicorn for managing sanic crash, it launch sanic as a worker and if worker crash it will relaunch it.
I don't know if it's pertinent to use gunicorn in your setup.

[Garito]

it doesn't seems pertinent to me for this particular reason
you can use the restart command on compose (you don't even need kubernetes or swarm)
if you are using a kubernetes or swarm you could scale the service instead of workers

any other reason? (i'm trying to check my own rationale)

[ahopkins]

Here is my $.02.

I have Sanic running inside K8S and behind Nginx. Why?

Well, I am using Nginx to reverse proxy and split my requests based upon their path to one of several different microservices.

Something sort of like this:

server {
    ...

    #### HTTP Services ####

    # Authentication service
    location /auth/ {
        proxy_pass http://auth-service/;
    }

    # Customer information service
    location /customer/ {
        proxy_pass http://customer-service/;
    }

    # Computed data service
    location /data/ {
        proxy_pass http://data-service/;
    }

    ...
}

This is built into a docker container, and that is exposed by K8S via an ingress controller something like this.

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myservice
spec:
  rules:
  - host: my.domain
    http:
      paths:
      - path: /
        backend:
          serviceName: my-balancer
          servicePort: http
  tls:
    - secretName: my-domain-tls
      hosts:
        - my.domain

And, as @Garito pointed out, this allows me to scale my services and manage crashes from within Kubernetes.

One thing to note, and this is a decision you need to make, where do you want to do the TLS translation? It could be handled with Nginx running in front of Kubernetes, by Kubernetes in an Ingress controller, by Nginx inside K8S, etc. Depending on your use and deployment, this might change for you. Just a thought to keep in mind.

[jibinmathew69]

Is it possible to route based on the path using nginx ingress?

[arnulfojr]

I have used sanic in different architectures, but I think from my experience, the best way of deploying sanic in a containerize environment is to run sanic directly from the python executable.

I have deployed sanic ([blabla] == 1 container):

• [nginx] -> [gunicorn -> sanic]
• [nginx] -> [sanic]
• [traefik] -> [gunicorn -> sanic]
• [traefik] -> [sanic]
• [sanic]

But in my findings I found better the [lb] -> [sanic] set-up:

• Gunicorn spawns multiple processes, cool?? kubernetes scales your applications horizontally and brings them back to life if they die :D so gunicorn is not required, unless your application has memory leaks and processes need to restart from time to time...?? 1 container = 1 process is the idea here.
• Spawning threads and process workers require more memory inside a cgroup... resources are not easily predictable and consume more than a simple process.
• Running sanic directly from a python module: maybe was in our case, our monitoring tools had issues monitoring the [gunicorn -> sanic] set-up, and in exchange we got fewer metrics. We ran sanic as a python module, our monitoring tool brought metrics from the python interpreter and brought better insights. (Maybe was a bug, not clear to me tho...)

Personal recommendations:
I love using sanic in a micro-service architecture, but have also used it to power some simple web backend with high I/O load. And I would totally recommend avoiding using gunicorn if you use a container orchestrator, if you are running it as a plain container, then gunicorn may give you the benefits you're missing from an orchestrator. But it comes down on the infrastructure you plan to have or you have...

Keep in mind that gunicorn is a process manager... Normally containers should only have 1 process :) nothing to manage IMHO (just make sure your container exits with the proper exit code)

Nginx as an ingress is faster, definitely. As sanic has yet no support for http2, I'd say then that nginx is better as the payload is bigger. Overall (jokes a side), LB is good only if you'll scale horizontally your containers, else you're just consuming resources for a container that is simply proxying stuff to 1 container which, in case of sanic, I'd say is not required as sanic can handle the traffic really good.

[skewty]

How would you do a broadcast to all clients (imagine a chat app) with multiple sanic processes?
I know Django Channels uses (by default) a Redis backend to handle this. Does sanic have some sort of backend to handle this already?

[ahopkins]

@skewty

I am doing this right now with aioredis, but have also had it working inside Sanic with aredis.

Both are excellent redis client packages that work with asyncio and have pubsub (aka publish/subscribe) functionality.

So, in Sanic, what you would do is setup a websocket route. Inside that route, you need to:

• subscribe to a pubsub channel using redis
• listen to messages on that channel, and when received

If you wanted two way communication on that same connection (chat being a good example), you need to really do three things:

• subscribe to a pubsub channel using redis
• listen to messages on that channel, and when received
• listen to messages coming from the client and process them (aka publish to the channel)

One or both of those need to be pushed to a task with loop.create_task() since they both are going to create a loop.

With that being said ... it is sort of off topic here and if you have further discussion on this, perhaps it would be better to move to its own issue.

[c-goosen]

My current setup (AWS btw):

Load Balancer <-> Virtual Machine <-> Nginx <-> gunicorn <-> sanic (started via gunicorn and supervisord).

Obvisouly with docker supervisord does not work as well as the rule of thumb is one (launch) process per container.

Currently running 1 sanic api with 4 workers, then all of them connect to a networked redis for state/auth state and an additional networked redis cache. Further scale on heavy I/O is done through a worker setup similar to celery that waits for a queue, populated by sanic.

[ahopkins]

I think some of this information in this thread would make for a nice blog post if anyone wants to take a stab at it 😉

Till then I am closing since the question has been answered.

[c-goosen]

@ahopkins If I find some time I can write some posts on it. Where to post it?

Some nice caveats and deployment posts.

[c-goosen]

I would love to contribute more back. I took a wild leap in building infrastructure from scratch with sanic and I haven't regretted the choice a single day. Fast paced development in community, seamless upgrades, plugins. This is a testament to the power of opensource and good ideas.

[sjsadowski]

@c-goosen I'm sure we can add it as content on sanicframework.org, possibly also medium since that seems to be all the rage these days.

[ahopkins]

@c-goosen 💪

@sjsadowski Agreed.