Fiber: 🤔 What is the purpose of prefork?

[peterbourgon]

Question description

What is the intent behind the prefork option?

[welcome[bot]]

Thanks for opening your first issue here! 🎉 Be sure to follow the issue template!

[Fenny]

Prefork enables use of the SO_REUSEPORT socket option, which is available in newer versions of many operating systems, including DragonFly BSD and Linux (kernel version 3.9 and later). This socket option allows multiple sockets to listen on the same IP address and port combination. The kernel then load balances incoming connections across the sockets.

SO_REUSEPORT scales perfectly when a lot of concurrent client connections (at least thousands) are established over the real network (preferrably 10Gbit with per-CPU hardware packet queues). When small number of concurrent client connections are established over localhost, then SO_REUSEPORT usually doesn't give any performance gain.

Benchmarks where preforking is enabled.
https://www.techempower.com/benchmarks/#section=test&runid=350f0783-cc9b-4259-9831-28987799782a&hw=ph&test=json&l=zijocf-1r

NGINX on socket sharding
https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/

[peterbourgon]

Thanks for that explanation. Do you have evidence to suggest a single Go process cannot support "[thousands] of concurrent client connections" without SO_REUSEPORT?

[Fenny]

A single go process can easily support thousands of concurrent connections. Preforking makes use of single go processes but will load balance connections on OS level.

It's up to you if preforking has an advantage for your web app, we only provide the experimental option to enable it.

Feel free to re-open this issue if you have further questions!