Go-ethereum: Accept error: accept tcp 127.0.0.1:8545: too many open files; retrying in 1s

[ghost]

I want to re-open https://github.com/ethereum/go-ethereum/issues/1460, we're running Geth 1.7.3 with the following options:

geth --rpcport=9545 --datadir=/var/geth --rpcaddr=127.0.0.1 --rpc --rpcapi=admin,debug,eth,net,web3

and keep hitting the following error:

http: Accept error: accept tcp 127.0.0.1:9545: accept4: too many open files; retrying in 1s

our unit file looks like this:

[Unit]
Description=geth
Wants=network.target
After=network.target

[Service]
ExecStart=/opt/geth/1.7.3-4bb3c89d/bin/geth --rpcport=9545 --datadir=/var/geth --rpcaddr=127.0.0.1 --rpc --rpcapi=admin,debug,eth,net,web3
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=TERM
Restart=on-failure
RestartSec=15s
User=geth
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Note LimitNOFILE=65536.

However, when I run lsof on the process I see nowhere near 65k open files:

lsof -p 31006 | wc -l
2059

Here is the output from /proc/31006/limits:

cat /proc/31006/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             29849                29849                processes
Max open files            2048                 65536                files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       29849                29849                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

How can I increase the soft limit/is it possible?

[karalabe]

Geth v1.7.3 hard limited the number of open file descriptors to 2K, even if you specified more on startup. This was fixed in v1.8.0 to not decrease it https://github.com/ethereum/go-ethereum/pull/16009.

That being said, if you are hitting this error, it usually means you are wasting TCP connections on single shot http requests. You might want to try to check your client and ensure it reuses the connections instead of making a new one for each request.

[chfast]

I confirm the fix is working.