Zeronet: Err: IOError: [Errno 24] Too many open files

[yzou]

Server error

Err: IOError: [Errno 24] Too many open files: u'data/1Name2NXVi1RDPDgf5617UoW7xA6YrhM9F/data/names.json' in UiServer.py line 81 > UiRequest.py line 82 > MultiuserPlugin.py line 40 > UiRequest.py line 194 > SiteManagerPlugin.py line 59 > SiteManagerPlugin.py line 39 > SiteStorage.py line 230 > SiteStorage.py line 151

Please report it if you think this an error.

Details:

{
    "GATEWAY_INTERFACE": "CGI/1.1", 
    "HTTP_ACCEPT": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", 
    "HTTP_ACCEPT_ENCODING": "gzip, deflate, sdch, br", 
    "HTTP_ACCEPT_LANGUAGE": "en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4", 
    "HTTP_CONNECTION": "upgrade", 
    "HTTP_HOST": "bit.no.com", 
    "HTTP_REFERER": "https://www.google.com.hk/", 
    "HTTP_UPGRADE_INSECURE_REQUESTS": "1", 
    "HTTP_USER_AGENT": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36", 
    "HTTP_X_FORWARDED_FOR": "47.89.43.124", 
    "HTTP_X_REAL_IP": "47.89.43.124", 
    "PATH_INFO": "/shadowsocksr.bit/", 
    "QUERY_STRING": "", 
    "REMOTE_ADDR": "127.0.0.1", 
    "REMOTE_PORT": "38191", 
    "REQUEST_METHOD": "GET", 
    "SCRIPT_NAME": "", 
    "SERVER_NAME": "localhost", 
    "SERVER_PORT": "43111", 
    "SERVER_PROTOCOL": "HTTP/1.1", 
    "SERVER_SOFTWARE": "gevent/1.1 Python/2.7", 
    "arguments": {
        "action": "main", 
        "batch": false, 
        "bit_resolver": "1Name2NXVi1RDPDgf5617UoW7xA6YrhM9F", 
        "coffeescript_compiler": null, 
        "config_file": "zeronet.conf", 
        "connected_limit": 10, 
        "data_dir": "data", 
        "debug": false, 
        "debug_gevent": false, 
        "debug_socket": false, 
        "disable_db": false, 
        "disable_encryption": false, 
        "disable_sslcompression": true, 
        "disable_udp": false, 
        "fileserver_ip": "*", 
        "fileserver_port": 15441, 
        "homepage": "1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D", 
        "ip_external": null, 
        "keep_ssl_cert": false, 
        "log_dir": "log", 
        "max_files_opened": 102400, 
        "msgpack_purepython": true, 
        "multiuser_local": false, 
        "open_browser": null, 
        "proxy": null, 
        "size_limit": 10, 
        "stream_downloads": false, 
        "tor": "enable", 
        "tor_controller": "127.0.0.1:9051", 
        "tor_proxy": "127.0.0.1:9050", 
        "trackers": [
            "zero://boot3rdez4rzn36x.onion:15441", 
            "zero://boot.zeronet.io#f36ca555bee6ba216b14d10f38c16f7769ff064e0e37d887603548cc2e64191d:15441", 
            "udp://tracker.coppersurfer.tk:6969", 
            "udp://tracker.leechers-paradise.org:6969", 
            "udp://9.rarbg.com:2710", 
            "http://tracker.aletorrenty.pl:2710/announce", 
            "http://explodie.org:6969/announce", 
            "http://tracker1.wasabii.com.tw:6969/announce"
        ], 
        "trackers_file": false, 
        "ui_ip": "127.0.0.1", 
        "ui_port": 43111, 
        "ui_restrict": false, 
        "use_openssl": true, 
        "use_tempfiles": false, 
        "verbose": false
    }, 
    "plugins": [
        "Newsfeed", 
        "Zeroname", 
        "AnnounceZero", 
        "Stats", 
        "Multiuser", 
        "Trayicon", 
        "MergerSite", 
        "CryptMessage", 
        "Sidebar"
    ], 
    "version_gevent": "1.1.0", 
    "version_python": "2.7.12 (default, Jul  1 2016, 15:12:24) \n[GCC 5.4.0 20160609]", 
    "version_zeronet": "0.4.1 r1536", 
    "wsgi.url_scheme": "http"
}

[HelloZeroNet]

The proxy owner has to increase the opened file limit:
http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/

[yzou]

Thanks.

[WalnutATiie]

I have encounted the problem but as far as I'm concerned, once the http connection is closed, FD should be released other than stay in the process. I haven't review your code yet but I think may be there is something wrong with your http connection processing procedure. Hope for reply.

[HelloZeroNet]

python and the gc should take care of closing not used files

i think this problem is caused by too many socket opened and not fd "leaking"

for my client:

$ ls -l /proc/29511/fd | grep /home | wc -l
3

[WalnutATiie]

for my client, fd open is 3 as well. but what i use is keep-alive connection and still get problem. That's the console reports:
Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/gevent-1.0.2-py2.7-linux-i686.egg/gevent/greenlet.py", line 327, in run File "plugins/AnnounceZero/AnnounceZeroPlugin.py", line 43, in announceTracker File "/home/test/ZeroNet-master/src/Site/Site.py", line 725, in announceTracker File "/home/test/ZeroNet-master/src/lib/subtl/subtl.py", line 42, in __init__ File "/usr/local/lib/python2.7/dist-packages/gevent-1.0.2-py2.7-linux-i686.egg/gevent/socket.py", line 229, in __init__ self._sock = _realsocket(family, type, proto) error: [Errno 24] Too many open files

[HelloZeroNet]

@WalnutATiie Can you please check these commands?

$ ls -l /proc/13302/fd | wc -l
211
$ ulimit -a
Maximum size of core files created                           (kB, -c) 0
Maximum size of a process’s data segment                     (kB, -d) unlimited
Maximum size of files created by the shell                   (kB, -f) unlimited
Maximum size that may be locked into memory                  (kB, -l) 64
Maximum resident set size                                    (kB, -m) unlimited
Maximum number of open file descriptors                          (-n) 1024
Maximum stack size                                           (kB, -s) 8192
Maximum amount of cpu time in seconds                   (seconds, -t) unlimited
Maximum number of processes available to a single user           (-u) 5897
Maximum amount of virtual memory available to the shell      (kB, -v) unlimited

[HelloZeroNet]

At startup ZeroNet tries to modify the it's limit to 2048, I just found a bug in that code, so please update and try again: https://github.com/HelloZeroNet/ZeroNet/commit/6f2445c417e59c3bf948ff5c3fd0a722c710f9d3

[WalnutATiie]

I think to deal with the problem is not just changing system max FD settings. All opened FD should be closed after use.

What I want to do is use a local zeronet client as a web proxy to crawl webpages in zeronet. I use python gevent to request webpages asynchronously but once the number of my requests reach about 1000, the problem occured. Netstat is all normal, juse take about 20 port as I set, so I don't think it's all of tcp collections.

Or you can tell me what should I do if I want to use the client as a web proxy.:) thank you very much.

[HelloZeroNet]

My client running since 3 months and it has 223 opened files, so I don't think if there is any any FD closing problem.
ZeroNet very rarely close sockets, (check every 10 minute if the site has more than 10 connections) since it required to receive and distribute new updates. To reduce you can try to disable ssl via --disable_encryption, disable tor --tor disable and change --connected_limit, but I think it's safe to increase the limit, for example Debian 8 has 65536 as default.

https://console.cloud.google.com/cloudshell

$ ulimit -a
...
open files                      (-n) 65536

[slrslr]

@HelloZeroNet

Hello, i just got this error when i downloaded new 64bit Linux ZeroBundle today and copied into its data folder my users.json, sites.json and started Zeronet.sh

My sites.json contains several hundred of sites and apparently Zeronet is made the way it is reaching my Linux OS limits.

$ ulimit -a

core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 15659
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 15659
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

$ python test_max.py

Max open files without changing settings: 1021
Current RLIMIT_NOFILE limit: 1024 4096 Changing to 3000...
Max open files after changing settings: 2997

---

ZeroHello error:

Server error
Err: IOError: [Errno 24] Too many open files: '/etc/mime.types' in UiServer.py line 104 > UiRequest.py line 144 > UiConfigPlugin.py line 21 > ContentFilterPlugin.py line 184 > UiRequest.py line 296 > UiRequest.py line 166 > mimetypes.py line 293 > mimetypes.py line 358 > mimetypes.py line 202
Please report it if you think this an error.
Details:

{
"GATEWAY_INTERFACE": "CGI/1.1",
"HTTP_ACCEPT": "text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8",
"HTTP_ACCEPT_ENCODING": "gzip, deflate",
"HTTP_ACCEPT_LANGUAGE": "en-US,en;q=0.5",
"HTTP_CONNECTION": "keep-alive",
"HTTP_HOST": "127.0.0.1:43110",
"HTTP_UPGRADE_INSECURE_REQUESTS": "1",
"HTTP_USER_AGENT": "Mozilla/5.0 (X11; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0",
"PATH_INFO": "/1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D",
"QUERY_STRING": "",
"REMOTE_ADDR": "127.0.0.1",
"REMOTE_PORT": "46856",
"REQUEST_METHOD": "GET",
"SCRIPT_NAME": "",
"SERVER_NAME": "localhost",
"SERVER_PORT": "43110",
"SERVER_PROTOCOL": "HTTP/1.1",
"SERVER_SOFTWARE": "gevent/1.0 Python/2.7",
"arguments": {
"action": "main",
"autodownload_bigfile_size_limit": 1,
"batch": false,
"bigfile_size_limit": false,
"bind": null,
"bit_resolver": "1Name2NXVi1RDPDgf5617UoW7xA6YrhM9F",
"broadcast_port": 1544,
"coffeescript_compiler": null,
"config_file": "./zeronet.conf",
"connected_limit": 8,
"data_dir": "./data",
"db_mode": "speed",
"debug": false,
"debug_socket": false,
"disable_db": false,
"disable_encryption": false,
"disable_sslcompression": true,
"disable_udp": false,
"download_optional": "manual",
"end": true,
"file_size_limit": 10,
"fileserver_ip": "*",
"fileserver_port": 14044,
"fileserver_port_range": "10000-40000",
"fix_float_decimals": false,
"force_encryption": false,
"global_connected_limit": 512,
"homepage": "1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D",
"ip_external": null,
"ip_local": [
"127.0.0.1"
],
"keep_ssl_cert": false,
"language": "en",
"log_dir": "./log",
"log_level": "DEBUG",
"max_files_opened": 2048,
"msgpack_purepython": false,
"open_browser": "default_browser",
"optional_limit": "10%",
"optional_limit_exclude_minsize": 20,
"proxy": null,
"silent": false,
"size_limit": 10,
"stack_size": null,
"stream_downloads": false,
"tor": "enable",
"tor_controller": "127.0.0.1:9051",
"tor_hs_limit": 10,
"tor_hs_port": 15441,
"tor_proxy": "127.0.0.1:9050",
"tor_use_bridges": false,
"trackers": [
"zero://boot3rdez4rzn36x.onion:15441",
"zero://zero.booth.moe#f36ca555bee6ba216b14d10f38c16f7769ff064e0e37d887603548cc2e64191d:443",
"udp://tracker.coppersurfer.tk:6969",
"udp://5.79.83.193:6969",
"udp://104.238.198.186:8000",
"http://tracker.swateam.org.uk:2710/announce",
"http://open.acgnxtracker.com:80/announce",
"http://retracker.mgts.by:80/announce"
],
"trackers_file": false,
"trackers_proxy": "disable",
"ui_host": null,
"ui_ip": "127.0.0.1",
"ui_port": 43110,
"ui_restrict": false,
"ui_trans_proxy": false,
"updatesite": "1UPDatEDxnvHDo7TXvq6AEBARfNkyfxsp",
"use_openssl": true,
"use_tempfiles": false,
"verbose": false,
"workers": 5,
"working_shared_trackers_limit": 5
},
"plugins": [
"AnnounceLocal",
"AnnounceShare",
"AnnounceZero",
"Bigfile",
"Chart",
"ContentFilter",
"Cors",
"CryptMessage",
"FilePack",
"MergerSite",
"Newsfeed",
"OptionalManager",
"PeerDb",
"Sidebar",
"Stats",
"TranslateSite",
"Trayicon",
"UiConfig",
"Zeroname"
],
"version_gevent": "1.0.2",
"version_python": "2.7.11 |Continuum Analytics, Inc.| (default, Dec 6 2015, 18:08:32) \n[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]",
"version_zeronet": "0.6.4 r3688",
"wsgi.url_scheme": "http"
}

Maybe it is not such a big problem, because when i retried to load ZeroHello in a couple of minute, error was no longer there, instead very high CPU activity (Firefox, thanks to ZeroHello page utilization of too many sites), the utilization was like 100% of the 3000 passmark rated CPU going for several minutes, then i quit Zeronet because i not wanted to redownload every site, instead i decided to restore my zeronet data dir backup.