Bert-as-service: Resource temporarily unavailable

[zhant09]

Prerequisites

Please fill in by replacing [ ] with [x].

• [x] Are you running the latest bert-as-service?
• [x] Did you follow the installation and the usage instructions in README.md?
• [x] Did you check the FAQ list in README.md?
• [x] Did you perform a cursory search on existing issues?

System information

Some of this information can be collected via this script.

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Debian 4.9.82
• TensorFlow installed from (source or binary): binary
• TensorFlow version: 1.10.0
• Python version: 3.6.8
  

  - bert-as-service version: 1.7.6

Description

Please replace YOUR_SERVER_ARGS and YOUR_CLIENT_ARGS accordingly. You can also write your own description for reproducing the issue.

I'm using this command to start the server:

bert-serving-start -cpu -max_batch_size 16 -model_dir ${MODEL_PATH}/chinese_L-12_H-768_A-12/ -num_worker=4 -max_seq_len=100 -pooling_strategy=NONE -port=18888 -port_out=18889

and calling the server via:

bc = BertClient(port=18888, port_out=18889, timeout=2000)
bc.encode([xxx[:103]]) 

Then this issue shows up:
I run the client side code in a python flask framework with gunicorn. Everything runs fine, but after sometime, the following error occurred:

terminate called after throwing an instance of 'std::system_error'
  what():  Resource temporarily unavailable
[2019-01-22 02:46:41 +0800] [28832] [INFO] Unhandled exception in main loop
Traceback (most recent call last):
  File "/home/recsys/anaconda3/envs/article-classification-v2/lib/python3.6/site-packages/gunicorn/arbiter.py", line 211, in run
    self.manage_workers()
  File "/home/recsys/anaconda3/envs/article-classification-v2/lib/python3.6/site-packages/gunicorn/arbiter.py", line 544, in manage_workers
    self.spawn_workers()
  File "/home/recsys/anaconda3/envs/article-classification-v2/lib/python3.6/site-packages/gunicorn/arbiter.py", line 611, in spawn_workers
    self.spawn_worker()
  File "/home/recsys/anaconda3/envs/article-classification-v2/lib/python3.6/site-packages/gunicorn/arbiter.py", line 566, in spawn_worker
    pid = os.fork()
BlockingIOError: [Errno 11] Resource temporarily unavailable

I think the problem might due to https://stackoverflow.com/a/14374796, is this an issue of bert-as-service?
...

[hanxiao]

hmm, not sure the reason behind, will dig in.

fyi, if your goal is to use it as a HTTP service, e.g. call the service via RESTful API. You may want to refer to this tutorial: https://github.com/hanxiao/bert-as-service/#using-bert-as-service-to-serve-http-requests-in-json

Or you may want to refactor this file.

[zhant09]

Thanks for the advice.

I have read the code of http file. But I think the problem is due to sock.send() as mentioned in the above stackoverflow links, the http restful api seemed to only encapsulate the bert client.

Does my understanding of http api right?

[zhant09]

The problem is due to http service but not bert service. After changing gunicorn worker-class from gevent to sync, the problem has been resolved.