Bert-as-service: Shutting down the server and restarting

[nyck33]

Prerequisites

Please fill in by replacing [ ] with [x].

• [x ] Are you running the latest bert-as-service?
• [ x] Did you follow the installation and the usage instructions in README.md?
• [x ] Did you check the FAQ list in README.md?
• [x ] Did you perform a cursory search on existing issues?

System information

Some of this information can be collected via this script.

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04

TensorFlow installed from (source or binary): binary

TensorFlow version: 1.12.0

Python version: Python 3.6.5 :: Anaconda custom (64-bit)

bert-as-service version: newest downloaded with:

pip install bert-serving-server  # server
pip install bert-serving-client  # client, independent of `bert-serving-server`

GPU model and memory: compute capability 2.1 Nvdia NVS-4200m (I'm using the -cpu option when I run the server)

CPU model and memory: Intel i3 and 8GB DDR3

Description

Please replace YOUR_SERVER_ARGS and YOUR_CLIENT_ARGS accordingly. You can also write your own description for reproducing the issue.

I'm using this command to start the server:

bert-serving-start -pooling_strategy NONE -model_dir /media/nobu/"Ubuntu Backup"/"Comp Sci"/_DeepLearning/NLP/multi_cased_L-12_H-768_A-12/ -cpu -max_batch_size 16 -num_worker=2

and calling the server via: NOT APPLICABLE

bc = BertClient(YOUR_CLIENT_ARGS)
bc.encode()

Then this issue shows up:
I cannot restart the server after Ctrl-C/Ctrl-Z to terminate a running server.
I get the following output in the terminal:

/home/nobu/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
usage: /home/nobu/anaconda3/bin/bert-serving-start -pooling_strategy NONE -model_dir /media/nobu/Ubuntu Backup/Comp Sci/_DeepLearning/NLP/multi_cased_L-12_H-768_A-12/ -cpu -max_batch_size 16 -num_worker=2
                 ARG   VALUE
__________________________________________________
           ckpt_name = bert_model.ckpt
         config_name = bert_config.json
                cors = *
                 cpu = True
          device_map = []
                fp16 = False
 gpu_memory_fraction = 0.5
       graph_tmp_dir = None
    http_max_connect = 10
           http_port = None
        mask_cls_sep = False
      max_batch_size = 16
         max_seq_len = 25
           model_dir = /media/nobu/Ubuntu Backup/Comp Sci/_DeepLearning/NLP/multi_cased_L-12_H-768_A-12/
          num_worker = 2
       pooling_layer = [-2]
    pooling_strategy = NONE
                port = 5555
            port_out = 5556
       prefetch_size = 10
 priority_batch_size = 16
     tuned_model_dir = None
             verbose = False
                 xla = False

I:VENTILATOR:[__i:__i: 63]:freeze, optimize and export graph, could take a while...
I:GRAPHOPT:[gra:opt: 48]:model config: /media/nobu/Ubuntu Backup/Comp Sci/_DeepLearning/NLP/multi_cased_L-12_H-768_A-12/bert_config.json
I:GRAPHOPT:[gra:opt: 50]:checkpoint: /media/nobu/Ubuntu Backup/Comp Sci/_DeepLearning/NLP/multi_cased_L-12_H-768_A-12/bert_model.ckpt
I:GRAPHOPT:[gra:opt: 54]:build graph...
I:GRAPHOPT:[gra:opt:123]:load parameters from checkpoint...
I:GRAPHOPT:[gra:opt:126]:optimize...
I:GRAPHOPT:[gra:opt:134]:freeze...
I:GRAPHOPT:[gra:con:192]:freezing 181 variables...
I:GRAPHOPT:[gra:con:246]:Converted 181 variables to const ops.
I:GRAPHOPT:[gra:opt:139]:write graph to a tmp file: /tmp/tmp80d_hwgf
I:VENTILATOR:[__i:__i: 71]:optimized graph is stored at: /tmp/tmp80d_hwgf
I:VENTILATOR:[__i:_ru:103]:bind all sockets
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/nobu/anaconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/nobu/anaconda3/lib/python3.6/site-packages/bert_serving/server/__init__.py", line 89, in run
    self._run()
  File "/home/nobu/anaconda3/lib/python3.6/site-packages/zmq/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "/home/nobu/anaconda3/lib/python3.6/site-packages/zmq/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "/home/nobu/anaconda3/lib/python3.6/site-packages/zmq/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "/home/nobu/anaconda3/lib/python3.6/site-packages/bert_serving/server/zmq_decor.py", line 27, in wrapper
    return func(*args, **kwargs)
  File "/home/nobu/anaconda3/lib/python3.6/site-packages/bert_serving/server/__init__.py", line 104, in _run
    frontend.bind('tcp://*:%d' % self.port)
  File "zmq/backend/cython/socket.pyx", line 547, in zmq.backend.cython.socket.Socket.bind
  File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Address already in use

I tried to sudo killsudo lsof -t -i:5555and sudo killsudo lsof -t -i:5556` and then restart but I get the same error. The only solution I have right now is to restart Ubuntu. Is there some way to shutdown or restart the server without restarting the OS?

...

[hanxiao]

Ctrl-C is the safe way to terminate the server. No need to reboot the machine.

kill is not the a safe way to terminate the server.

[hanxiao]

but then on the second thought, you are right. I need to provide a more gentle CLI/API for shutting down server besides keyboard interrupt. I'm marking it as a feature for now.

[breakmo]

I've got the same issue. I tried 'sudo kill' many times, still cannot restart the bert_serving_start.

[ffabulous]

I'm not sure this would help, but I use the commands below to start/stop the server.
It kills all running workers. Wrap it up in to a script like bert-server-ctl.sh and enjoy 😺

# Start
nohup bert-serving-start ${SERVER_ARGS} > tmp/bert-server.log 2>&1 &
echo $! > tmp/bert-server.pid
# Stop
BERT_PID=$(cat tmp/bert-server.pid)
kill -TERM -$(ps -p ${BERT_PID} -o pgid --no-headers)

[yyz561]

Basically I had the same problem, you don't need to restart, maybe they already fixed something now. simply kill -9 pid would solve this problem. However, after two days work of installing & uninstalling, I finally succeeded in running the sentence encoding for once, so hype.

[edit] Hi, HanXiao, thanks again for the brilliant work. Another interesting fact is that when I run bert-serving-start ... in the ~ directory, it works fine. However, when I run the same command in another folder, say ~/myfolder, the zmq.error.ZMQError file not found error raised. If you had explained this issue somewhere I didn't notice, sorry for it.

[hanxiao]

fyi, this issue has been fixed in #286 and the new feature is available since 1.8.6. Please do

pip install -U bert-serving-server bert-serving-client

for the update.

To shutdown a server, please do bert-serving-terminate --help and check README for details.

[bb9696aa]

This issue still occurred on Windows Server, when the bert-service was running, I just close the 'Git Bash' command window directly without the 'Ctrl+C' operation.
Finally I solved it through ending the background process named 'Python.exe' in the Windows Task Manager.

[jy2k]

I need to second what bb969aa did. I was running on mac and despite looking for all the python processes through the terminal CLI I had to open the activity monitor to finally succeed.