Bert-as-service: BertClient con't connect

[k-oul]

Prerequisites

Please fill in by replacing [ ] with [x].

• [x] Are you running the latest bert-as-service?
• [x] Did you follow the installation and the usage instructions in README.md?
• [x] Did you check the FAQ list in README.md?
• [x] Did you perform a cursory search on existing issues?

System information

Some of this information can be collected via this script.

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Ubuntu 18.04
• TensorFlow installed from (source or binary): source
• TensorFlow version: 1.14.0
• Python version:3.7.0
• bert-as-service version: 1.10.0
• GPU model and memory:
• CPU model and memory:

---

Description

Please replace YOUR_SERVER_ARGS and YOUR_CLIENT_ARGS accordingly. You can also write your own description for reproducing the issue.

I'm using this command to start the server:

from bert_serving.server.helper import get_args_parser
from bert_serving.server import BertServer
args = get_args_parser().parse_args(['-model_dir', './bert/chinese_L-12_H-768_A-12/',
                                     '-num_worker', '4',
                                     '-port', '5555',
                                     '-port_out', '5556',
                                     '-pooling_strategy','NONE',
                                     '-mask_cls_sep',
                                     '-cpu'
                                     ])
server = BertServer(args)
server.start()

and calling the server via:

bc = BertClient()
bc.encode()

Then this issue shows up:
cpu: 8 core
memory: 16G
when num_worker > 1, BertClient con't connect
...

[asankasan]

I'm also facing the same issue. The client is not able to connect to the server. It looks like the server is not ready.

[saggar]

Facing BertClient TimeOut error with Python 3.7.4

from bert_serving.client import BertClient                                                                                                                                                   
bc = BertClient(ip='127.0.0.1', port=5555, port_out=5556, show_server_config=True, ignore_all_checks=False, timeout=20000)                                                                          

bc.encode(['First do it', 'then do it right', 'then do it better'])                                                                                                                        

Error

TimeoutError: no response from the server (with "timeout"=20000 ms), please check the following:is the server still online? is the network broken? are "port" and "port_out" correct? are you encoding a huge amount of data whereas the timeout is too small for that?

Checked that the BertServer is up and running

(base) ➜  docker docker run -p 5555:5555 -p 5556:5556  e98983cd289f
I:VENTILATOR:[__i:__i: 67]:freeze, optimize and export graph, could take a while...
usage: /usr/local/bin/bert-serving-start -model_dir /app/multi_cased_L-12_H-768_A-12 -num_worker=1 -cpu
                 ARG   VALUE
__________________________________________________
           ckpt_name = bert_model.ckpt
         config_name = bert_config.json
                cors = *
                 cpu = True
          device_map = []
       do_lower_case = True
  fixed_embed_length = False
                fp16 = False
 gpu_memory_fraction = 0.5
       graph_tmp_dir = None
    http_max_connect = 10
           http_port = None
        mask_cls_sep = False
      max_batch_size = 256
         max_seq_len = 25
           model_dir = /app/multi_cased_L-12_H-768_A-12
no_position_embeddings = False
    no_special_token = False
          num_worker = 1
       pooling_layer = [-2]
    pooling_strategy = REDUCE_MEAN
                port = 5555
            port_out = 5556
       prefetch_size = 10
 priority_batch_size = 16
show_tokens_to_client = False
     tuned_model_dir = None
             verbose = False
                 xla = False

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_serving/server/helper.py:186: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_serving/server/helper.py:186: The name tf.logging.ERROR is deprecated. Please use tf.compat.v1.logging.ERROR instead.

I:GRAPHOPT:[gra:opt: 53]:model config: /app/multi_cased_L-12_H-768_A-12/bert_config.json
I:GRAPHOPT:[gra:opt: 56]:checkpoint: /app/multi_cased_L-12_H-768_A-12/bert_model.ckpt
I:GRAPHOPT:[gra:opt: 60]:build graph...
I:GRAPHOPT:[gra:opt:132]:load parameters from checkpoint...
I:GRAPHOPT:[gra:opt:136]:optimize...
I:GRAPHOPT:[gra:opt:144]:freeze...
I:GRAPHOPT:[gra:opt:149]:write graph to a tmp file: /tmp/tmprdty0_mo
I:VENTILATOR:[__i:__i: 75]:optimized graph is stored at: /tmp/tmprdty0_mo
I:VENTILATOR:[__i:_ru:129]:bind all sockets
I:VENTILATOR:[__i:_ru:133]:open 8 ventilator-worker sockets
I:VENTILATOR:[__i:_ru:136]:start the sink
I:SINK:[__i:_ru:306]:ready
I:VENTILATOR:[__i:_ge:222]:get devices
I:VENTILATOR:[__i:_ge:255]:device map: 
                worker  0 -> cpu
I:WORKER-0:[__i:_ru:531]:use device cpu, load graph from /tmp/tmprdty0_mo
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_serving/server/helper.py:186: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

Ports 5555 & 5556 are also accessible

(base) ➜  EKSApps nc -vz 127.0.0.1 5556      
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
        outif lo0
        src 127.0.0.1 port 52875
        dst 127.0.0.1 port 5556
        rank info not available
        TCP aux info available

Connection to 127.0.0.1 port 5556 [tcp/freeciv] succeeded!

(base) ➜  EKSApps nc -vz 127.0.0.1 5555
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
        outif lo0
        src 127.0.0.1 port 52877
        dst 127.0.0.1 port 5555
        rank info not available
        TCP aux info available

Connection to 127.0.0.1 port 5555 [tcp/personal-agent] succeeded!

Any thoughts?

[Bramy2014]

I've got the same issue. Does anyone know how to fix it?

[winlp4ever]

Same problem here. Would be great if someone takes a look

[Delvify]

+1

[jageshmaharjan]

uninstall the latest version and install this version 1.9.8 and try again.

Clear the tf.graph() or kill the background service.
If not restart the machine.

[zhsuiy]

I had the same issue. The reason might be the lack of memory due to the running servers on the machine. Here is what I did. I found all the pid(s) by the port and port_out of all the previous servings in command line, then mannually killed them. After this, when I run bert-serving-start, the log ends with 'all set, ready to serve request! '