Bert-as-service: the bert-serving service does not work normaly

[xiaobengou01]

the server reponse anything and hungs when running code bc.encode(['first do it']),i start the service with"bert-serving-start -model_dir /root/nlp/bert/models/chinese_L-12_H-768_A-12 -num_worker=1 -cpu" .I hava alraedy update to the lastest version 1.6.1.my python version is 3.6.3。system is ubuntu-18.04. CPU only.what can I do to solve this problem.

[hanxiao]

1. please fill the issue form, save time for both of us.
2. please provide detailed server logs.

[xiaobengou01]

@hanxiao

[hanxiao]

got it, may relate to https://github.com/hanxiao/bert-as-service/issues/90#issuecomment-446074747

could you do bc.server_status and bc.status and paste the result here? remember to remove sensitive info for the sake of your privacy.

[xiaobengou01]

[hanxiao]

ok got it. i think it's same as #90 which happens only on tf 1.12 CPU version. will do a fix.

[hanxiao]

fyi, this issue is fixed in #155 and is available since 1.6.2, please do

pip install -U bert-serving-server bert-serving-client

for the update.

[xiaobengou01]

after upgrade the version.it works fine now .thank you very much

[kvdesai]

I am still having this issue, even after setting prefetch_size to 0 and using latest version of Bert.

Bert 1.8.6
Python 3.6.8
Ubuntu 16.04, cpu only

Below is the server log

---

$ bert-serving-start -model_dir ./models/cased_L-24_H-1024_A-16 -num_worker=2 -cpu -prefetch_size 0
usage: /home/sphatik/miniconda3/envs/textmining/bin/bert-serving-start -model_dir ./models/cased_L-24_H-1024_A-16 -num_worker=2 -cpu -prefetch_size 0
                 ARG   VALUE
__________________________________________________
           ckpt_name = bert_model.ckpt
         config_name = bert_config.json
                cors = *
                 cpu = True
          device_map = []
  fixed_embed_length = False
                fp16 = False
 gpu_memory_fraction = 0.5
       graph_tmp_dir = None
    http_max_connect = 10
           http_port = None
        mask_cls_sep = False
      max_batch_size = 256
         max_seq_len = 25
           model_dir = ./models/cased_L-24_H-1024_A-16
          num_worker = 2
       pooling_layer = [-2]
    pooling_strategy = REDUCE_MEAN
                port = 5555
            port_out = 5556
       prefetch_size = 0
 priority_batch_size = 16
show_tokens_to_client = False
     tuned_model_dir = None
             verbose = False
                 xla = False

I:VENTILATOR:[__i:__i: 66]:freeze, optimize and export graph, could take a while...
I:GRAPHOPT:[gra:opt: 52]:model config: ./models/cased_L-24_H-1024_A-16/bert_config.json
I:GRAPHOPT:[gra:opt: 55]:checkpoint: ./models/cased_L-24_H-1024_A-16/bert_model.ckpt
I:GRAPHOPT:[gra:opt: 59]:build graph...
I:GRAPHOPT:[gra:opt:128]:load parameters from checkpoint...
I:GRAPHOPT:[gra:opt:132]:optimize...
I:GRAPHOPT:[gra:opt:140]:freeze...
I:GRAPHOPT:[gra:opt:145]:write graph to a tmp file: /tmp/tmpsn_ot94b
I:VENTILATOR:[__i:__i: 74]:optimized graph is stored at: /tmp/tmpsn_ot94b
I:VENTILATOR:[__i:_ru:118]:bind all sockets
I:VENTILATOR:[__i:_ru:122]:open 8 ventilator-worker sockets
I:VENTILATOR:[__i:_ru:125]:start the sink
I:SINK:[__i:_ru:284]:ready
I:VENTILATOR:[__i:_ge:202]:get devices
I:VENTILATOR:[__i:_ge:235]:device map: 
        worker  0 -> cpu
        worker  1 -> cpu
I:WORKER-0:[__i:_ru:492]:use device cpu, load graph from /tmp/tmpsn_ot94b
I:WORKER-1:[__i:_ru:492]:use device cpu, load graph from /tmp/tmpsn_ot94b
I:VENTILATOR:[__i:_ru:176]:new encode request   req id: 3   size: 1 client: b'688a67a7-4a06-4430-9b50-b77d2def7744'
I:SINK:[__i:_ru:331]:job register   size: 1 job id: b'688a67a7-4a06-4430-9b50-b77d2def7744#3'

---

Client side:

>>> from bert_serving.client import BertClient
>>> bc = BertClient()
>>> bc.encode(['hello there friend'])

[gauravkaila]

Hi Han - Thanks for your great work. Really appreciate it :)

Though I could install all the requirements, I am still facing the same as @kvdesai for the following system config:

Ubuntu v16.04 (CPU)
Python v3.5.2
TensorFlow 1.13.1
bert 1.8.6

The server does not go beyond

I:WORKER-0:[__i:_ru:492]:use device cpu, load graph from

Below is my full log:

Any help on this is greatly appreciated. Thanks :)

[nikhilpadal]

Hi Han,
I'm still facing the same issue, even though i'm on latest package of BERT.
Could you check if the problem is really resolved on a CPU only machine ?