Bert-as-service: Error with bert-serving-start "p.is_ready.wait"

Created on 28 Mar 2019 · 8Comments · Source: hanxiao/bert-as-service

I am unable to get the server up and running in Docker. I run into this error on start (full trace at the bottom of the issue).

  File "/usr/local/lib/python3.5/dist-packages/bert_serving/server/__init__.py", line 160, in _run
    p.is_ready.wait()
AttributeError: 'function' object has no attribute 'wait'

I noticed that the code associated with that error was added just 15 hours ago, so I thought that was likely something to do with the error. https://github.com/hanxiao/bert-as-service/commit/842f273755e8531681c6b041f07380a221d4568b#diff-a946a72133b5fd9202e68ef043143212

Prerequisites

[x] Are you running the latest bert-as-service?
[X] Did you follow the installation and the usage instructions in README.md?
[X] Did you check the FAQ list in README.md?
[X] Did you perform a cursory search on existing issues?

System information

I'm following the docker install instructions, so I don't think this information it relevant. But I am running on CPU, so I'm using tensorflow/tensorflow:1.12.0-py3 (I also tried tensorflow/tensorflow:latest-py3). That might have something to do with it?

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS Sierra: 10.12.6
TensorFlow installed from (source or binary): (from docker)
TensorFlow version: (from docker)
Python version:
bert-as-service version: 1.8.7
GPU model and memory: -
CPU model and memory: -

Description

This is my Dockerfile:

FROM tensorflow/tensorflow:latest-py3
RUN pip install -U bert-serving-server[http]
COPY ./ /app
COPY ./docker/entrypoint.sh /app
RUN chmod +x /app/entrypoint.sh
WORKDIR /app
ENTRYPOINT ["/app/entrypoint.sh"]
CMD []

This is my entrypoint.sh file:

#!/bin/sh
bert-serving-start -cpu -num_worker=$1 -model_dir /model -http_port 3011

And my docker run command:

docker run -it -p 5555:5555 -p 5556:5556  -p 3011:3011 -v $PATH_MODEL:/model -t bert-as-service $NUM_WORKER

The process gets running, then shows this error and won't respond to http requests:

  File "/usr/local/lib/python3.5/dist-packages/bert_serving/server/__init__.py", line 160, in _run
    p.is_ready.wait()
AttributeError: 'function' object has no attribute 'wait'

Here's the full output:

usage: /usr/local/bin/bert-serving-start -cpu -num_worker=1 -model_dir /model -http_port 3011
                 ARG   VALUE
__________________________________________________
           ckpt_name = bert_model.ckpt
         config_name = bert_config.json
                cors = *
                 cpu = True
          device_map = []
  fixed_embed_length = False
                fp16 = False
 gpu_memory_fraction = 0.5
       graph_tmp_dir = None
    http_max_connect = 10
           http_port = 3011
        mask_cls_sep = False
      max_batch_size = 256
         max_seq_len = 25
           model_dir = /model
          num_worker = 1
       pooling_layer = [-2]
    pooling_strategy = REDUCE_MEAN
                port = 5555
            port_out = 5556
       prefetch_size = 10
 priority_batch_size = 16
show_tokens_to_client = False
     tuned_model_dir = None
             verbose = False
                 xla = False

I:VENTILATOR:[__i:__i: 66]:freeze, optimize and export graph, could take a while...
I:GRAPHOPT:[gra:opt: 52]:model config: /model/bert_config.json
I:GRAPHOPT:[gra:opt: 55]:checkpoint: /model/bert_model.ckpt
I:GRAPHOPT:[gra:opt: 59]:build graph...

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.

I:GRAPHOPT:[gra:opt:128]:load parameters from checkpoint...
I:GRAPHOPT:[gra:opt:132]:optimize...
I:GRAPHOPT:[gra:opt:140]:freeze...
I:GRAPHOPT:[gra:opt:145]:write graph to a tmp file: /tmp/tmpmn3y4_p_
I:VENTILATOR:[__i:__i: 74]:optimized graph is stored at: /tmp/tmpmn3y4_p_
I:VENTILATOR:[__i:_ru:128]:bind all sockets
I:VENTILATOR:[__i:_ru:132]:open 8 ventilator-worker sockets
I:VENTILATOR:[__i:_ru:135]:start the sink
I:SINK:[__i:_ru:303]:ready
I:VENTILATOR:[__i:_ge:219]:get devices
I:VENTILATOR:[__i:_ge:252]:device map: 
        worker  0 -> cpu
I:VENTILATOR:[__i:_ru:151]:start http proxy
I:WORKER-0:[__i:_ru:514]:use device cpu, load graph from /tmp/tmpmn3y4_p_
I:WORKER-0:[__i:gen:542]:ready and listening!
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.5/dist-packages/bert_serving/server/__init__.py", line 114, in run
    self._run()
  File "/usr/local/lib/python3.5/dist-packages/zmq/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/zmq/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/zmq/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/bert_serving/server/zmq_decor.py", line 27, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/bert_serving/server/__init__.py", line 160, in _run
    p.is_ready.wait()
AttributeError: 'function' object has no attribute 'wait'

...

Source

gregsherrid

Most helpful comment

aha got it! the error shows up when you start with http, fixing it now

hanxiao on 30 Mar 2019

❤2

All 8 comments

hmm, I can't see how this line breaks the code. Test is fine and I can't reproduce this in my environment.

hanxiao on 29 Mar 2019

Had the same issue. After downgrading to my previously used version 1.8.2 it worked.

bjornnes on 29 Mar 2019

I'm running into the same error using bert-serving-server[http] (Within or outside Docker).

pip3 install bert-serving-server[http]
bert-serving-start -model_dir $PATH_MODEL -num_worker=1 -cpu -http_port 8125

Here is a similar trace

usage: /usr/local/bin/bert-serving-start -model_dir /Users/****/Desktop/publication/uncased_L-24_H-1024_A-16 -num_worker=1 -cpu -http_port 8125
                 ARG   VALUE
__________________________________________________
           ckpt_name = bert_model.ckpt
         config_name = bert_config.json
                cors = *
                 cpu = True
          device_map = []
  fixed_embed_length = False
                fp16 = False
 gpu_memory_fraction = 0.5
       graph_tmp_dir = None
    http_max_connect = 10
           http_port = 8125
        mask_cls_sep = False
      max_batch_size = 256
         max_seq_len = 25
           model_dir = /Users/*****/Desktop/publication/uncased_L-24_H-1024_A-16
          num_worker = 1
       pooling_layer = [-2]
    pooling_strategy = REDUCE_MEAN
                port = 5555
            port_out = 5556
       prefetch_size = 10
 priority_batch_size = 16
show_tokens_to_client = False
     tuned_model_dir = None
             verbose = False
                 xla = False

I:VENTILATOR:[__i:__i: 66]:freeze, optimize and export graph, could take a while...
I:GRAPHOPT:[gra:opt: 52]:model config: /Users/*****/Desktop/publication/uncased_L-24_H-1024_A-16/bert_config.json
I:GRAPHOPT:[gra:opt: 55]:checkpoint: /Users/*****/Desktop/publication/uncased_L-24_H-1024_A-16/bert_model.ckpt
I:GRAPHOPT:[gra:opt: 59]:build graph...

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.

I:GRAPHOPT:[gra:opt:128]:load parameters from checkpoint...
I:GRAPHOPT:[gra:opt:132]:optimize...
I:GRAPHOPT:[gra:opt:140]:freeze...
I:GRAPHOPT:[gra:opt:145]:write graph to a tmp file: /var/folders/lm/s8kzdmsn0nd_3gp9zyffcxd80000gn/T/tmpz7uj9871
I:VENTILATOR:[__i:__i: 74]:optimized graph is stored at: /var/folders/lm/s8kzdmsn0nd_3gp9zyffcxd80000gn/T/tmpz7uj9871
I:VENTILATOR:[__i:_ru:128]:bind all sockets
I:VENTILATOR:[__i:_ru:132]:open 8 ventilator-worker sockets
I:VENTILATOR:[__i:_ru:135]:start the sink
I:SINK:[__i:_ru:303]:ready
I:VENTILATOR:[__i:_ge:219]:get devices
I:VENTILATOR:[__i:_ge:252]:device map: 
        worker  0 -> cpu
I:VENTILATOR:[__i:_ru:151]:start http proxy
I:WORKER-0:[__i:_ru:514]:use device cpu, load graph from /var/folders/lm/s8kzdmsn0nd_3gp9zyffcxd80000gn/T/tmpz7uj9871
I:WORKER-0:[__i:gen:542]:ready and listening!
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.6/site-packages/bert_serving/server/__init__.py", line 114, in run
    self._run()
  File "/usr/local/lib/python3.6/site-packages/zmq/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/zmq/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/zmq/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/bert_serving/server/zmq_decor.py", line 27, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/bert_serving/server/__init__.py", line 160, in _run
    p.is_ready.wait()
AttributeError: 'function' object has no attribute 'wait'

Using 1.8.2 worked for me!

GillesFerone on 29 Mar 2019

aha got it! the error shows up when you start with http, fixing it now

hanxiao on 30 Mar 2019

❤2

fyi, this issue if fixed in #299 and the new feature is available since 1.8.8, please do

pip install -U bert-serving-server bert-serving-client

for the update.

hanxiao on 30 Mar 2019

🎉1

sorry guys, this needs to be opened again

hanxiao on 2 Apr 2019

fyi, this issue is fixed in #305 and the new feature is available since 1.8.9, please do

pip install -U bert-serving-server bert-serving-client

for the update.

When you start the server with -http_port, you should see something like this

I:VENTILATOR:[__i:_ru:163]:all set, ready to serve request!

this means the server is ready.

hanxiao on 2 Apr 2019

Does anyone how long does it take to run this? Mine stayed in I:GRAPHOPT:[gra:opt:140]:freeze... for nearly one day and have not continued

chingting on 16 May 2019

Was this page helpful?

0 / 5 - 0 ratings

Related issues

BertClient con't connect

k-oul · 7Comments

Lower layer giving better results

astariul · 7Comments

Doubt: Can I use this service to obtain docvecs/Paragraph vector of an entire article

kapilkd13 · 4Comments

zmq.error.ZMQError: Address already in use

calliwen · 7Comments

fail to optimize the graph!

flyzaway · 3Comments