Bert-as-service: Why not make max_seq_len a client hyper-parameter?

[hugochan]

Prerequisites

Please fill in by replacing [ ] with [x].

• [x] Are you running the latest bert-as-service?
• [x] Did you follow the installation and the usage instructions in README.md?
• [x] Did you check the FAQ list in README.md?
• [x] Did you perform a cursory search on existing issues?

---

Description

Firstly, great work!

I wonder if it makes sense to make max_seq_len a client hyper-parameter. The benefit is that we can hence dynamically change max_seq_len when encoding sequences with various lengths. In my experiment, I have to deal with long sequences (i.e., >1000 tokens) as well as short sequences (i.e., < 10 tokens), and I don't think it is efficient to set max_seq_len to 512 on the server side.

Please let me know what you think about it. Thanks!

[hanxiao]

You are partially right. setting a restriction max_seq_len on the server side isn't the best option. The reason for doing that was because in Google's BERT implementation max_seq_len has to be a python integer (not a tf.tensor and then dynamically determined).

In #236 I've removed such restriction by patching the BERT implementation. You can now set max_seq_len=NONE when starting a server. In this case, the max_seq_len will be determined by the longest sequence in a batch (or mini-batch if parallelization is activated). That means you can send any sequence shorter than max_position_embeddings (usually 512) defined in bert.json.

You may also want to check the new argument -fixed_embed_length by bert-serving-start --help especially if you intend to use it as ELMo-like embedding.

However, using BERT to encode a long long sentence (> 1000 tokens as you mentioned) is not a good idea, see https://github.com/hanxiao/bert-as-service/issues/232#issuecomment-462762704

[hanxiao]

fyi, this issue is fixed in #236 and the new feature is available since 1.8.2. Please do

pip install bert-serving-client bert-serving-server -U

for the update.