Bert-as-service: Any ideas about sentence similarity in Chinese language?

[denity]

Hi. This project is wonderful. But I try it for sentence similarity in Chinese, the result is bad.

Here was my process:
I used the default parameters and loaded Chinese BERT model (chinese_L-12_H-768_A-12), passed Chinese sentences ( or splited word with space ) to BERT-SERVICE. Then I got ndarrays from service. Finally, I calculated cos() of them. But the result wasn't well.

Is there any suggestion for me? Where am I wrong? Should I pass the original sentences or make some preprocess, like split etc?

[hanxiao]

my experience is that the Chinese BERT released by Google gives reasonable results in simple clustering task, i.e. sentences have similar semantics are gathered. I also observe this applies to text from other domains (the released BERT is trained on Chinese Wiki).

Some remarks may help you solve the problem:

1. Chinese BERT is a character-based model. No need to do segmentation beforehand.
2. Try different pooling strategy, please read https://github.com/hanxiao/bert-as-service#q-what-are-the-available-pooling-strategies
3. How long is your sentence? It shouldn't be too short, say less than 5 Chinese characters. It shouldn't be too long, if the sequence is longer than max_seq_len defined on the server side, the sequence will be truncated on the right side. Say you have a "sentence" that has 500 characters, whereas server-side max_seq_len=20, then the last 480 characters are not fed to BERT at all.

[denity]

Thanks for your reply. I will try again.

[lingdavid]

@hanxiao I also encountered this problem,I tried the methods you provided，but not work. Some irrelevant sentences will also calculate very high similarities（cosin）.