Farm: Long Sequence for NER Tagging

[bohanzhou]

Question

Hello,

Currently, I have developed a NER Tagging model with BERT using FARM version 0.4.5
When I run the model in Inference Mode it works well for text strings shorter than 512 tokens.
However, when we exceed the token limit, BERT just truncates it to the 512 limit which results in the model not tagging all the entities.
This is quite annoying since there often are long texts to process.

I have seen the function chunk_into_passages but I'm not sure how to implement it due to the lack of documentation of what the function inputs are and how the output of this function should be used.
Also, the NER model is for Swedish and I have only found BERT models that perform well on Swedish

Am I on the right track or is there a better solution to this problem?

Thanks in advance!

[Timoeller]

Hey @bohanzhou Bert for longer sequences is of course interesting.

1. Your proposed spot "chunk into passages" is making Question Answering with Bert work for any text length. Though this is something created internally and rather complex, as it involves splitting and especially aggregating across splits.
2. Since you only want inference you could wrap around sliding window functionality around the inferencer by feeding it chunks of text and aggregating the labels. There you would only need to take of overlapping tokens.
3. Try out the models that support longer sequences for NER, like longformer or reformer - but I havent seen pretrained NER models for this. Do you know more about this?

Either way, we would help you implementing 1 or 2 within FARM and would be glad about contributions.

[bohanzhou]

Thank you for your reply @Timoeller!

I have not tried the models that support longer sequences because I couldn't find any Swedish models for these
Currently, I am trying to use method number 2 to solve this problem but haven't come very far

It would be great if FARM could automatically handle longer sequences for other problems than QA

[Timoeller]

If you are going for 2, then maybe not try having overlapping windows. A Simpler solution would be to split the text into sentences and feed in sentence by sentence.
Spacy works rather well for sentence separation (unsure how good for Swedish, but English and German is close to perfect)

[bohanzhou]

Thanks alot for the tip! I have had alot of other stuff to take care of but I will definately try it 👍

[Timoeller]

You could also try using the excellent library flair: https://github.com/flairNLP/flair
They have good NER integrations, also for long sequences that you are interested into.

[bohanzhou]

Yeah, I agree that flair is a great library! I have tried to use flair but it did not perform as well as using FARM to fine tune a BERT model.

Update regarding splitting sequence into sentences: It worked! However, it was quite slow so I sped up it by processing three sentences at once

[Timoeller]

awesome, you mean you put three sentences into one batch and before were computing NER for each sentence separately?

You could even try increase the number of sentences and batch size, it should roughly increase speed linearly (up to a certain point). And if the issue is fixed, could you close it please?

[bohanzhou]

Yes, it's currently processing the text in batches of three. I will try to increase it and see how it goes

Once again, thank you so much for your help!