Text: Sequence Tagging custom dataset

[nth-attempt]

❓ Questions and Help

Description
I have a sequence tagging dataset with text and label spans. E.g.
text = 'Apple is located\nin Palo Alto'
label_spans = [(0, 5, 'Company'), (20, 29, 'Location')]

What is the best approach to create a Dataset and a DataLoader?

I also have a function to pre-process labels into their corresponding BIO labels. Where would I pre-process the labels?

Any guidance will help. Thanks!

[zhangguanheng66]

@akurniawan has re-written our sequence tagging datasets here. You might take a look at the pipeline here. The pipeline should be fully compatible with DataLoader.

[nth-attempt]

Thank you. Is my understanding correct that 'raw' folder has IterableDatasets downloading from a URL and the main folder has Datasets from local paths?

[zhangguanheng66]

Yes. The raw dataset download, unzip files and generate IterableDataset. From the link above, you materialize the IterableDataset and generate the dataset.

[nth-attempt]

Nice. Is the idea to give more control to the user? How would this fit in with the existing Field class? With the new pipeline, where would be batching, padding be handled? Also, if we want to add special tokens like <sos>, <eos> to the vocab, I guess we incorporate them while building our Vocab?

[zhangguanheng66]

Field class will be retired and users should use DataLoader for batching and collate_func for padding. And yes, you will have to build vocab by yourself. Here is the issue to review the new abstraction https://github.com/pytorch/text/issues/664

[nth-attempt]

Awesome. Thanks!

[nth-attempt]

How do we go about using char level and word level features in the new pipeline?

[akurniawan]

@nth-attempt I wrote this example for machine translation for using char level features, hopefully can give you some idea for using both char and word features

[zhangguanheng66]

@akurniawan, do you think it's worthy to merge the example to the folder text/examples?

[akurniawan]

sure, will create a PR for that

[nth-attempt]

@akurniawan thanks for the example. So, I believe I could have char_vocab, char_transforms, word_vocab, word_transforms in the Dataset class and return both char_tensor and word_tensor on get_item. At that point, I am not using anything from the torchtext library other than Vocab class right?

[zhangguanheng66]

I believe you can also use SentencePiece to get the subword level information. Some utils are available here

[akurniawan]

@nth-attempt not necessarily, you can also utilize some of the functionalities in torchtext library such as vocab_func and sequential_transforms for transforming the word level and as @zhangguanheng66 said SentencePiece tokenizer for the subword level. I needed to create separate function for the character level as I felt the current function can't handle nested dimension as what happened in the char level.
If you want, you can try to look at text/torchtext/data/functional.py and text/torchtext/experimental/ for more active development in new torchtext API

[nth-attempt]

Got it. Will be watching that space. Thanks!

[zhangguanheng66]

Here is an example to use sentencepiece API for text classification dataset link. I'm thinking about a new example or an updated one for spm.