Text: Problem processing a text regression dataset

[felipebravom]

Hi,
I'm trying to train a regression model for emotion intensity prediction.
My data looks as follows:

ID  Tweet   Affect_Dimension    Intensity_Score
2017-En-20968   @RockSolidShow @Pat_Francis #revolting cocks if you think I'm sexy! fear    0.292
2017-En-21816   @Its_just_Huong I will beat you !!! Always thought id be gryffindor so this is a whole new world for me 😨😨😨 #excited #afraid    fear    0.667
2017-En-21532   “What worries you masters you.” - Haddon Robinson @ChoGMinistries #Jesusisthesubject #worry #anxiety #anxious   fear    0.604

So, I would like to get the intensity_score appended to my batches as a float column, so I can apply a regression loss function such as L1Loss.

However, I can't figure out how to declare numeric labels in torchtext.
My code is given below:

TEXT = data.Field(sequential=True,tokenize='spacy', lower=True, include_lengths=True)
LABEL = data.Field(sequential=False, use_vocab=False, tensor_type='torch.FloatTensor')


train, valid, test = data.TabularDataset.splits(path='.data/', 
                                   train = 'EI-reg-En-fear-train.txt',
                                   validation = '2018-EI-reg-En-fear-gold-dev.txt',
                                   test = '2018-EI-reg-En-fear-gold-test.txt',
                                   format='tsv', 
                                   fields=[(None,None),("Tweet",TEXT), (None,None), ("Intensity Score",LABEL)],
                                   skip_header=True)



TEXT.build_vocab(train, max_size=25000, vectors="glove.6B.100d")

BATCH_SIZE = 64

train_iterator, valid_iterator, test_iterator = data.BucketIterator.splits(
    (train, valid, test), 
    batch_size=BATCH_SIZE, 
    sort_key=lambda x: len(x.text), 
    repeat=False)


for batch in train_iterator:
    print(batch.Tweet)
    break

and I'm getting the following error:

ValueError: Specified Field tensor_type torch.FloatTensor can not be used with use_vocab=False because we do not know how to numericalize it. Please raise an issue at https://github.com/pytorch/text/issues

any help would be appreciated.

Cheers.

[mttk]

Which torchtext version are you using?
I tried loading on master, and while there is an unrelated bug (which will be fixed shortly edit: fixed now in master), everything works fine.

[felipebravom]

Hi,
I was using version 0.2.3. Just updated to the master version via:

pip install --upgrade git+https://github.com/pytorch/text

Then redefined my LABEL field as follows:
LABEL = data.Field(sequential=False, use_vocab=False, dtype=torch.float)

and was able to load my data into batches.

Thanks!

[mttk]

Great!