Farm: Wrong indent for "results" variable prevents data set processing of squad like json file

[RobKnop]

Describe the bug
Just a wrong indent of the variable "results". In the current version, there is one indent and so the results just get executed if the else block is true before. That's why in my case the results variable is not defined and I get the following error:

Error message

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/farm/data_handler/data_silo.py in _load_data(self, train_dicts, dev_dicts, test_dicts)
    188             train_file = self.processor.data_dir / self.processor.train_filename
    189             logger.info("Loading train set from: {} ".format(train_file))
--> 190             self.data["train"], self.tensor_names = self._get_dataset(train_file)
    191 
    192         # dev data

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/farm/data_handler/data_silo.py in _get_dataset(self, filename, dicts)
    159 
    160             with tqdm(total=len(dicts), unit=' Dicts', desc="Preprocessing Dataset") as pbar:
--> 161                 for dataset, tensor_names in results:
    162                     datasets.append(dataset)
    163                     pbar.update(multiprocessing_chunk_size)

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py in next(self, timeout)
    746         if success:
    747             return value
--> 748         raise value
    749 
    750     __next__ = next                    # XXX

IndexError: Cannot choose from an empty sequence

Expected behavior
sequence should not be empty

To Reproduce
Just try

# and fine-tune it on your own custom dataset (should be in SQuAD like format)
train_data = "training_data"
reader.train(data_dir=train_data, train_filename="2020-02-09_answers.json", use_gpu=False, n_epochs=1)

System:

• OS: MacOS Mojave
• GPU/CPU: CPU
• FARM version: 0.4.1

FIX:
Under data_handler > data_silo.py > DataSilo > _get_dataset

with ExitStack() as stack:
            if self.max_processes > 1:  # use multiprocessing only when max_processes > 1
                p = stack.enter_context(mp.Pool(processes=num_cpus_used))

                logger.info(
                    f"Got ya {num_cpus_used} parallel workers to convert {num_dicts} dictionaries "
                    f"to pytorch datasets (chunksize = {multiprocessing_chunk_size})..."
                )
                log_ascii_workers(num_cpus_used, logger)

                results = p.imap(
                    partial(self._dataset_from_chunk, processor=self.processor),
                    grouper(dicts, multiprocessing_chunk_size),
                    chunksize=1,
                )
            else:
                logger.info(
                    f"Multiprocessing disabled, using a single worker to convert {num_dicts}"
                    f"dictionaries to pytorch datasets."
                )
            #######################################################
            ####fix indent here ###############
            results = map(partial(self._dataset_from_chunk, processor=self.processor), grouper(dicts, num_dicts))
            #########################################
            datasets = []

            with tqdm(total=len(dicts), unit=' Dicts', desc="Preprocessing Dataset") as pbar:
                for dataset, tensor_names in results:
                    datasets.append(dataset)
                    pbar.update(multiprocessing_chunk_size)

            concat_datasets = ConcatDataset(datasets)
            return concat_datasets, tensor_names

I would have created a pull request but I'm not able to create a new branch.
Hopefully this helps. Let me know if you have further questions.

[RobKnop]

maybe related to #238

[tanaysoni]

Hi @RobKnop, thank you for raising the issue.

The results variable should get defined as it is present in both if and else blocks.

I suspect in your case the training data that you have in 2020-02-09_answers.json isn't complying with the SQuAD2.0 format? You could try again with the SQuAD dataset from this link and see if you still get an error.

As for creating a pull request, you can _fork_ the FARM repository and create a new branch there with your changes. Then, you can raise a pull request with that branch. You can find more documentation on this here.

[RobKnop]

Hey @tanaysoni
Thank you for your fast reply.
Yes, you are right. Sorry for falsely raising the issues.
Your data set from the given link works fine.
--> there is some issue in mine. But I created this dataset with https://annotate.deepset.ai/ actually.
If I have more information on this I can share / open a issue again.

[tanaysoni]

Hey @RobKnop, I'm unable to reproduce the error using an export of labels from https://annotate.deepset.ai/. Would it be possible (and if the dataset is public) to share the labels file you used?

[RobKnop]

Sure @tanaysoni, it's public:
2020-02-17_answers.txt
You just have to rename it to .json (I couldn’t upload a json file)
I know it is a very long text.

when I remove the indent it works.

[RobKnop]

I found it. It is because my answers has empty 'qas'. Sorry for messing with your time :/

{
"paragraphs": [
{
"qas": [],
"context": "4,2020-02-09 17:51:43,2020-02-09 17:51:43

[tanaysoni]

Hi @RobKnop, thanks again for raising the issue. The empty qas is indeed a valid case that FARM and possibly the labelling tool(not exporting documents without answers) should deal with.

[RobKnop]

@tanaysoni Should I go ahead and close the issue?
That's fine for me.