Haystack: **Urgent** structure data to predict the whole paragraph as an answer

[sadakmed]

Question

I have a question answer data, for each question there's an answer (paragraph), I want to have the whole paragraphs as an answer.

Any idea on how to structure the data. I tried cdqa but it's not that good,

[guillim]

The retriever only should do the trick, just skip the Reader in your case. See tutorial 4 : the FAQ style

Try to match the question with the paragraph embedded. To say it another way, use the answer as if it was an FAQ style question.

[sadakmed]

@guillim ~I think it will generalize or in other words the match will be lexical~

Edit: I just check the code, does using the reader is a plus, or doesn't help at all?

[maxupp]

The retriever identifies candidate passages, which the reader analyzes to find the exact answer. LIke guillim said, just use the retriever without a reader.

[sadakmed]

The code is well written, for the use of retriever only, it uses the paper dense passage retrieval two transformers for both question and passage, However it for my case did not gave the expected results. Using the reader I think BM25 is better than Tf-idf if you want to use in-memory.