Haystack: Articles/News/Paper about QA

[Utomo88]

https://rajpurkar.github.io/SQuAD-explorer/

https://www.forbes.com/sites/forbestechcouncil/2020/07/06/using-conversational-ai-to-find-answers/#154822f94efd

https://ai.google.com/research/NaturalQuestions/competition

I think we can post new things about Question and Answering AI in this thread
Please delete/ close if this is NOT Allowed.

┆Issue is synchronized with this Jira Task by Unito

[tholor]

Sure, feel free to add interesting articles here.
Some further resources:

• QA Datasets in different languages https://towardsdatascience.com/going-beyond-squad-part-1-question-answering-in-different-languages-8eac6cf56f21
• QA Datasets for other styles of QA https://towardsdatascience.com/going-beyond-squad-part-2-d085fb736191
• NeurIPS competition about efficient Open-Domain QA https://efficientqa.github.io/

[Utomo88]

https://bdtechtalks.com/2020/07/13/ai-barrier-meaning-understanding/

https://bdtechtalks.com/2020/01/13/melanie-mitchell-ai-guide-for-thinking-humans/

[Utomo88]

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
https://www.aclweb.org/anthology/volumes/2020.acl-main/

[tholor]

I like the overview of ACL here: https://www.paperdigest.org/2020/06/acl-2020-highlights/

Some papers I found interesting:

• DeFormer: "We introduce DeFormer, a decomposed transformer, which substitutes the full self-attention with question-wide and passage-wide self-attentions in the lower layers." https://arxiv.org/abs/2005.00697
• Span Selection Pre-Training: "In this paper we introduce a new pre-training task inspired by reading comprehension to better align the pre-training from memorization to understanding." https://arxiv.org/abs/1909.04120
• TechQA: "We introduce TECHQA, a domain-adaptation question answering dataset for the technical support domain." https://www.aclweb.org/anthology/2020.acl-main.117/
• Unsupervised Domain Clusters in Pretrained Language Models: "We harness this property and propose domain data selection methods based on such models, which require only a small set of in-domain monolingual data." https://www.aclweb.org/anthology/2020.acl-main.692

[Utomo88]

GPT 3
https://medium.com/@haroonchoudery/gpt-3-what-you-should-know-about-ais-big-breakthrough-3eff8e895fb5

https://www.forbes.com/sites/robtoews/2020/07/19/gpt-3-is-amazingand-overhyped/#103aa22f1b1c

https://openai.com/blog/openai-api/
https://arxiv.org/abs/2005.14165

[ahotrod]

I found this article & notebook interesting & applicable to FARM/Haystack:

NLP Text Preprocessing & Vectorizing

Covid-19 8M Tweets Example Notebook using GPU TfidfVectorizer

[tholor]

ScANN: new, highly scalable vector similarity search by google https://ai.googleblog.com/2020/07/announcing-scann-efficient-vector.html

[tholor]

ANCE - New dense retriever that uses harder negatives via contrastive Learning (kind of a logical next step from DPR)
https://arxiv.org/abs/2007.00808

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs.

[Utomo88]

FastFormers: 233x Faster Transformers inference on CPU
Yes, 233x on CPU with the multi-head self-attentive Transformer architecture. This is not an LSTM or an RNN

https://parthplc.medium.com/fastformers-233x-faster-transformers-inference-on-cpu-4c0b7a720e1