Question Answering

Question answering

Question Answering (QA) requires a machine to answer the given question by reading documents, graphs or web pages. It is not only useful for applications such as search engines and chat-bots, but also a meaningful test bed for evaluating machines' ability to understand and reason over natural language text. Our group leads the state-of-the-art techniques for various question answering tasks, such as a reading comprehension model with bi-directional attention flow, a real-time open-domain QA model, a sequence generation model for math equations, and a question decomposition model for multi-hop QA. We have also developed useful resources such as real-time demos of reading comprehension, open-domain QA & multi-hop QA, and datasets for GRE math problems with finegrained supervision, so please give it a try!

Related Publications

2024

Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, Hannaneh Hajishirzi
ICLR 2024
Project page PDF Source code

2023

RealTime QA: What's the Answer Right Now?
Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, Kentaro Inui
NeurIPS (Dataset & Benchmark) 2023
Project page PDF Source code

How to Train Your Dragon: Diverse Augmentation Towards Generalizable Dense Retrieval
Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, Xilun Chen
EMNLP 2023 (Findings)
PDF Source code

Crystal: Introspective Reasoners Reinforced with Self-Feedback
Jiacheng Liu, Ramakanth Pasunuru, Hannaneh Hajishirzi, Yejin Choi, Asli Celikyilmaz
EMNLP 2023
Project page PDF Demo Source code

Vera: A General-Purpose Plausibility Estimation Model for Commonsense Statements
Jiacheng Liu, Wenya Wang, Dianzhuo Wang, Noah A. Smith, Yejin Choi, Hannaneh Hajishirzi
EMNLP 2023
Project page PDF Demo Source code

Fine-Grained Human Feedback Gives Better Rewards for Language Model Training
Zeqiu Wu*, Yushi Hu*, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari Ostendorf, Hannaneh Hajishirzi
NeurIPS 2023
Project page PDF Source code

When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, Hannaneh Hajishirzi
ACL 2023
PDF Source code

xPQA: Cross-Lingual Product Question Answering across 12 Languages
Xiaoyu Shen, Akari Asai, Bill Byrne, Adrià de Gispert
ACL 2023 (Industry)
PDF Source code

CREPE: Open-Domain Question Answering with False Presuppositions
Xinyan Velocity Yu, Sewon Min, Luke Zettlemoyer, Hannaneh Hajishirzi
ACL 2023
PDF Source code

Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, Huan Sun
ACL 2023
PDF Source code

INSCIT: Information-Seeking Conversations with Mixed-Initiative Interactions
Zeqiu Wu, Ryu Parish, Hao Cheng, Sewon Min, Prithviraj Ammanabrolu, Mari Ostendorf, Hannaneh Hajishirzi
TACL 2023
PDF Source code

2022

ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts
Akari Asai, Mohammadreza Salehi, Matthew E. Peters, Hannaneh Hajishirzi
EMNLP 2022
PDF Source code

Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering
Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi, Yejin Choi
EMNLP 2022
PDF Demo Source code

CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning
Zeqiu Wu, Yi Luan, Hannah Rashkin, David Reitter, Hannaneh Hajishirzi, Mari Ostendorf, Gaurav Singh Tomar
EMNLP 2022
PDF

Exploring The Landscape of Distributional Robustness for Question Answering Models
Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Hannaneh Hajishirzi, Ludwig Schmidt
Findings of EMNLP 2022
PDF

Re-Examining Calibration: The Case of Question Answering
Chenglei Si, Chen Zhao, Sewon Min, Jordan Boyd-Graber
Findings of EMNLP 2022
PDF

Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks
Akari Asai, Matt Gardner, Hannaneh Hajishirzi
NAACL 2022
PDF Semantic scholar Source code

Retrieval-guided Counterfactual Generation for QA
Bhargavi Paranjape, Matthew Lamm, Ian Tenney
ACL 2022
PDF

Generated Knowledge Prompting for Commonsense Reasoning
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, Hannaneh Hajishirzi
ACL 2022
PDF Source code

2021

One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval
Akari Asai, Xinyan Yu, Jungo Kasai, Hannaneh Hajishirzi
NeurIPS 2021
PDF Semantic scholar Source code

GooAQ: Open Question Answering with Diverse Answer Types
Daniel Khashabi, Amos Ng, Tushar Khot, Ashish Sabharwal, Hannaneh Hajishirzi, Chris Callison-Burch
EMNLP Findings, 2021
PDF Semantic scholar Source code

FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation
Kushal Lakhotia*, Bhargavi Paranjape, Asish Ghoshal, Wen-tau Yih, Yashar Mehdad, Srinivasan Iyer*
EMNLP 2021
PDF

Joint Passage Ranking for Diverse Multi-Answer Retrieval
Sewon Min, Kenton Lee, Ming-Wei Chang, Kristina Toutanova, Hannaneh Hajishirzi
EMNLP 2021
PDF

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, and EfficientQA participants
PMLR 2021
PDF

Challenges in Information Seeking QA: Unanswerable Questions and Paragraph Retrieval
Akari Asai, Eunsol Choi
ACL 2021
PDF Semantic scholar Source code

Efficient Passage Retrieval with Hashing for Open-domain Question Answering
Ikuya Yamada, Akari Asai, Hannaneh Hajishirzi
ACL 2021
PDF Semantic scholar Source code

XOR QA: Cross-lingual Open-Retrieval Question Answering
Akari Asai, Jungo Kasai, Jonathan H. Clark, Kenton Lee, Eunsol Choi, Hannaneh Hajishirzi
NAACL 2021
Project page PDF Semantic scholar Source code

ReConsider: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
Srinivasan Iyer, Sewon Min, Yashar Mehdad, Wen-tau Yih
NAACL 2021
PDF Source code

MultiModalQA: Complex Question Answering over Text, Tables and Images
Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, Jonathan Berant
ICLR 2021
PDF

2020

MedICaT: A Dataset of Medical Images, Captions, and Textual References
Sanjay Subramanian, Lucy Lu Wang, Ben Bogin, Sachin Mehta, Madeleine van Zuylen, Sravanthi Parasa, Sameer Singh, Matt Gardner, Hannaneh Hajishirzi
Findings of EMNLP 2020.
PDF Source code

An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction
Bhargavi Paranjape, Mandar Joshi, John Thickstun, Hannaneh Hajishirzi, Luke Zettlemoyer
EMNLP 2020
PDF Source code

LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, Yuji Matsumoto
EMNLP 2020
PDF Semantic scholar Source code

Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia
Ikuya Yamada, Akari Asai, Jin Sakuma, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji, Yuji Matsumoto
EMNLP 2020 (system demonstrations)
Project page PDF Semantic scholar Demo Source code

AmbigQA: Answering Ambiguous Open-domain Questions
Sewon Min, Julian Michael, Hannaneh Hajishirzi, Luke Zettlemoyer
EMNLP 2020
Project page PDF Semantic scholar Source code

UnifiedQA: Crossing Format Boundaries With a Single QA System
Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, Hannaneh Hajishirzi
Findings of EMNLP 2020
PDF Semantic scholar Source code

Efficient One-Pass End-to-End Entity Linking for Questions
Belinda Z. Li, Sewon Min, Srinivasan Iyer, Yashar Mehdad, Wen-tau Yih
EMNLP 2020

Dense Passage Retrieval for Open-domain Question Answering
Vladimir Karpukhin*, Barlas Oguz*, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih
EMNLP 2020
PDF Semantic scholar Demo Source code

Contextualized Sparse Representation with Rectified N-Gram Attention for Open-Domain Question Answering
Jinhyuk Lee, Minjoon Seo, Hannaneh Hajishirzi, Jaewoo Kang
ACL 2020
PDF Source code

Logic-Guided Data Augmentation and Regularization for Consistent Question Answering
Akari Asai, Hannaneh Hajishirzi
ACL 2020
PDF Semantic scholar Source code

Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, Caiming Xiong
ICLR 2020
PDF Semantic scholar Source code

2019

A Discrete Hard EM Approach for Weakly Supervised Question Answering
Sewon Min, Danqi Chen, Hannaneh Hajishirzi, Luke Zettlemoyer
EMNLP 2019
PDF Source code

Real-Time Open-Domain QA with Dense-Sparse Phrase Index (DenSPI)
Minjoon Seo*, Jinhyuk Lee*, Tom Kwiatkowski, Ankur Parikh, Ali Farhadi, Hannaneh Hajishirzi
ACL 2019
Project page PDF Demo Source code

Multi-hop Reading Comprehension through Question Decomposition and Rescoring
Sewon Min, Victor Zhong, Luke Zettlemoyer, Hannaneh Hajishirzi
ACL 2019
PDF Demo Source code

Compositional Questions Do Not Necessitate Multi-hop Reasoning
Sewon Min*, Eric Wallace*, Sameer Singh, Matt Gardner, Hannaneh Hajishirzi, Luke Zettlemoyer
ACL 2019
PDF Source code

MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms
Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, Hannaneh Hajishirzi
NAACL 2019
Project page PDF

2018

Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension
Minjoon Seo, Tom Kwiatkowski, Ankur P. Parikh, Ali Farhadi, Hannaneh Hajishirzi
EMNLP 2018
PDF Source code

Neural Speed Reading via Skim-RNN
Minjoon Seo*, Sewon Min*, Ali Farhadi, Hannaneh Hajishirzi
ICLR 2018
PDF

2017

Zero-Shot Relation Extraction via Reading Comprehension
Omer Levy, Minjoon Seo, Eunsol Choi, Luke Zettlemoyer
CoNLL 2017
Project page PDF

Question Answering through Transfer Learning from Large Fine-grained Supervision Data
Sewon Min, Minjoon Seo, Hannaneh Hajishirzi
ACL 2017
PDF Source code

Bidirectional Attention Flow for Machine Comprehension
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, Hannaneh Hajishirzi
ICLR 2017
Project page PDF Demo Source code

Query-Reduction Networks for Question Answering
Minjoon Seo, Sewon Min, Ali Farhadi, Hannaneh Hajishirzi
ICLR 2017
PDF Source code

Are You Smarter Than A Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension
Aniruddha Kembhavi, Minjoon Seo, Eric Klove, Dustin Schwenk, Hannaneh Hajishirzi, Ali Farhadi
CVPR 2017
Project page PDF