Exploring Sentence Retrieval

Sentence Retrieval is the task of retrieving a relevant sentence in response to a query, a question, or a reference sentence. In this work we begin by demonstrating that because sentences are much smaller than documents, the performance of typical document retrieval systems on the retrieval of sentences is significantly worse. We propose several solutions to the problem of sentence retrieval, based on statistical translation models, and investigate these solutions the application areas of sentence retrieval for question answering, novelty detection, and information provenance. Statistical trans­lation models are appropriate for tasks where the sentence to be retrieved benefits from the addition of related terms and synonyms. The context of a sentence affects its meaning, and smoothing from the local context of the sentence improves retrieval. A brief investigation of conditional models for sentence retrieval suggests conditional models outperform language modeling approaches, for some tasks. This book is addressed to students and professionals working on language technology systems that are dependent on sentence or passage retrieval.