NTR Webinar: Semi-parametric Methods for Extracting and Representing Knowledge From a Text Corpus

NTR organizes and hosts scientific webinars on neural networks and invites speakers from all over the world to present their recent work. 

On November 23 Yury Zemlyanskiy, University of Southern California, Los Angeles, USA, presented a technical Zoom webinar on Towards Trustworthy Natural Language Processing

About the webinar: 

Knowledge-intensive NLP tasks, such as question answering, often require assimilating information from multiple sources in the text. 

Unfortunately, modern Transformer-based methods can only process input of a rather limited size. This makes them harder to use on tasks where input is large such as a whole book or Wikipedia. 

We proposed to address this problem by extracting and representing information in text corpus in a semi-parametric manner. Specifically, our method represents knowledge with “Mention Memory,” a table of high-dimensional vector encodings of every entity mentioned in the text. 

Furthermore, we integrated Mention Memory into a Transformer model allowing the synthesis of, and reasoning over, many disparate sources of information. Finally, we experimented with a challenging Narrative QA task, with questions about entire books (https://aclanthology.org/2021.naacl-main.408), and several open-domain benchmarks with questions about the whole Wikipedia (https://arxiv.org/abs/2110.06176).

Webinar presentation.

Moderator and contact: NTR CEO Nick Mikhailovsky: nickm@ntrlab.com.

Leave a Reply

Your email address will not be published. Required fields are marked *