NTR organizes and hosts scientific webinars on neural networks and invites speakers from all over the world to present their recent work.
On November 23 Yury Zemlyanskiy, University of Southern California, Los Angeles, USA, presented a technical Zoom webinar on Towards Trustworthy Natural Language Processing.
About the webinar:
Knowledge-intensive NLP tasks, such as question answering, often require assimilating information from multiple sources in the text.
Unfortunately, modern Transformer-based methods can only process input of a rather limited size. This makes them harder to use on tasks where input is large such as a whole book or Wikipedia.
We proposed to address this problem by extracting and representing information in text corpus in a semi-parametric manner. Specifically, our method represents knowledge with “Mention Memory,” a table of high-dimensional vector encodings of every entity mentioned in the text.
Furthermore, we integrated Mention Memory into a Transformer model allowing the synthesis of, and reasoning over, many disparate sources of information. Finally, we experimented with a challenging Narrative QA task, with questions about entire books (https://aclanthology.org/2021.naacl-main.408), and several open-domain benchmarks with questions about the whole Wikipedia (https://arxiv.org/abs/2110.06176).
Webinar presentation.
Moderator and contact: NTR CEO Nick Mikhailovsky: nickm@ntrlab.com.