Reading Group

This page is dedicated to the NLP reading group @ Université de Lorraine. Meetings are scheduled every other Thursday.

This reading group is a spiritual child of last year's reading group, organized by Hoa Le. You can find all the relevant information on that previous reading group here.

One of the goals of this reading group is to provide a space dedicated to discussion for the different NLP labs across Nancy (Synalp, Atilf, ...). To that end, we try to make the presentations as inclusive as possible.

If you'd like to present something, but can't think of a paper to present, there's a spreadsheet with some suggestions. The sheet is editable: don't hesitate to add a suggestion yourself!

Session of Feb 14, 19

Date: Feb 14, 19
Presented by: Timothee Mickus
Paper presented: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (link)
Download slides: link

A recent advance in the field of word embeddings is the rise of contextualized embeddings. This first session of the reading group was dedicated to one of the most popular contextual embedding architecture, BERT.

BERT is also related to the Transformer architecture of Vaswani and colleagues. See also the Transformer-XL of Dai and al. Here's a github of variations on the Transformer.

Session of Feb 28, 19

Date: Feb 28, 19
Presented by: Timothee Mickus
Paper presented: No Training Required: Exploring Random Encoders for Sentence Classification (link)
Download slides: link

Sentence embeddings have been a focus of research over the past few years, with new algorithms, datasets and benchmarks. The paper for this second session takes a step back and tries to evaluate how much these sentence embeddings actually gain over random methods.

Session of Mar 14, 19

Date: Mar 14, 19
Presented by: Angela Fan
Paper presented: Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context (link)
Download slides: link

Session of Mar 28, 19

Date: Mar 28, 19
Presented by: Hoa Le
Paper presented: Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling (link)
Link to slides: link

Session of Apr 11, 19

Date: Apr 11, 19 at 14:30
Location: Loria, B009
Presented by: Anastasia Shimorina
Paper presented: Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages (link)
Download slides: link

Session of Apr 25, 19

Date: Apr 25, 19 at 14:30
Location: Loria, A015
Presented by: Hoa Le
Papers presented: Sequence Level Training with Recurrent Neural Networks (link) An Actor-Critic Algorithm for Sequence Prediction (link) Self-critical Sequence Training for Image Captioning (link)
Link to slides: link

This session is dedicated to giving a broad understanding of reinforcement learning.

Session of Jun 13, 19

Date: Jun 13, 19 at 10:30
Location: Loria, A217
Presented by: Timothee Mickus
Paper presented: Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned (link)
Download slides: link

Session of Sep 13, 19

Date: Sep 13, 19 at 14:30
Location: Loria, B009
Presented by: Anastasia Shimorina
Papers presented: Deep Dominance - How to Properly Compare Deep Neural Models (link) We Need to Talk about Standard Splits (link) Aiming beyond the Obvious: Identifying Non-Obvious Cases in Semantic Similarity Datasets (link)
Download slides: link

In this session, Anastasia will give us a highlight of some papers from ACL 2019, which was held in Florence this summer.