Overview
Part of our fortnightly series of paper reading sessions.
We're reading BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Devlin et al., 2018). It takes the pre-training idea from the GPT paper we read last time and makes it bidirectional: instead of predicting the next word from the words before it, the model hides words and fills them in using the text on both sides. Read it before you come: https://arxiv.org/abs/1810.04805
Check out the journal club channel on our applicant Discord server: https://discord.gg/y6AXyzbXUX
How the session works
This is not a walkthrough. We'll split into small groups to work through a set of questions, then regroup for answers and an open discussion.
Who is this for
Experienced software engineers. You don't need a research background, though you'll get more out of the evening if you're comfortable with the basics of neural networks. You don't need to have come to the earlier sessions.
What to bring
The paper, read beforehand. You won't need a laptop.
Full text on Meetup
.