deephack_NLP.md

Natural Language Processing by DeepHackLab

Курс был построен на лекциях by Richard Socher

1. Intro to NLP and Deep Learning [lecture]

   • Трансляция
   • Слайды
   • Доп. материалы:
     • Lecture Notes 1
     • Linear Algebra Review
     • Probability Review
     • Convex Optimization Review
     • More Optimization (SGD) Review
     • From Frequency to Meaning: Vector Space Models of Semantics
     • Python Numpy Tutorial

2. Word Vectors [lecture]

   • Слайды
   • Доп. материалы:
     • Distributed Representations of Words and Phrases and their Compositionality
     • Efficient Estimation of Word Representations in Vector Space

3. More Word Vectors [lecture]

   • Слайды
   • Доп. материалы:
     • Lecture Notes 2
     • GloVe: Global Vectors for Word Representation
     • Improving Word Representations via Global Context and Multiple Word Prototypes

4. Neural Networks and backpropagation — for named entity recognition [lecture]

   • Слайды
   • Задание: https://github.com/deephacklab/courses/blob/master/NLP/home_work_1/HM1_w2v.ipynb
   • Доп. материалы:
     • Lecture Notes 3
     • UFLDL tutorial
     • Learning Representations by Backpropagating Errors

5. Project Advice, Neural Networks and Back-Prop (in full gory detail) [lecture]

   • Слайды
   • Доп. материалы:
     • Natural Language Processing (almost) from Scratch
     • A Neural Network for Factoid Question Answering over Paragraphs
     • Grounded Compositional Semantics for Finding and Describing Images with Sentences
     • Deep Visual-Semantic Alignments for Generating Image Descriptions
     • Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

6. Practical tips: gradient checks, overfitting, regularization, activation functions, details [lecture]

   • Слайды
   • Доп. материалы:
     • Practical recommendations for gradient-based training of deep architectures
     • UFLDL page on gradient checking

7. Recurrent neural networks — for language modeling and other tasks [lecture]

   • Слайды
   • Доп. лекция по Tensorflow
   • Слайды к доп. лекции
   • Доп. материалы:
     • Recurrent neural network based language model
     • Extensions of recurrent neural network language model
     • Opinion Mining with Deep Recurrent Neural Networks
     • Tensorflow

8. GRUs and LSTMs — for machine translation [lecture]

   • Слайды
   • Доп. материалы:
     • Lecture Notes 4
     • Long Short-Term Memory
     • Gated Feedback Recurrent Neural Networks
     • Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

9. Convolutional neural networks – for sentence classification [lecture *Note: RNN here stands for Recursive NN not for Recurrent NN]

   • Слайды
   • Доп. материалы:
     • A Convolutional Neural Network for Modelling Sentences

10. Applications of DL in NLP [lecture *Note: RNN here stands for Recursive NN not for Recurrent NN]