NLP

Word Embeddings

POS Tagging

  • Wang Ling, Tiago Luís, Luís Marujo, Ramón Fernandez Astudillo, Silvio Amir, Chris Dyer, Alan W. Black, Isabel Trancoso: Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation. http://arxiv.org/abs/1508.02096
  • Barbara Plank, Anders Søgaard, Yoav Goldberg: Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss. https://arxiv.org/abs/1604.05529
  • Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, Richard Socher: A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks. https://arxiv.org/abs/1611.01587

NER

Parsing

Neural Machine Translation

  • Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio: Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. https://arxiv.org/abs/1406.1078
  • Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio Neural Machine Translation by Jointly Learning to Align and Translate. https://arxiv.org/abs/1409.0473
  • Ilya Sutskever, Oriol Vinyals, Quoc V. Le: Sequence to Sequence Learning with Neural Networks. https://arxiv.org/abs/1409.3215
  • Rico Sennrich, Barry Haddow, Alexandra Birch: Neural Machine Translation of Rare Words with Subword Units. https://arxiv.org/abs/1508.07909
  • Orhan Firat, Kyunghyun Cho, Yoshua Bengio: Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism. https://arxiv.org/abs/1601.01073
  • Rico Sennrich, Barry Haddow, Alexandra Birch: Edinburgh Neural Machine Translation Systems for WMT 16. https://arxiv.org/abs/1606.02891
  • Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T. Yarman Vural, Kyunghyun Cho: Zero-Resource Translation with Multi-Lingual Neural Machine Translation. https://arxiv.org/abs/1606.04164
  • Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, Wei Xu: Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation. https://arxiv.org/abs/1606.04199
  • Yonghui Wu et al.: Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. https://arxiv.org/abs/1609.08144
  • Jason Lee, Kyunghyun Cho, Thomas Hofmann: Fully Character-Level Neural Machine Translation without Explicit Segmentation. https://arxiv.org/abs/1610.03017
  • Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aaron van den Oord, Alex Graves, Koray Kavukcuoglu: Neural Machine Translation in Linear Time. https://arxiv.org/abs/1610.10099
  • Melvin Johnson et al.: Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. https://arxiv.org/abs/1611.04558
  • Thanh-Le Ha, Jan Niehues, Alexander Waibel: Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder. https://arxiv.org/abs/1611.04798

Language Modelling

Language Correction

  • Ziang Xie, Anand Avati, Naveen Arivazhagan, Dan Jurafsky, Andrew Y. Ng: Neural Language Correction with Character-Based Attention. https://arxiv.org/abs/1603.09727

Summarization

Speech Synthesis

  • Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, Koray Kavukcuoglu: WaveNet: A Generative Model for Raw Audio. https://arxiv.org/abs/1609.03499

Image Processing

Image Classification

Image Segmentation

Image Labeling

  • Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erhan: Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge. https://arxiv.org/abs/1609.06647

Image Recognition

Image Enhancement

Deep Learning

Training Methods

Regularization

  • Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov: Improving neural networks by preventing co-adaptation of feature detectors. https://arxiv.org/abs/1207.0580
  • Sergey Ioffe, Christian Szegedy: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. https://arxiv.org/abs/1502.03167
  • César Laurent, Gabriel Pereyra, Philémon Brakel, Ying Zhang, Yoshua Bengio: Batch Normalized Recurrent Neural Networks. https://arxiv.org/abs/1510.01378
  • Stanislau Semeniuta, Aliaksei Severyn, Erhardt Barth: Recurrent Dropout without Memory Loss. https://arxiv.org/abs/1603.05118
  • Tim Cooijmans, Nicolas Ballas, César Laurent, Çağlar Gülçehre, Aaron Courville: Recurrent Batch Normalization. https://arxiv.org/abs/1603.09025
  • David Krueger, Tegan Maharaj, János Kramár, Mohammad Pezeshki, Nicolas Ballas, Nan Rosemary Ke, Anirudh Goyal, Yoshua Bengio, Hugo Larochelle, Aaron Courville, Chris Pal: Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations. https://arxiv.org/abs/1606.01305
  • Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton: Layer Normalization. https://arxiv.org/abs/1607.06450

Network Architectures

  • Rupesh Kumar Srivastava, Klaus Greff, Jürgen Schmidhuber: Training Very Deep Networks. https://arxiv.org/abs/1507.06228
  • Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, Nando de Freitas: Learning to learn by gradient descent by gradient descent. https://arxiv.org/abs/1606.04474
  • Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, Jürgen Schmidhuber: Recurrent Highway Networks. https://arxiv.org/abs/1607.03474
  • Lingpeng Kong, Chris Alberti, Daniel Andor, Ivan Bogatyy, David Weiss: DRAGNN: A Transition-Based Framework for Dynamically Connected Neural Networks. https://arxiv.org/abs/1703.04474
  • Bowen Baker, Otkrist Gupta, Nikhil Naik, Ramesh Raskar: Designing Neural Network Architectures using Reinforcement Learning. https://arxiv.org/abs/1611.02167

Non-diffentiable Loss Functions

  • Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, Wojciech Zaremba: Sequence Level Training with Recurrent Neural Networks. https://arxiv.org/abs/1511.06732
  • Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, Yang Liu: Minimum Risk Training for Neural Machine Translation. https://arxiv.org/abs/1512.02433
  • Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, Yoshua Bengio: An Actor-Critic Algorithm for Sequence Prediction. https://arxiv.org/abs/1607.07086

Structured Prediction

  • Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, Michael Collins: Globally Normalized Transition-Based Neural Networks. https://arxiv.org/abs/1603.06042
  • Sam Wiseman, Alexander M. Rush: Sequence-to-Sequence Learning as Beam-Search Optimization. https://arxiv.org/abs/1606.02960

Activation Functions

Reinforcement Learning

Variational Autoencoders

Explicit Memory

Adversarial Networks

Books

Blogs

Paper Lists