Deep Learning Seminar, Summer 2017/18

In recent years, deep neural networks have been used to solve complex machine-learning problems and have achieved significant state-of-the-art results in many areas. The whole field of deep learning has been developing rapidly, with new methods and techniques emerging steadily.

The goal of the seminar is to follow the newest advancements in the deep learning field. The course takes form of a reading group – each lecture a paper is presented by one of the students. The paper is announced in advance, hence all participants can read it beforehand and can take part in the discussion of the paper.

If you want to receive announcements about chosen paper, sign up to our mailing list ufal-rg@googlegroups.com.

About

SIS code: NPFL117
Semester: summer
E-credits: 3
Examination: 0/2 C
Guarantor: Milan Straka

Timespace Coordinates

The Deep Learning Seminar takes place on Tuesday at 14:00 in S1. We will first meet on Tuesday Feb 27.

Requirements

To pass the course, you need to present a research paper and sufficiently attend the presentations.

License

Unless otherwise stated, teaching materials for this course are available under CC BY-SA 4.0.

If you want to receive announcements about chosen paper, sign up to our mailing list ufal-rg@googlegroups.com.

To add your name and paper to the table below, edit the source code on GitHub and send a PR.

Date Who Paper(s)
27 Feb 2018 Milan Straka Alexis Conneau, Guillaume Lample, Marc'Aurelio Ranzato, Ludovic Denoyer, Hervé Jégou: Word Translation Without Parallel Data
06 Mar 2018 Martin Popel Michal Rolínek, Georg Martius: L4: Practical loss-based stepsize adaptation for deep learning
13 Mar 2018 Jan Hajič Matthias Dorfer, Jan Schlüter, Andreu Vall, Filip Korzeniowsky, Gerhard Widmer: End-to-End Cross-Modality Retrieval with CCA Projection and Pairwise Ranking Loss
20 Mar 2018 Tomas Soucek Barret Zoph, Vijay Vasudevan, Jonathon Shlens, Quoc V. Le: Learning Transferable Architectures for Scalable Image Recognition
27 Mar 2018 Petr Bělohlávek Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Łukasz Kaiser, Noam Shazeer, Alexander Ku: Image Transformer
03 Apr 2018 Petr Houška Jason Liang, Elliot Meyerson, Risto Miikkulainen: Evolutionary Architecture Search For Deep Multitask Networks
10 Apr 2018 Milan Straka Esteban Real, Alok Aggarwal, Yanping Huang, Quoc V Le: Regularized Evolution for Image Classifier Architecture Search
Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy: Progressive Neural Architecture Search
17 Apr 2018 Ronald Cardenas Jiatao Gu, Hany Hassan, Jacob Devlin, Victor O.K. Li: Universal Neural Machine Translation for Extremely Low Resource Languages
24 Apr 2018 Jakub Náplava Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, Oriol Vinyals: Understanding deep learning requires rethinking generalization
Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky: Deep Image Prior
01 May 2018 No DL Seminar Holiday – May Day
08 May 2018 No DL Seminar Holiday – Victory Day
15 May 2018 Vojtěch Čermák Adversarial Patch and Defense-GAN: Protecting classifiers against adversarial attacks using generative models
22 May 2018 Karel Ha CapsuleGAN: Generative Adversarial Capsule Network (with the background in Dynamic Routing Between Capsules) - slides

You can choose any paper you find interesting, but if you would like some inspiration, you can look at the following list.

Current Deep Learning Papers

Parsing

Neural Machine Translation

Language Modelling

Paraphrasing

Natural Language Generation

Speech Synthesis

  • Aaron van den Oord, Yazhe Li, Igor Babuschkin, Karen Simonyan, Oriol Vinyals, Koray Kavukcuoglu, George van den Driessche, Edward Lockhart, Luis C. Cobo, Florian Stimberg, Norman Casagrande, Dominik Grewe, Seb Noury, Sander Dieleman, Erich Elsen, Nal Kalchbrenner, Heiga Zen, Alex Graves, Helen King, Tom Walters, Dan Belov, Demis Hassabis: Parallel WaveNet: Fast High-Fidelity Speech Synthesis
  • Jonathan Shen, Ruoming Pang, Ron J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, RJ Skerry-Ryan, Rif A. Saurous, Yannis Agiomyrgiannakis, Yonghui Wu: Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions

Image Classification

Image Recognition

Image Enhancement

Image 3D Reconstruction

Training Methods

Activation Functions

Regularization

Network Architectures

Network Interpretation

Reinforcement Learning

Explicit Memory

Hyperparameter Optimization

Generative Adversarial Networks

Adversarial Images

  • Tom B. Brown, Dandelion Mané, Aurko Roy, Martín Abadi, Justin Gilmer: Adversarial Patch

Adversarial Text

Adversarial Speech

Artificial Intelligence