In recent years, deep neural networks have been used to solve complex machine-learning problems. They have achieved significant state-of-the-art results in many areas.
The goal of the course is to introduce deep neural networks, from the basics to the latest advances. The course will focus both on theory as well as on practical aspects (students will implement and train several deep neural networks capable of achieving state-of-the-art results, for example in named entity recognition, dependency parsing, machine translation, image labeling or in playing video games). No previous knowledge of artificial neural networks is required, but basic understanding of machine learning is advisable.
SIS code: NPFL114
Semester: summer
E-credits: 7
Examination: 3/2 C+Ex
Guarantor: Milan Straka
1. Introduction to Deep Learning Slides PDF Slides 2018 Video numpy_entropy mnist_layers_activations
2. Training Neural Networks Slides PDF Slides 2018 Video mnist_training gym_cartpole
3. Training Neural Networks II Slides PDF Slides 2018 Video mnist_regularization mnist_ensemble uppercase
4. Convolutional Neural Networks Slides PDF Slides 2018 Video mnist_cnn cifar_competition
5. Convolutional Neural Networks II Slides PDF Slides 2018 Video mnist_multiple fashion_masks
6. Convolutional Neural Networks III, Recurrent Neural Networks Slides PDF Slides 2018 Video I 2018 Video II caltech42_competition sequence_classification
7. Recurrent Neural Networks II Slides PDF Slides 2018 Video I 2018 Video II 2018 Video III 2018 Video IV tagger_we tagger_cle_rnn tagger_cle_cnn speech_recognition
8. Easter Monday tagger_competition 3d_recognition
9. Recurrent Neural Networks III Slides PDF Slides 2018 Video I 2018 Video II 2018 Video III 2018 Video IV lemmatizer_noattn lemmatizer_attn lemmatizer_competition
10. Deep Generative Models Slides PDF Slides 2018 Video vae gan dcgan nli_competition
11. Speech Synthesis, Reinforcement Learning Slides PDF Slides 2018 Video I 2018 Video II omr_competition monte_carlo reinforce reinforce_baseline reinforce_pixels
12. Transformer, External Memory Networks Slides PDF Slides
To pass the practicals, you need to obtain at least 80 points, which are awarded for home assignments. Note that up to 40 points above 80 will be transfered to the exam.
To pass the exam, you need to obtain at least 60, 75 and 90 out of 100 points for the written exam (plus up to 40 points from the practicals), to obtain grades 3, 2 and 1, respectively.
The lecture content, including references to study materials. The main study material is the Deep Learning Book by Ian Goodfellow, Yoshua Bengio and Aaron Courville, (referred to as DLB).
References to study materials cover all theory required at the exam, and sometimes even more – the references in italics cover topics not required for the exam.
Mar 04 Slides PDF Slides 2018 Video numpy_entropy mnist_layers_activations
Mar 11 Slides PDF Slides 2018 Video mnist_training gym_cartpole
Mar 18 Slides PDF Slides 2018 Video mnist_regularization mnist_ensemble uppercase
Mar 25 Slides PDF Slides 2018 Video mnist_cnn cifar_competition
Apr 01 Slides PDF Slides 2018 Video mnist_multiple fashion_masks
Apr 08 Slides PDF Slides 2018 Video I 2018 Video II caltech42_competition sequence_classification
Apr 15 Slides PDF Slides 2018 Video I 2018 Video II 2018 Video III 2018 Video IV tagger_we tagger_cle_rnn tagger_cle_cnn speech_recognition
Apr 22 tagger_competition 3d_recognition
Easter Monday
Apr 29 Slides PDF Slides 2018 Video I 2018 Video II 2018 Video III 2018 Video IV lemmatizer_noattn lemmatizer_attn lemmatizer_competition
Word2vec
word embeddings, notably the CBOW and Skip-gram architectures [Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean: Efficient Estimation of Word Representations in Vector Space]
May 06 Slides PDF Slides 2018 Video vae gan dcgan nli_competition
May 13 Slides PDF Slides 2018 Video I 2018 Video II omr_competition monte_carlo reinforce reinforce_baseline reinforce_pixels
Study material for Reinforcement Learning is the Reinforcement Learning: An Introduction; second edition by Richard S. Sutton and Andrew G. Barto (reffered to as RLB), available online.
May 20 Slides PDF Slides
The tasks are evaluated automatically using the ReCodEx Code Examiner. The evaluation is performed using Python 3.6, TensorFlow 2.0.0a0, NumPy 1.16.1 and OpenAI Gym 0.9.5.
You can install all required packages either to user packages using
pip3 install --user tensorflow==2.0.0a0 gym==0.9.5
,
or create a virtual environment using python3 -m venv VENV_DIR
and then installing the packages inside it by running
VENV_DIR/bin/pip3 install tensorflow==2.0.0a0 gym==0.9.5
.
If you have a GPU, you can install GPU-enabled TensorFlow by using
tensorflow-gpu
instead of tensorflow
.
Working in teams of size 2 (or at most 3) is encouraged. All members of the team must submit in ReCodEx individually, but can have exactly the same sources/models/results. However, each such solution must explicitly list all members of the team to allow plagiarism detection using this template.
Deadline: Mar 17, 23:59 3 points
The goal of this exercise is to famirialize with Python, NumPy and ReCodEx submission system. Start with the numpy_entropy.py.
Load a file numpy_entropy_data.txt
, whose lines consist of data points of our
dataset, and load numpy_entropy_model.txt
, which describes a model probability distribution,
with each line being a tab-separated pair of (data point, probability).
Example files are in the labs/01.
Then compute the following quantities using NumPy, and print them each on
a separate line rounded on two decimal places (or inf
for positive infinity,
which happens when an element of data distribution has zero probability
under the model distribution):
Use natural logarithms to compute the entropies and the divergence.
Deadline: Mar 17, 23:59 3 points
The templates changed on Mar 11 because of the upgrade to TF 2.0.0a0, be sure to use the updated ones when submitting!
In order to familiarize with TensorFlow and TensorBoard, start by playing with
example_keras_tensorboard.py.
Run it, and when it finishes, run TensorBoard using tensorboard --logdir logs
.
Then open http://localhost:6006 in a browser and explore the active tabs.
Your goal is to modify the mnist_layers_activations.py template and implement the following:
layers
.activation
, with supported values of none
, relu
, tanh
and sigmoid
.In addition to submitting the task in ReCodEx, please also run the following variations and observe the results in TensorBoard:
0
layers, activation none
1
layer, activation none
, relu
, tanh
, sigmoid
10
layers, activation sigmoid
, relu
Deadline: Mar 24, 23:59 4 points
This exercise should teach you using different optimizers, learning rates, and learning rate decays. Your goal is to modify the mnist_training.py template and implement the following:
SGD
or Adam
).SGD
optimizer.exponential
or polynomial
(with degree 1, so inverse time decay).
Additionally, the final learning rate is given and the decay should gradually
decrease the learning rate to reach the final learning rate just after the
training.In addition to submitting the task in ReCodEx, please also run the following variations and observe the results in TensorBoard:
SGD
optimizer, learning_rate
0.01;SGD
optimizer, learning_rate
0.01, momentum
0.9;SGD
optimizer, learning_rate
0.1;Adam
optimizer, learning_rate
0.001;Adam
optimizer, learning_rate
0.01;Adam
optimizer, exponential
decay, learning_rate
0.01 and learning_rate_final
0.001;Adam
optimizer, polynomial
decay, learning_rate
0.01 and learning_rate_final
0.0001.Deadline: Mar 24, 23:59 4 points
Solve the CartPole-v1 environment from the OpenAI Gym, utilizing only provided supervised training data. The data is available in gym_cartpole-data.txt file, each line containing one observation (four space separated floats) and a corresponding action (the last space separated integer). Start with the gym_cartpole.py.
The solution to this task should be a model which passes evaluation on random
inputs. This evaluation is performed by running the
gym_cartpole_evaluate.py,
which loads a model and then evaluates it on 100 random episodes (optionally
rendering if --render
option is provided). In order to pass, you must achieve
an average reward of at least 475 on 100 episodes. Your model should have either
one or two outputs (i.e., using either sigmoid of softmax output function).
The size of the training data is very small and you should consider it when designing the model.
When submitting your model to ReCodEx, submit:
h5
suffix),py
suffix),
and possibly indicating teams.Deadline: Mar 31, 23:59 6 points
You will learn how to implement three regularization methods in this assignment. Start with the mnist_regularization.py template and implement the following:
args.dropout
. Add a dropout layer after the
first Flatten
and also after all Dense
hidden layers (but not after the
output layer).args.l2
. Use
tf.keras.regularizers.L1L2
as a regularizer for all kernels and biases
of all Dense
layers (including the last one).args.label_smoothing
. Instead
of SparseCategoricalCrossentropy
, you will need to use
CategoricalCrossentropy
which offers label_smoothing
argument.In ReCodEx, there will be three tests (one for each regularization methods) and you will get 2 points for passing each one.
In addition to submitting the task in ReCodEx, also run the following variations and observe the results in TensorBoard (notably training, development and test set accuracy and loss):
0
, 0.3
, 0.5
, 0.6
, 0.8
;0
, 0.001
, 0.0001
, 0.00001
;0
, 0.1
, 0.3
, 0.5
.Deadline: Mar 31, 23:59 2 points
Your goal in this assignment is to implement model ensembling.
The mnist_ensemble.py
template trains args.models
individual models, and your goal is to perform
an ensemble of the first model, first two models, first three models, …, all
models, and evaluate their accuracy on the development set.
In addition to submitting the task in ReCodEx, run the script with
args.models=7
and look at the results in mnist_ensemble.out
file.
Deadline: Mar 31, 23:59 4-9 points
This assignment introduces first NLP task. Your goal is to implement a model which is given Czech lowercased text and tries to uppercase appropriate letters. To load the dataset, use uppercase_data.py module which loads (and if required also downloads) the data. While the training and the development sets are in correct case, the test set is lowercased.
This is an open-data task, where you submit only the uppercased test set together with the training script (which will not be executed, it will be only used to understand the approach you took, and to indicate teams). Explicitly, submit exactly one .txt file and at least one .py file.
The task is also a competition. Everyone who submits a solution which achieves at least 96.5% accuracy will get 4 points; the rest 5 points will be distributed depending on relative ordering of your solutions, i.e., the best solution will get total 9 points, the worst solution (but at least with 96.5% accuracy) will get total 4 points. The accuracy is computed per-character and can be evaluated by uppercase_eval.py script.
You may want to start with the uppercase.py template, which uses the uppercase_data.py to load the data, generate an alphabet of given size containing most frequent characters, and generate sliding window view on the data. The template also comments on possibilities of character representation.
Do not use RNNs or CNNs in this task (if you have doubts, contact me).
Deadline: Apr 07, 23:59 5 points
To pass this assignment, you will learn to construct basic convolutional
neural network layers. Start with the
mnist_cnn.py
template and assume the requested architecture is described by the cnn
argument, which contains comma-separated specifications of the following layers:
C-filters-kernel_size-stride-padding
: Add a convolutional layer with ReLU
activation and specified number of filters, kernel size, stride and padding.
Example: C-10-3-1-same
CB-filters-kernel_size-stride-padding
: Same as
C-filters-kernel_size-stride-padding
, but use batch normalization.
In detail, start with a convolutional layer without bias and activation,
then add batch normalization layer, and finally ReLU activation.
Example: CB-10-3-1-same
M-kernel_size-stride
: Add max pooling with specified size and stride.
Example: M-3-2
R-[layers]
: Add a residual connection. The layers
contain a specification
of at least one convolutional layer (but not a recursive residual connection R
).
The input to the specified layers is then added to their output.
Example: R-[C-16-3-1-same,C-16-3-1-same]
F
: Flatten inputs. Must appear exactly once in the architecture.D-hidden_layer_size
: Add a dense layer with ReLU activation and specified
size. Example: D-100
An example architecture might be --cnn=CB-16-5-2-same,M-3-2,F,D-100
.
After a successful ReCodEx submission, you can try obtaining the best accuracy
on MNIST and then advance to cifar_competition
.
Deadline: Apr 07, 23:59 5-10 points
The goal of this assignment is to devise the best possible model for CIFAR-10. You can load the data using the cifar10.py module. Note that the test set is different than that of official CIFAR-10.
This is an open-data task, where you submit only the test set labels together with the training script (which will not be executed, it will be only used to understand the approach you took, and to indicate teams). Explicitly, submit exactly one .txt file and at least one .py file.
The task is also a competition. Everyone who submits a solution which achieves at least 60% test set accuracy will get 5 points; the rest 5 points will be distributed depending on relative ordering of your solutions. Note that my solutions usually need to achieve at least ~73% on the development set to score 60% on the test set.
You may want to start with the cifar_competition.py template.
Deadline: Apr 14, 23:59 4 points
In this assignment you will implement a model with multiple inputs, multiple outputs, manual batch preparation, and manual evaluation. Start with the mnist_multiple.py template and:
Deadline: Apr 14, 23:59 5-11 points
This assignment is a simple image segmentation task. The data for this task is available through the fashion_masks_data.py The inputs consist of 28×28 greyscale images of ten classes of clothing, while the outputs consist of the correct class and a pixel bit mask.
This is an open-data task, where you submit only the test set annotations
together with the training script (which will not be executed, it will be
only used to understand the approach you took, and to indicate teams).
Explicitly, submit exactly one .txt
file and at least one .py
file.
Note that all .zip
files you submit will be extracted first.
Performance is evaluated using mean IoU, where IoU for a single example is defined as an intersection of the gold and system mask divided by their union (assuming the predicted label is correct; if not, IoU is 0). The evaluation (using for example development data) can be performed by fashion_masks_eval.py script.
The task is a competition and the points will be awarded depending on your test set score. If your test set score surpasses 75%, you will be awarded 5 points; the rest 6 points will be distributed depending on relative ordering of your solutions. Note that quite a straightfoward model surpasses 80% on development set after an hour of computation (and 90% after several hours), so reaching 75% is not that difficult.
You may want to start with the fashion_masks.py template, which loads the data and generates test set annotations in the required format (one example per line containing space separated label and mask, the mask stored as zeros and ones, rows first).
Deadline: Apr 21, 23:59 Apr 22, 23:59
5-10 points
The goal of this assignment is to try transfer learning approach to train image recognition on a small dataset with 42 classes. You can load the data using the caltech42.py module. In addition to the training data, you should use a MobileNet v2 pretrained network (details in caltech42_competition.py).
This is an open-data task, where you submit only the test set labels together with the training script (which will not be executed, it will be only used to understand the approach you took, and to indicate teams). Explicitly, submit exactly one .txt file and at least one .py file.
The task is also a competition. Everyone who submits a solution which achieves at least 94% test set accuracy will get 5 points; the rest 5 points will be distributed depending on relative ordering of your solutions.
You may want to start with the caltech42_competition.py template.
Deadline: Apr 21, 23:59 Apr 22, 23:59
6 points
The goal of this assignment is to introduce recurrent neural networks, manual TensorBoard log collection, and manual gradient clipping. Considering recurrent neural network, the assignment shows convergence speed and illustrates exploding gradient issue. The network should process sequences of 50 small integers and compute parity for each prefix of the sequence. The inputs are either 0/1, or vectors with one-hot representation of small integer.
Your goal is to modify the sequence_classification.py template and implement the following:
SimpleRNN
, GRU
and LSTM
) and dimensionality.In addition to submitting the task in ReCodEx, please also run the following variations and observe the results in TensorBoard. Concentrate on the way how the RNNs converge, convergence speed, exploding gradient issues and how gradient clipping helps:
--rnn_cell=SimpleRNN --sequence_dim=1
, --rnn_cell=GRU --sequence_dim=1
, --rnn_cell=LSTM --sequence_dim=1
--sequence_dim=2
--sequence_dim=10
--rnn_cell=LSTM --hidden_layer=50 --rnn_cell_dim=30 --sequence_dim=30
and the same with --clip_gradient=1
--rnn_cell=SimpleRNN
--rnn_cell=GRU --hidden_layer=150
Deadline: Apr 28, 23:59 3 points
In this assignment you will create a simple part-of-speech tagger. For training and evaluation, we will use Czech dataset containing tokenized sentences, each word annotated by gold lemma and part-of-speech tag. The morpho_dataset.py module (down)loads the dataset and can generate batches.
Your goal is to modify the tagger_we.py template and implement the following:
GRU
and LSTM
) and dimensionality.After submitting the task to ReCodEx, continue with tagger_cle_rnn
assignment.
Deadline: Apr 28, 23:59 3 points
This task is a continuation of tagger_we
assignment. Using the
tagger_cle_rnn.py
template, implement the following features in addition to tagger_we
:
Once submitted to ReCodEx, continue with tagger_cle_cnn
assignment. Additionaly, you should experiment with the effect of CLEs compared
to plain tagger_we
, and the influence of their dimensionality.
Note that tagger_we
has by default word embeddings twice the
size of word embeddings in tagger_cle_rnn
.
Deadline: Apr 28, 23:59 2 points
This task is a continuation of tagger_cle_rnn
assignment. Using the
tagger_cle_cnn.py
template, implement the following features compared to tagger_cle_rnn
:
Deadline: Apr 28, 23:59 7-12 points
This assignment is a competition task in speech recognition area. Specifically, your goal is to predict a sequence of letters given a spoken utterance. We will be using TIMIT corpus, with input sound waves passed through the usual preprocessing – computing Mel-frequency cepstral coefficients (MFCCs). You can repeat exactly this preprocessing on a given audio using the timit_mfcc_preprocess.py script.
Because the data is not publicly available, you can download it only through ReCodEx. Please do not distribute it. To load the dataset using the timit_mfcc.py module.
This is an open-data task, where you submit only the test set labels together with the training script (which will not be executed, it will be only used to understand the approach you took, and to indicate teams). Explicitly, submit exactly one .txt file and at least one .py file.
The task is also a competition. The evaluation is performed by computing edit
distance to the gold letter sequence, normalized by its length (i.e., exactly as
tf.edit_distance
). Everyone who submits a solution which achieves
at most 50% test set edit distance will get 7 points; the rest 5 points will be distributed
depending on relative ordering of your solutions. An evaluation (using for example development data)
can be performed by
speech_recognition_eval.py.
You should start with the speech_recognition.py template.
tf.nn.ctc_loss
),tf.nn.ctc_beam_search_decoder
)
and possibly evaluate results using normalized edit distance (tf.edit_distance
).Deadline: May 5, 23:59 5-13 points
In this assignment, you should extend
tagger_we
/tagger_cle_rnn
/tagger_cle_cnn
into a real-world Czech part-of-speech tagger. We will use
Czech PDT dataset loadable using the morpho_dataset.py
module. Note that the dataset contains more than 1500 unique POS tags and that
the POS tags have a fixed structure of 15 positions (so it is possible to
generate the POS tag characters independently).
You can use the following additional data in this assignment:
The assignment is again an open-data task, where you submit only the annotated test set
together with the training script (which will not be executed, it will be
only used to understand the approach you took, and to indicate teams).
Explicitly, submit exactly one .txt file and at least one .py file.
Note that all .zip
files you submit will be extracted first.
The task is also a competition. Everyone who submits a solution which achieves at least 92% label accuracy will get 5 points; the rest 5 points will be distributed depending on relative ordering of your solutions. Lastly, 3 bonus points will be given to anyone surpassing pre-neural-network state-of-the-art of 95.89% from Spoustová et al., 2009. You can evaluate generated file against a golden text file using the morpho_evaluator.py module.
You can start with the tagger_competition.py template, which among others generates test set annotations in the required format.
Deadline: May 5, 23:59 5-10 points
Your goal in this assignment is to perform 3D object recognition. The input is voxelized representation of an object, stored as a 3D grid of either empty or occupied voxels, and your goal is to classify the object into one of 10 classes. The data is available in two resolutions, either as 20×20×20 data or 32×32×32 data. To load the dataset, use the modelnet.py module.
The official dataset offers only train and test sets, with the test set having a different distributions of labels. Our dataset contains also a development set, which has nearly the same label distribution as the test set.
The assignment is again an open-data task, where you submit only the test set labels together with the training script (which will not be executed, it will be only used to understand the approach you took, and to indicate teams). Explicitly, submit exactly one .txt file and at least one .py file.
The task is also a competition. Everyone who submits a solution which achieves at least 85% label accuracy will get 5 points; the rest 5 points will be distributed depending on relative ordering of your solutions.
You can start with the 3d_recognition.py template, which among others generates test set annotations in the required format.
Deadline: May 12, 23:59 4 points
The goal of this assignment is to create a simple lemmatizer. For training and evaluation, we will use Czech dataset containing tokenized sentences, each word annotated by gold lemma and part-of-speech tag. The morpho_dataset.py module (down)loads the dataset and can generate batches.
Your goal is to modify the lemmatizer_noattn.py template and implement the following:
After submitting the task to ReCodEx, continue with lemmatizer_attn
assignment.
Deadline: May 12, 23:59 3 points
This task is a continuation of lemmatizer_noattn
assignment. Using the
lemmatizer_attn.py
template, implement the following features in addition to lemmatizer_noattn
:
Once submitted to ReCodEx, you should experiment with the effect of using the attention, and the influence of RNN dimensionality on network performance.
Deadline: May 12, 23:59 5-13 points
In this assignment, you should extend
lemmatizer_noattn
/lemmatizer_attn
into a real-world Czech lemmatizer. We will again use
Czech PDT dataset loadable using the morpho_dataset.py
module.
You can use the following additional data in this assignment:
The assignment is again an open-data task, where you submit only the annotated test set
together with the training script (which will not be executed, it will be
only used to understand the approach you took, and to indicate teams).
Explicitly, submit exactly one .txt file and at least one .py file.
Note that all .zip
files you submit will be extracted first.
The task is also a competition. Everyone who submits a solution which achieves at least 92% accuracy will get 5 points; the rest 5 points will be distributed depending on relative ordering of your solutions. Lastly, 3 bonus points will be given to anyone surpassing pre-neural-network state-of-the-art of 97.86%. You can evaluate generated file against a golden text file using the morpho_evaluator.py module.
You can start with the lemmatizer_competition.py template, which among others generates test set annotations in the required format.
Deadline: May 19, 23:59 3 points
In this assignment you will implement a simple Variational Autoencoder for three datasets in the MNIST format. Your goal is to modify the vae.py template and implement a VAE.
After submitting the assignment to ReCodEx, you can experiment with the three
available datasets (mnist
, mnist-fashion
, and mnist-cifarcars
) and
different latent variable dimensionality (z_dim=2
and z_dim=100
).
The generated images are available in TensorBoard logs.
Deadline: May 19, 23:59 3 points
In this assignment you will implement a simple Generative Adversarion Network for three datasets in the MNIST format. Your goal is to modify the gan.py template and implement a GAN.
After submitting the assignment to ReCodEx, you can experiment with the three
available datasets (mnist
, mnist-fashion
, and mnist-cifarcars
) and
maybe try different latent variable dimensionality. The generated images are
available in TensorBoard logs.
You can also continue with dcgan
assignment.
Deadline: May 19, 23:59 1 points
This task is a continuation of gan
assignment, which you will modify to
implement the Deep Convolutional GAN (DCGAN).
Start with the
dcgan.py
template and implement a DCGAN. Note that most of the TODO notes are from
the gan
assignment.
After submitting the assignment to ReCodEx, you can experiment with the three
available datasets (mnist
, mnist-fashion
, and mnist-cifarcars
). However,
note that you will need a lot of computational power (preferably a GPU) to
generate the images.
Deadline: May 19, 23:59 6-10 points
In this competition you will be solving the Native Language Identification task. In that task, you get an English essay writen by a non-native individual and your goal is to identify their native language.
We will be using NLI Shared Task 2013 data, which contains documents in 11 languages. For each language, the train, development and test sets contain 900, 100 and 100 documents, respectively. Particularly interesting is the fact that humans are quite bad in this task (in a simplified settings, human professionals achieve 40-50% accuracy), while machine learning models can achive high performance. Notably, the 2013 shared tasks winners achieved 83.6% accuracy, while current state-of-the-art is at least 87.1% (Malmasi and Dras, 2017).
Because the data is not publicly available, you can download it only through ReCodEx. Please do not distribute it. To load the dataset, use nli_dataset.py script.
The assignment is again an open-data task, where you submit only the annotated test set
together with the training script (which will not be executed, it will be
only used to understand the approach you took, and to indicate teams).
Explicitly, submit exactly one .txt file and at least one .py file.
Note that all .zip
files you submit will be extracted first.
The task is also a competition. If your test set accuracy surpasses 60%, you will be awarded 6 points; the rest 4 points will be distributed depending on relative ordering of your solutions.
You can start with the nli_competition.py template, which loads the data and generates predictions in the required format (language of each essay on a line).
Deadline: May 26, 23:59 7-15 points
You should implement optical music recognition in your final competition assignment. The inputs are PNG images of monophonic scores starting with a clef, key signature, and a time signature, followed by several staves. The dataset is loadable using the omr_dataset.py module and is downloaded automatically if missing (note that is has 185MB). No other data or pretrained models are allowed for training.
The assignment is again an open-data task, where you submit only the annotated test set
together with the training script (which will not be executed, it will be
only used to understand the approach you took, and to indicate teams).
Explicitly, submit exactly one .txt file and at least one .py file.
Note that all .zip
files you submit will be extracted first.
The task is also a competition. The evaluation is performed by computing edit
distance to the gold mark sequence, normalized by its length (i.e., exactly as
tf.edit_distance
). Everyone who submits a solution which achieves
at most 10% test set edit distance will get 7 points; the rest 4 points will be distributed
depending on relative ordering of your solutions. Furthermore, 4 bonus points
will be given to anyone surpassing current state-of-the-art of 0.80%.
An evaluation (using for example development data) can be performed by
speech_recognition_eval.py.
You can start with the omr_competition.py template, which among others generates test set annotations in the required format.
Deadline: May 26, 23:59 2 points
Solve the discretized CartPole-v1 environment environment from the OpenAI Gym using the Monte Carlo reinforcement learning algorithm.
Use the supplied cart_pole_evaluator.py module (depending on gym_evaluator.py) to interact with the discretized environment. The environment has the following methods and properties:
states
: number of states of the environmentactions
: number of actions of the environmentepisode
: number of the current episode (zero-based)reset(start_evaluate=False) → new_state
: starts a new episodestep(action) → new_state, reward, done, info
: perform the chosen action
in the environment, returning the new state, obtained reward, a boolean
flag indicating an end of episode, and additional environment-specific
informationrender()
: render current environment stateOnce you finish training (which you indicate by passing start_evaluate=True
to reset
), your goal is to reach an average return of 475 during 100
evaluation episodes. Note that the environment prints your 100-episode
average return each 10 episodes even during training.
You can start with the monte_carlo.py template, which parses several useful parameters, creates the environment and illustrates the overall usage.
During evaluation in ReCodEx, two different random seeds will be employed, and you need to reach the required return on all of them. Time limit for each test is 5 minutes.
Deadline: May 26, 23:59 2 points
Solve the continuous CartPole-v1 environment environment from the OpenAI Gym using the REINFORCE algorithm.
The supplied cart_pole_evaluator.py
module (depending on gym_evaluator.py)
can create a continuous environment using environment(discrete=False)
.
The continuous environment is very similar to the discrete environment, except
that the states are vectors of real-valued observations with shape environment.state_shape
.
Your goal is to reach an average return of 475 during 100 evaluation episodes. Start with the reinforce.py template.
During evaluation in ReCodEx, two different random seeds will be employed, and you need to reach the required return on all of them. Time limit for each test is 5 minutes.
Deadline: May 26, 23:59 2 points
This is a continuation of reinforce
assignment.
Using the reinforce_baseline.py template, solve the CartPole-v1 environment environment using the REINFORCE with baseline algorithm.
Using a baseline lowers the variance of the value function gradient estimator, which allows faster training and decreases sensitivity to hyperparameter values. To reflect this effect in ReCodEx, note that the evaluation phase will automatically start after 200 episodes. Using only 200 episodes for training in this setting is probably too little for the REINFORCE algorithm, but suffices for the variant with a baseline.
Your goal is to reach an average return of 475 during 100 evaluation episodes.
During evaluation in ReCodEx, two different random seeds will be employed, and you need to reach the required return on all of them. Time limit for each test is 5 minutes.
Deadline: May 26, 23:59 2 points
This is a continuation of reinforce
or reinforce_baseline
assignment.
The supplied cart_pole_pixels_evaluator.py
module (depending on gym_evaluator.py)
generates a pixel representation of the CartPole
environment
as an $80×80$ image with three channels, with each channel representing one time step
(i.e., the current observation and the two previous ones).
To pass the assignment, you need to reach an average return of 250 during 100 evaluation episodes. During evaluation in ReCodEx, two different random seeds will be employed, and you need to reach the required return on all of them. Time limit for each test is 10 minutes.
You can start with the reinforce_pixels.py template using the correct environment.
Training Neural Network
Assume the artificial neural network on the right, with mean square error loss
and gold output of 3. Compute the values of all weights $w_i$ after performing
an SGD update with learning rate 0.1.
Different networks architectures, activation functions (tanh
, sigmoid
,
softmax
) and losses (MSE
, NLL
) may appear in the exam.
Maximum Likelihood Estimation
Formulate maximum likelihood estimator for neural network parameters and derive
the following two losses:
Backpropagation Algorithm, SGD with Momentum
Write down the backpropagation algorithm. Then, write down the SGD algorithm
with momentum. Finally, formulate SGD with Nestorov momentum and explain the
difference to SGD with regular momentum.
Adagrad and RMSProp
Write down the AdaGrad algorithm and show that it tends to internally decay
learning rate by a factor of $1/\sqrt{t}$ in step $t$. Furthermore, write
down RMSProp algorithm and compare it to Adagrad.
Adam
Write down the Adam algorithm and explain the bias-correction terms
$(1-\beta^t)$.
Regularization
Define overfitting and sketch what a regularization is. Then describe
basic regularization methods like early stopping, L2 and L1 regularization,
dataset augmentation, ensembling and label smoothing.
Dropout
Describe the dropout method and write down exactly how is it used during training and
during inference. Then explain why it cannot be used on RNN state,
describe the variational dropout variant, and also describe layer
normalization.
Network Convergence
Describe factors influencing network convergence, namely:
Convolution
Write down equations of how convolution of a given image is computed. Assume the input
is an image $I$ of size $H \times W$ with $C$ channels, the kernel $K$
has size $N \times M$, the stride is $T \times S$, the operation performed is
n fact cross-correlation (as usual in convolutional neural networks)
and that $O$ output channels are computed. Explain both
$\textit{SAME}$ and $\textit{VALID}$ padding schemes and write down output
size of the operation for both these padding schemes.
Batch Normalization
Describe the batch normalization method and explain how it is used during
training and during inference. Explicitly write over what is being
normalized in case of fully connected layers, and in case of convolutional
layers. Compare batch normalization to layer normalization.
VGG and ResNet
Describe overall architecture of VGG and ResNet (you do not need to remember
exact number of layers/filters, but you should know when a BatchNorm is
executed, when ReLU, and how residual connections work when the number of
channels increases). Then describe two ResNet extensions (WideNet, DenseNet,
PyramidNet, ResNeXt).
Object Detection and Segmentation
Describe object detection and image segmentation tasks, and sketch Fast-RCNN, Faster-RCNN and
Mask-RCNN architectures. Notably, show what the overall architectures of the networks
are, explain the RoI-pooling and RoI-align layers, show how the network predicts RoI
sizes, how do the losses looks like, how are RoI chosen during training and
prediction, and what region proposal network does.
Object Detection
Describe object detection task, and sketch Fast-RCNN, Faster-RCNN and
RetinaNet architectures. Notably, show the overall architectures of the
networks, explain the RoI-pooling layer, show how the network predicts RoI
sizes, how do the losses looks like (classification loss, boundary prediction
loss, focal loss for RetinaNet), and what a feature pyramid network is.
LSTM
Write down how the Long Short-Term Memory cell operates.
GRU and Highway Networks
Show a basic RNN cell (using just one hidden layer) and then write down
how it is extended using gating into the Gated Recurrent Unit.
Finally, describe highway networks and compare them to RNN.
Sequence classification and CRF
Describe how RNNs, bidirectional RNNs and multi-layer RNNs can be used to
classify every element of a given sequence (i.e., what the architecture of
a tagger might be; include also residual connections and suitable places
for dropout layers). Then, explain how a CRF layer works, define score
computation for a given sequence of inputs and sequence of labels,
describe the loss computation during training, and sketch the inference
algorithm.
CTC Loss
Describe CTC loss and the whole settings which can be solved utilizing CTC
loss. Then show how CTC loss can be computed. Finally, describe greedy
and beam search CTC decoding.
Word2vec and Hierarchical and Negative Sampling
Explain how can word embeddings be precomputed using the CBOW and Skip-gram
models. First start with the variants where full softmax is performed, and
then describe how hierarchical softmax and negative sampling is used to speedup
training of word embeddings.
Character-level word embeddings
Describe why are character-level word embeddings useful. Then describe the
two following methods:
Neural Machine Translation and BPE
Draw/write how an encoder-decoder architecture is used for machine translation,
both during training and during inference, including attention. Furthermore,
elaborate on how subword units are used to reduce out-of-vocabulary problem and
sketch BPE algorithm for constructing fixed number of subword units.
Variational Autoencoders
Describe deep generative modelling using variational autoencoders – show VAE
architecture, devise training algorithm, write training loss, and propose sampling
procedure.
Generative Adversarial Networks
Describe deep generative modelling using generative adversarial networks -- show GAN
architecture and describe training procedure and training loss. Mention also
CGAN (conditional GAN) and sketch generator and discriminator architecture in a DCGAN.
Speech Synthesis
Describe the WaveNet network (what a dilated convolution and gated activations
are, how the residual block looks like, what the overall architecture is, and how
global and local conditioning work). Discuss parallelizability of training and
inference, show how Parallel WaveNet can speedup inference, and sketch how it is
trained.
Reinforcement learning
Describe the general reinforcement learning settings and describe the Monte
Carlo algorithm. Then, formulate the policy gradient theorem (proof not
needed), write down the REINFORCE algorithm, the REINFORCE with baseline
algorithm, and sketch now it can be used to design the NasNet.
Transformer
Describe Transformer architecture, namely the self-attention layer, multi-head
self-attention layer, and overall architecture of an encoder and a decoder.
Also discuss the positional embeddings.
Neural Turing Machines
Sketch an overall architecture of a Neural Turing Machine with an LSTM
controller, assuming $R$ reading heads and one write head. Describe the
addressing mechanism (content addressing and its combination with previous
weights, shifts, and sharpening), and reading and writing operations.