Final Exam Logistics

Date and Time:

• The Final Exam will be held during the official exam on Friday April 8 at 7:30pm in M3 1006.
• It will be scheduled and run by the campus exam operations, so there will be strict protocols for entry and exit as well as notes and materials.
• Make sure you know where the room is before April 8!

Final Exam Scope

Scope: Content from live lectures and self-study videos from the entire course, but focussed on the new concepts learned in *weeks 9 - 12

Topics: High level knowledge about topics from the entire course, including all Deep Learning topics, see detailed breakdown below.

Final Exam”Cheat-Sheet” and Exam Materials

As for the midterm, you can bring three sheets of paper (US Letter or A4), double sided, with hand written notes on it of any kind. I say handwritten because when I’ve allowed it to be printed in the past, many people end up just getting a shared document from their peers, and the best thing about a “cheat sheet” is how much you remember and learn by making it yourself. Studies show that you learn much better from notes written by hand under space or time constraints than electronic reference documents. You also bring cheat-sheet again, so a total of six pages.

• Pen and pencil (a mechanical pencil is suggested for multiple choice questions so you can erase it if needed).
• An eraser :)
• Calculator - for arithmetic calculations
• Your midterm cheatsheet, up to three pages of notes for first half of course
• You final exam cheatsheet, up to three additional pages of notes
• Valid UWaterloo student ID (WatCard)

Final Exam Question Topics

Topics from before the midterm

Questions may require high level knowledge about topics from the entire course. This is most likely for topics which relate to training a Deep Learning model (eg. Cross-validation, Train/Validate/Test, Error measures, Distance metrics,…)

For all topics you should:

• Know definitions of methods and algorithms.
• Know relative strengths and weaknesses between them, when they apply.
• Perform small calculations of some core algorithms, formulas and metrics.
• Visually represent the expected behaviour of some algorithms on presented data.
• Be familiar with the overall flow of Data Analysis from data preprocessing, feature analysis, algorithm choice and application, experimental design and validation.

Neural Network Fundamentals

• Relation to logistic regression
• Network Design Considerations (width, depth)
• Input, hidden, output units
• Activation functions
• Comparison to other supervised learning algorithms
• Types of Deep Learning: Fully Connected, CNN, RNN, Autoencoders

Effective Deep Learning Training Methods:

• Problems with NNs - Vanishing Gradients Problem, Catastrophic Forgetting, Overfitting, etc.
• Regularization methods - L1/L2 penalties, dropout, early stopping, data augmentation
• Model capacity learning curves
• Backpropagation algorithm
• Understanding core concepts about gradient descent
• Benefits of various Gradient Optimizers
• Practical Methodology for building Deep Learning systems

For CNNs

• Benefits, pros/cons, general design

• Analysis of impact of a given filter

• Identify difference between convolution, deep, sparsely connected structures

• Pooling, downsampling, strides, padding

For Autoencoders

• standard autoencoders
• denoising autoencoders

For the Following Complex Methods

The following architectures or methods were introduced and talked about at a high level. So questions could relate to them, but would be limited to

• understanding purpose and usage of these architecture/methods
• questions would not require detailed implementation or analysis

List of Complex Methods:

• Inception, Resnet, Densenet
• LSTM, LRCN
• Recurrent Neural Networks
• Variational Autoencoders
• ~BERT~

Final Exam Question Types:

• True or False [1 mark each]- these questions merely have a true or false answer to choose
• Multiple Choice [2-4 marks each] - all questions have one correct answer only, some will include “all of the above” as an option. If “all of the above” is the correct answer, then check one other answer will receive a zero for that question.
• Data/Output/Algorithm Matching [2-4 marks each]
  • Given a visualization of a dataset and the resulting output of an algorithm: identify the algorithm that was used
  • Distinguish the parameters of that algorithm from a set of choices
• MCx2 [4 marks each] - Some multiple choice questions have a followup, such as how to improve the situation from the first part.
• CNN Calculation Questions (see below)
• DNN Architecture Questions (see below)
• Other Potential Topics for Calculations and Formula questions:
  • Write down, or identify from a list, the correct mathematical form for a given Neural Network in terms of:
    • the full neural network discriminant function for a given design or
    • the back propagation update or
    • the gradient for some component of the network or
    • the loss function

CNN Calculation Questions [5 points each]

Actual setup text for questions:

“Consider the following convolutional neural network and answer the questions below: The network takes input images of size (?x?) with ? colour channels. The first convolutional layer uses filters of size ?, a stride of ?, and zero-padding of width ?.”

• What will be the dimensions of the resulting activation maps of this first layer?”
  • Choose from options in the form (? x ? x ?)
  • “Consider the following convolutional neural network and answer the questions below:
• The 4x4 layer is provided as input to a pooling layer as described in each question, which output would have the correct values in each situation?”
  • You are given a 4x4 matrix of integers as the input to the pooling calculation
  • “Which of the following outputs would be correct if we were using max pooling, with a ?x? pooling filter and a stride of ? ?”
  • “Which of the following outputs would be correct if we were using average pooling, with a ?x? pooling filter and a stride of ? ?”
  • Choices are presented as 2x2 matricies of numbers to choose from.

DNN Architecture Question

“Which of the following formulations is the correct one for the output of the following Neural Network architecture:

• This network takes a flat vector of numeric input \(X\) , using three fully connected hidden layers where:
• The first hidden layer has ? nodes, the second hidden layer has ? nodes, and the third hidden layer has ? nodes.
• The first and second hidden layers use ? as the activation function. “

Definitions:

• \(x_i\) : input data values
• \(f_A(x)\) : activation function for type \(A\) where:
  • \(A\in\{R=RELU, M=Softmax, S=Sigmoid, E=ELU, G=Gaussian\}\).
• \(w^{(c)}_{ab}\) : hidden weights for layer \(c\), connecting incoming feature \(b\) from the previous layer to the hidden unit \(a\).
• \(a*b\) : normal, pairwise multiplication of number \(a\) and \(b\), used for reading clarity.

Which net activation output if the correct one? (multiple formulae or the following form will be presented to choose from)

• \(net_m = f_E\left( \sum_{k=1}^{5} w^{(3)}_{mk} * f_E\left( \sum_{j=1}^{10} w^{(2)}_{kj} * f_E\left( \sum_{i=1}^{15} w^{(1)}_{ji} * x_i + w^{(1)}_{j0} \right) + w^{(2)}_{k0} \right) + w^{(3)}_{m0} \right)\)​

Midterm Exam

Some more information on the midterm will be up soon on LEARN with sample questions, details about format and details about notes.

The midterm will be held, as planned, on March 1, during class time and in-person.

• If you have accessibility concerns, there is an official procedure for that through AccessAbiltiy Services (https://uwaterloo.ca/accessability-services/).
• If you cannot attend the midterm in-person, you need to talk to Prof. Crowley about it to explain and make other arrangements.
• Note that the Final Exam will be in-person, and you are expected to attend it in person. Existing university policies around missed exams will apply.
• The Final Exam will be held during the official exam on Friday April 8 at 7:30pm in M3 1006.
• It will be scheduled and run by the campus exam operations, so there will be strict protocols for entry and exit as well as notes and materials.
• Make sure you know where the room is before April 8!

Midterm Topics

At a high level, the topics of midterm will cover any content from Course Weeks 1 - 6 from Data Understanding and Preprocessing up to Decision Trees and Ensemble Methods.

Midterm “Cheat-Sheet”

For the midterm you can bring one sheet a paper (US Letter or A4), double sided, with hand written notes on it of any kind. I say handwritten because when I’ve allowed it to be printed in the past, many people end up just getting a shared document from their peers, and the best thing about a “cheat sheet” is how much you remember and learn by making it yourself. Studies show that you learn much better from notes written by hand under space or time constraints than electronic reference documents. When the Final Exam comes, you will be allowed to bring your midterm cheat-sheet and at least one additional page of notes.