This module instructs students on the basics of deep learning as well as building better and faster deep network classifiers for sensor data. The module is strongly project-based, with two main phases. In the first phase, students will learn the basics of deep learning and Computer Vision, e.g. stochastic gradient descent, multi-layer perceptron, convolutional neural networks, filtering, and corner detection. At the end of this first phase, students should be ready to run simple networks in TensorFlow and implement basic computer vision methods in Python. In the second phase, students will be divided into teams of 2 or 3. Each team will tackle a problem of their choosing, from fields such as computer vision, pattern recognition, distributed computing. Example projects include face recognition and emotion recognition.

• Instructor: Prof. Shih-Chii Liu
• Language of Instruction: English
• Prerequisites Basic programming (e.g. Python, C++), Basics in Linear Algebra
• TA: Yuhuang Hu, Iulia-Alexandra Lungu
• Contacts: yuhuang.hu@ini.uzh.ch, iulialexandra@ini.uzh.ch

Schedule

• Session: Thursdays 1-5pm
• Starting date: February 28th, 2019
• Place: Institute of Neuroinformatics, Dungeon Lab, Y35 E 30, University Zürich Irchel Campus
• Module ends: May 30th, 2019

Session 01

Basics of Linear Algebra (vector, matrix, tensors, etc), Introduction to Python, numpy basics, symbolic computation basics.

Session 02

Brief introduction to Linear Regression, Logistic Regression, Stochastic Gradient Descent and its variants.

Session 03

Brief introduction to Multi-layer Perceptron and Convolutional Neural Networks.

Session 04

Basic techniques of Computer Vision using OpenCV, such as thresholding, edge detection, etc.

Session 05

Advanced techniques of Computer Vision such as filtering, corner detection, keypoints, etc.

Technical Tutorials and Resources

Git Crash Course

Python for Scientific Computing

Introduction to Colaboratory

Extra Deep Learning Resources

Projects

• Baby Jarvis: Implement a face recognition system using Keras, OpenCV, and Raspberry Pi. The device will voice the name of the face it sees. (Difficulty: 5)

• Baby Jarvis II: Distinguish between happy and sad faces using Keras, OpenCV and Raspberry Pi. (Difficulty: 5)

• Baby Jarvis III: Proof-of-concept driver monitoring system by detecting if the user is sleeping, checking the phone or doing other reckless activities while driving. (Difficulty: 6)

• Are You Human?: Break a CAPTCHA that uses simple math as the question. (Difficulty: 6)

• Which station is this?: Take a picture of the station name, recognize the station and find the position from Google Maps! (Difficulty: 7)

• Implementing backpropagation: Implement the backpropagation method in basic python without relying on any high level APIs like Keras or Tensorflow. (Difficulty: 7)

• (Advanced) Paper Piano: Print the piano key on a paper, use CV techniques to identify which key is pressed and play the sound. (Difficulty: 8)

• (Advanced) Raspberry that talks: Analyse an image via image captioning, then broadcast the information to a speaker. (Difficulty: 8)

• (Advanced) Object Tracking: Develop a model-free object tracker for VOT Challenge! (Difficulty: 9)

• (Advanced) Real-world Image Classification: Develop a classifier for TinyImageNet using state-of-the-art Deep Networks and run it as fast as you can on Raspberry Pi! (Difficulty: 9)

• Or… your ideas!

Remark: Difficulty scale: 1 (so easy) – 10 (way too hard)

Remark: First-come-First-serve basis in selecting project.

Timeline

• Week 5 (March 28th): Project Briefing

• Week 6 (April 4th): Assign project based on the team’s decision. Assign team name based on the team’s decision. Setup project, first group discussion on the plan to complete the project.

• Week 7-9 (April 11th - May 2rd): Collecting data, writing utility functions, first prototype by end of the 9th week. No class on April 25th

• Week 10 (May 9th, May 16th): Refine the model and implementation so that the system can run as fast as possible

• Week 11 (May 16th): Project presentation

• Week 12-13 (May 23th, May 30th): Project write up

Acknowledgment

This module uses many online resources on Deep Learning and Raspberry Pi. We would like to acknowledge the following resources:

• Deep Learning
• Unsupervised Feature Learning and Deep Learning Tutorial
• cs231n: Convolutional Neural Networks for Visual Recognition
• RPI-Wireless-Hotspot
• OpenCV Python Tutorials

Previous editions

• PnS2018: the first edition of the module focused on Deep Learning.