Welcome to Machine learning course. What is machine learning? It is concerned with the question of how to write computer programs that automatically improve with experience. The field has expanded so much in every direction of our daily lives that we mostly are unaware of its presence. Interestingly, there may be two perspectives to this oblivion. First one is that the machine learning applications have greatly improved themselves over time and became part of your lifestyle. However, a completely opposite perspective is that we became part of it, and unknowingly feeding input to the program as if we are in a computer simulation? In this class, you will learn about machine learning, its application in diverse domains, get to implement them yourself and raise awareness of its presence and associated implications in our day-to-day lives. Once again welcome aboard.

The course is based on fundamental knowledge of computer science principles and techniques, probability and statistics, calculus, and the theory and application of linear algebra. The course provides a broad introduction to pattern recognition from given data and how it can relate to machine learning. Topics include: supervised learning, unsupervised learning, semi-supervised learning, neural network / deep learning, and reinforcement learning algorithms. The course will also discuss recent applications of these machine learning concepts, such as in robotics, data mining, autonomous navigation, speech recognition, bioinformatics, meteorology, and so on.

Course objectives

By the end of the course you are expected to gain the following skills:

1. Develop an understanding on how to extract patterns from data.
2. Develop an understanding on a wide variety of machine learning algorithms – how the algorithms work, and their practical usages.
3. Understand different types of optimization techniques which are heavily utilized in many of the learning algorithms.
4. Capable of discussing pros and cons of the learning algorithms.
5. Be able to implement the covered algorithms in class by themselves using any of the programming language of their choice from the set {Matlab, Python, R}.
6. Apply the algorithms to solve real world problems.

Prerequisites

For undergraduates:

1. MATH-3195 (Linear algebra and differential equations)
2. CSCI-3412 (Algorithms).

For graduate students:

1. The graduate standing.

Topics covered

1. Introduction to Machine learning
2. Linear regression
3. Overfitting and complexity; training; validation; testing
4. Logistic regression
5. Classification – Bayesian classifier
6. Classification – kNN
7. Binary vs. Multi-class classification
8. Classification - Decision tree; Ensemble methods: Bagging, random forests, boosting
9. Classification – LDA, QDA
10. Clustering - kMeans
11. Clustering – hierarchical clustering
12. Dimensionality reduction: Fisher’s LDA
13. Dimensionality reduction: PCA, SVD
14. Dimensionality reduction: Kernel PCA
15. Classification – Linear SVM
16. Classification – Kernel SVM;
    1. Kernel methods
    2. Support Vector regression
17. Neural networks
18. Neural networks / Deep learning (CNN)
19. Recommender systems: Nonnegative Matrix Factorization (NMF)
20. Recommender systems: IMC, RIMC, collaborative filtering
21. Reinforcement learning
22. Time series data analysis; Hidden Markov Model
23. Markov Random Field vs Conditional Random Field
24. Semi-supervised learning
25. How to scale your algorithms for the big data analytics?
26. Spectral clustering