CSCI-5800: Machine Learning

Cross-listed course (ugrad+grad), Lawrence Street Center - 840, 2018

Welcome to Machine learning course. What is machine learning? It is concerned with the question of how to write computer programs that automatically improve with experience. The field has expanded so much in every direction of our daily lives that we mostly are unaware of its presence. Interestingly, there may be two perspectives to this oblivion. First one is that the machine learning applications have greatly improved themselves over time and became part of your lifestyle. However, a completely opposite perspective is that we became part of it, and unknowingly feeding input to the program as if we are in a computer simulation? In this class, you will learn about machine learning, its application in diverse domains, get to implement them yourself and raise awareness of its presence and associated implications in our day-to-day lives. Once again welcome aboard.

The course is based on fundamental knowledge of computer science principles and techniques, probability and statistics, calculus, and the theory and application of linear algebra. The course provides a broad introduction to pattern recognition from given data and how it can relate to machine learning. Topics include: supervised learning, unsupervised learning, semi-supervised learning, neural network / deep learning, and reinforcement learning algorithms. The course will also discuss recent applications of these machine learning concepts, such as in robotics, data mining, autonomous navigation, speech recognition, bioinformatics, meteorology, and so on.

Course objectives

By the end of the course you are expected to gain the following skills:

  1. Develop an understanding on how to extract patterns from data.
  2. Develop an understanding on a wide variety of machine learning algorithms – how the algorithms work, and their practical usages.
  3. Understand different types of optimization techniques which are heavily utilized in many of the learning algorithms.
  4. Capable of discussing pros and cons of the learning algorithms.
  5. Be able to implement the covered algorithms in class by themselves using any of the programming language of their choice from the set {Matlab, Python, R}.
  6. Apply the algorithms to solve real world problems.


For undergraduates:

  1. MATH-3195 (Linear algebra and differential equations)
  2. CSCI-3412 (Algorithms).

For graduate students:

  1. The graduate standing.

Topics covered

  1. Introduction to Machine learning
  2. Linear regression
  3. Overfitting and complexity; training; validation; testing
  4. Logistic regression
  5. Classification – Bayesian classifier
  6. Classification – kNN
  7. Binary vs. Multi-class classification
  8. Classification - Decision tree; Ensemble methods: Bagging, random forests, boosting
  9. Classification – LDA, QDA
  10. Clustering - kMeans
  11. Clustering – hierarchical clustering
  12. Dimensionality reduction: Fisher’s LDA
  13. Dimensionality reduction: PCA, SVD
  14. Dimensionality reduction: Kernel PCA
  15. Classification – Linear SVM
  16. Classification – Kernel SVM;
    1. Kernel methods
    2. Support Vector regression
  17. Neural networks
  18. Neural networks / Deep learning (CNN)
  19. Recommender systems: Nonnegative Matrix Factorization (NMF)
  20. Recommender systems: IMC, RIMC, collaborative filtering
  21. Reinforcement learning
  22. Time series data analysis; Hidden Markov Model
  23. Markov Random Field vs Conditional Random Field
  24. Semi-supervised learning
  25. How to scale your algorithms for the big data analytics?
  26. Spectral clustering