CSE 390 Special Topics in Computer Science -- Machine Learning
Date: Monday and Wednesday
Room: CHEMISTRY 123
Instructor: Luis Ortiz
Email: le [last name] at cs followed
by .sunysb and finally .edu
Phone: (631) 632-1805
Office Hours: 12:30pm-2:00pm, Monday and Wednesday; or by appointment (please e-mail me in advance)
Roughly speaking, machine learning techniques strive to
automatically acquire expertise to effectively perform a task of
interest by efficiently processing task-related information, such as a
(usually large) data set of examples, and successfully extracting and
generalizing knowledge embedded within the available information.
There is growing demand for computer scientists with proficiency in
For example, the advent of technology for the
collection of vast amounts of digital data, such as that generated by
an ever expanding population of Internet users, has increased interest
in the development of machine learning software applications. Machine-learning-based technology such as driver assistance and
voice-activated systems in cars, automatic system personalization and
adaptation to individual user preferences and behavior, speech-driven
phone systems for customer service, speech-to-text capabilities,
recommender systems and e-mail spam filtering, are now commonplace.
Machine learning application areas include marketing, e-commerce,
software systems, networking, telecommunications,
banking, finance, economics, social science, computer vision, speech
recognition, natural-language processing, and robotics. Some problems
addressed using machine learning techniques include pattern
recognition and classification, knowledge discovery and data mining,
anomaly detection, credit/loan approval, credit-card fraud,
quantitative trading, automatic categorization of very large collections, such
as web pages, documents and images, effective ranking of web search
results (e.g., Google's PageRank), face recognition, tracking, machine
translation, and more recently, intelligent, adaptive control of virtual player behavior in
video games, smart debugging of computer programs
and memory management in operating systems. There is also recent interest in
creating computationally tractable machine learning tools for
recognizing and predicting general trends in individual or group
behavior in large populations, such as spending behavior, adoption of
new products, technology or habits, sharing in peer-to-peer systems,
and predicting the development of online communities within large
social networks such as Facebook and MySpace.
This course provides an introduction to fundamental concepts and
modern techniques in machine learning, and prepare students for future
work in the area. The objective is to provide students with basic
knowledge and understanding of both the theory and practice of machine
learning, and to train students on the use and application of machine
learning ideas, paradigms and techniques.
The course format involves formal lectures, discussions and presentations, some led by the students themselves. Reading material will be taken from a variety of sources, including textbooks, tutorials and literature in the area, as appropriate. The following is a list of some likely sources for reading material; it will be adapted as the instructor see fit based on the final list of topics and the students' backgrounds and interests.
- Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, first edition, 2007.
- Richard O. Duda, Peter E. Hart and David G. Stork. Pattern Classification. Wiley-Interscience, second edition, 2001.
- Thomas Mitchell. Machine Learning. McGraw Hill Higher Education, first edition, 1997.
- Stuart Russell and Peter Norvig. Artificial Intelligence: A
Modern Approach. Prentice Hall, second edition, 2003. (Machine-learning related chapters.)
- Russell Davidson and James G. MacKinnon. Estimation and Inference in Econometrics. Oxford University Press, 1993.
Students complete a project on an
application of machine learning to a particular problem, which the students
choose in consultation with the instructor. The chosen course project
requires instructor's approval. The project should have an experimental component. Ideally, the project will address a new problem and produce a
novel application. Students make an oral presentation of their project
proposal. To monitor the project's development, students periodically
make oral presentations as the project progresses. Students give a
final presentation of their project by the end of the course term.
Students are evaluated on their performance on homework assignments (35%), their participation in class discussions (20%) and the quality of their project and respective presentations (45%).
Tentative List of Topics
NOTE: The list of topics, as well as the emphasis on each topic,
will likely vary depending on the background and interests of the course
participants, among other things.
- Machine-learning fundamentals: classification, regression and clustering; noisy, noise-free and incomplete data; supervised and unsupervised learning; hypothesis classes, model complexity, model selection, Ockham's razor and the bias-variance dilemma; dynamic environments, reinforcement learning and the exploration-exploitation dilemma
- Basic models and methods: nearest neighbors, decision trees, linear discrimination, neural networks, support vector machines (SVMs), boosting and bagging, naive Bayes classifiers, gradient-descent, Q-learning
- Advanced topics: expectation-maximization (EM), Hidden Markov Models (HMMs), K-means, mixture-of-Gaussians, principal component analysis (PCA), independent component analysis (ICA), econometrics and simultaneous equation models
- DHS Ch. 1
- RN Ch. 18 Sec. 1-2
- Mitchell Ch. 1
- Bishop Ch. 1, pp. 1-4
- Nearest Neighbors Classification
- Mitchell Ch. 8
- RN Ch. 20 Sec. 4
- Bishop pp. 126-127
- Decision Trees