Syllabus
Probabilistic Models of
Human and Artificial Intelligence
CSCI 7222 / CSCI 4202
Spring 2007
Tu, Th 14:00-15:15
ECCR 108
Instructor
Course Objectives
The goal of the course is
to understand
the latest advances in probabilistic models in artificial intelligence
and cognitive science. The course participants are likely to
be a
diverse group of students and faculty, some with primarily an
engineering/CS focus and others primarily interested in cognitive
modeling (building computer simulation and mathematical models to
explain human perception, thought, and learning).
Probabilistic models have established a firm foothold in AI and ML over
the past decade, but only in the past few years have cognitive
scientists realized their potential for explaining human cognition.
One neat thing about probabilistic models is that they
straddle
the gap between AI/ML and cognitive science: The same
methodology
is useful for both understanding the brain and building intelligent
computer systems. Indeed, for many of the papers we'll
discuss,
the models contribute both to machine learning and to cognitive
science. Whether your primary interest is in engineering
applications of machine learning or in cognitive modeling, you'll see
that there's a lot of itnerplay between the two fields.
Prerequisites
The course is open to any
students who
have some background in cognitive science or artificial intelligence.
Some background in proababilty and statistics will be
helpful,
but iis not essential as long as you are willing to learn.
Course Requirements
Readings
In the style of graduate
seminars, your primary responsibility for the course will be to read
the series of papers before
class and be prepared to come into class to discuss the paper (asking
clarification questions, working through the math in the paper,
relating the paper to other readings, critiquing the paper, presenting
original ideas related to the paper).
Written Commentaries
For some of the readings,
I'll ask you to write a one-page commentary on the paper, The
commentary consists of
approximately one page of comments,
questions, or
critiques of the assigned reading(s) for that class. This page will be
due the
day of class, and can include one or more of the following:
- a summary of what you think the main or most interesting
ideas
are behind the reading(s).
- questions about the material for further discussion, either
clarification questions or points of disagreement with the authors (``I
don't see how such and such will work as the author claims...'').
- comments on how the assigned reading relates to other
course
readings, or, if you feel ambitious and want to track down some related
work in the field, how the assigned reading compares to this other
work.
- a critique of the work. What are the flaws in the ideas
presented? What are the limitations? Do the authors place their work in
the appropriate theoretical perspective? Do the authors overstate their
results? In what direction might the work be extended?
These commentaries are
intended to
promote careful thought about a
paper before
the session in
which it is discussed. The point is not
to give you more busy work, but rather to encourage you to jot down
notes and questions as you read the papers. They will not be accepted after
the
class in which the paper is discussed.
I
won't ask you to write commentaries on readings that are primarily
tutorial/review.
Presentation
You are required to
present a
share of the papers during the course of the semester. The
presentation is meant to be a summary of the paper and its main ideas.
Ideally, two class members will collaborate to do each
presentation, allowing you to work through the papers together.
I
expect grad students to do twice the presentations that undergrads do.
My guess is that grad students will do about 3 presentations
and
undergrads will do 1, but that depends on class size.
Research Project
All students enrolled in
the course
will implement and test a probabilistic model and will write up a
summary of their research for the end of the semester. For
grad
students, ideally the work you'll do will be useful in your own
research. For undergraduates, you should start by
implementing a
model that we will discuss during the semester, replicating the results
in the paper, and then going on to develop your own application of the
model.
Semester Grades
Grades will be based
roughly on the
following: project 20%,
oral
presentation
20%, class
discussions 10%, written commentary on papers 50%.
Class-By-Class Plan and Course Readings
Classes marked by (*) do not require commentaries.
| Date |
Activity |
Reading |
Lecture notes |
presenter |
| Jan
16 |
introduction (*) |
Mozer,
Jones, & Shettel (2007) |
download |
Mozer |
| Jan
18 |
introduction (*) |
Chater,
Tenenbaum, & Yuille (2006)
Tenenbaum (1999) |
download |
Mozer |
| Jan
23 |
a primer on
probabilistic inference (*) |
Griffiths &
Yuille (2006) |
download |
Mozer |
| Jan
25 |
Bayesian Ockam's razor (*) |
Jefferys & Berger (1991) |
download |
Mozer |
| Jan
30 |
inference in
graphical models (*)
|
Jordan
& Weiss (2002)
|
download |
Mozer |
| Feb 1 |
MCMC (*) |
Andrieu et al. (2003) |
download |
Mozer |
| Feb 6 |
learning in Bayes nets (*) |
Heckerman
(1995), pp. 1-24 |
download |
Mozer |
| Feb 8 |
learning in Bayes nets (*) |
Heckerman
(1995), pp. 25-57 |
download |
Mozer |
| Feb
13 |
topic model |
Griffiths & Steyvers (2002) |
download |
Mozer |
| Feb
15 |
attention |
Mozer, Shettel, & Vecera (2006) |
download |
Mozer |
| Feb
20 |
latent dirichlet allocation |
Blei, Ng, & Jordan (2002)
longer version available as Blei, Ng, & Jordan (2003) |
download |
Mozer |
| Feb
22 |
hierarchical topic model |
Blei, Griffiths, Jordan, & Tenenbaum (2004) |
|
Hadjar Homaie |
| Feb
27 |
integrating topics and syntax |
Griffiths, Steyvers, Blei, & Tenenbaum 2005 |
|
Noralie Sarver |
| Mar 1 |
NO CLASS
|
|
|
|
| Mar 6 |
topic and role discovery in social networks |
McCallum, Corrado-Emmanuel, & Wang 2005 |
download |
Mozer |
| Mar 8 |
motion illusions as optimal percepts |
Weiss,
Simoncelli, & Adelson (2002) |
|
Ted Fisher |
| Mar
13 |
vision as Bayesian inference |
Yuille
& Kersten (2006) |
|
Kristopher Nuttycombe |
| Mar
15 |
Gaussian Processes (*) |
Williams (1997) |
download |
Mozer |
| Mar
20 |
Gaussian Processes (Computing with Infinite Networks) |
Williams (1997b)
|
|
Sam Reid |
| Mar
22 |
Probabilistic computation in spiking populations |
Zemel, Huys, Natarajan, & Dayan (2005) |
|
Jason
Boorn |
| Mar
27, 29 |
SPRING BREAK |
|
|
|
| Apr 3 |
Hidden Markov models (*) |
Gharamani (2001) |
|
Michael Otte, Scott Richardson |
| Apr 5 |
Bayesian online changepoint detection |
Adams & MacKay (unpublished) |
|
Brian Loughry |
| Apr 10 |
Prediction and change detection |
Steyvers & Brown (2006) |
|
Michele Samorani |
| Apr 12 |
Hierarchical Dirichlet processes (and hierarchical beta processes) |
Teh, Jordan, Beal, & Blei (2003) |
|
Abhishek Jaiantilal |
| Apr
17 |
Theory-based causal inference |
Tenenbaum & Griffiths (2003) |
|
Chris DiHoust |
| Apr 19 |
Learning domain structures |
Kemp, Perfors, & Tenenbaum (2004) |
|
Mark Lewis-Prazen |
| Apr
24 |
Context sensitive induction |
Shafto et al. (2005) |
|
Kai Ching |
| Apr
26 |
Infinite relational models |
Kemp, Tenenbaum, Griffiths, Yamada, Ueda (2006) |
|
Richard Bell |
| May 1 |
Combining causal and similarity based reasoning |
Kemp, Shafto, Berke, & Tenenbaum (2007) |
|
Andrew Boehm |
| May 3 |
Sensorimotor control |
Koerding, Tenenbaum, & Shadmehr (2007)
|
|
Ben Pearre, Matt Wilder |
| May 5,
10:30 a.m. - 1 p.m. |
FINAL EXAM SLOT |
Not planning to hold class during exam period |
|
|
Queue
The queue is a list of
papers that we haven't yet scheduled but that I hope to cover during
the semester
LANGUAGE
PERCEPTION
visual adaptation as optimal information transmission Wainwright
MOTOR CONTROL
REASONING/INFERENCE
CATEGORY LEARNING
Discovering multiple structures that capture diffrent subsets of features Shafto et al.
Structure learning Kemp & Tenenbaum
OTHER
Interesting Links