Probabilistic
Models of
Human and Machine Intelligence
CSCI
7222
Assignments 8 and 9
Assignment
8 Due: 11/22/2013
Assignment 9 Due: 12/18/2013
Goal
The
goal of Assignments 8 and 9 is to apply your probabilistic modeling
skills to an
interesting problem domain. Either you will propose a domain of your
own, or I'll suggest a problem to work on.
Option 1: Propose your own
domain
For
assignment 8, I'd like you to specify a domain or problem you'd like to
tackle using probabilistic models, and to propose a model you'll use to
do this exploration. Hopefully you can find a problem from your own
research area that is amenable to a probabilistic modeling approach.
I'd like you to do more than run an existing model (e.g., topic
model). Hopefully you will customize models to your particular problem
domain. I'd also like to see you write your own code to implement
a model and not use prepackaged software.
Option 2: Predicting student
performance
I obtained data from a group of children playing a
battleship number line game.
Try it out
(with 'whole numbers' and 'fractions'). I will give you data from the
first portion of their play and will ask you to predict the last
portion of their play. At the end of the semester, we'll have a
competition and you will each submit your predictions and I'll evaluate
them. I'll spend some time in class talking about this data set
so that you understand the
excel
spreadsheet with the data.
What to hand in for Assignment
8
You are to write up a page or two which
includes:
- a description of the domain [include this even if you are using
the battleship data, so that I am confident that you understand the
game]
- a description of the
data set you have to experiment with, in
detail including: number of data points and a complete list of features
that you will include in your model
- A description of your modeling approach. If you are using
the battleship data, you may want to compare several models; if so,
describe each model. If you are using your own data, you must compare at least two models in
order to evaluate the relative benefits of the different approaches.
(You
could propose one model that is simpler and
one that is more complex; one model that is hierarchical and one that
isn't; etc.)
- details of the model,
e.g., the generative model assumptions
- your methodology for
experimentation: how will you set
hyperparameters? what sort of evaluation will you use (e.g., a single
split of the data into training and testing components, vs. cross
validation)
I want you to provide enough information that I understand the specific
approach you're going to take. I will try to give you rapid
feedback to help guide you in your implementation and evaluation.
Please submit to me via email.
What to hand in for Assignment 9
Expand on your write up for Assignment 8 and hand
in a complete report on your simulations. In addition to the
points above, include a summary of your results, with figures and
tables as appropriate, and some conclusions. I would like this to
be as close as you can come to writing a conference paper describing
novel research. Please submit to
me via email.