Probabilistic Models of
Human and Machine Intelligence

CSCI 7222
Assignments 8 and 9

Assignment 8 Due: 11/22/2013
Assignment 9 Due: 12/18/2013

Goal

The goal of Assignments 8 and 9 is to apply your probabilistic modeling skills to an interesting problem domain. Either you will propose a domain of your own, or I'll suggest a problem to work on.

Option 1: Propose your own domain

For assignment 8, I'd like you to specify a domain or problem you'd like to tackle using probabilistic models, and to propose a model you'll use to do this exploration. Hopefully you can find a problem from your own research area that is amenable to a probabilistic modeling approach.  I'd like you to do more than run an existing model (e.g., topic model). Hopefully you will customize models to your particular problem domain.  I'd also like to see you write your own code to implement a model and not use prepackaged software.

Option 2: Predicting student performance

I obtained data from a group of children playing a battleship number line game.  Try it out (with 'whole numbers' and 'fractions'). I will give you data from the first portion of their play and will ask you to predict the last portion of their play.  At the end of the semester, we'll have a competition and you will each submit your predictions and I'll evaluate them.  I'll spend some time in class talking about this data set so that you understand the excel spreadsheet with the data.

What to hand in for Assignment 8

You are to write up a page or two which includes:

  1. a description of the domain [include this even if you are using the battleship data, so that I am confident that you understand the game]
  2. a description of the data set you have to experiment with, in detail including: number of data points and a complete list of features that you will include in your model
  3. A description of your modeling approach.  If you are using the battleship data, you may want to compare several models; if so, describe each model.  If you are using your own data, you must compare at least two models in order to evaluate the relative benefits of the different approaches. (You could propose one model that is simpler and one that is more complex; one model that is hierarchical and one that isn't; etc.)
  4. details of the model, e.g., the generative model assumptions
  5. your methodology for experimentation:  how will you set hyperparameters? what sort of evaluation will you use (e.g., a single split of the data into training and testing components, vs. cross validation)

I want you to provide enough information that I understand the specific approach you're going to take.  I will try to give you rapid feedback to help guide you in your implementation and evaluation.  Please submit to me via email. 

What to hand in for Assignment 9

Expand on your write up for Assignment 8 and hand in a complete report on your simulations.  In addition to the points above, include a summary of your results, with figures and tables as appropriate, and some conclusions.  I would like this to be as close as you can come to writing a conference paper describing novel research.  Please submit to me via email.