Modeling Student Strategy Usage

with Mixed Membership Models

April Galyardt

Educational Psychology and Instructional Technology

University of Georgia

Athens, GA 30602 galyardt@uga.edu

The strategies that students use to complete a task or solve a problem are critical indicators of their developing expertise. Experts know more strategies and use different strategies than novices, but growth toward effective expert strategies is not straightforward. Individuals use multiple strategies, and will switch strategies between subsequent tasks. As children learn, the mixture of strategies changes; they use efficient strategies more frequently, but will still fall back on the more rudimentary strategies (Pellegrino et al., 2001). In order to estimate a student’s expertise, we need to estimate the mixture of strategies that each student uses.

The majority of current psychometric models do not model strategies at all. A few models, such as mixture IRT models, allow for the existence of different strategies, but assume that each student uses a single strategy consistently (Junker, 1999; Pellegrino et al., 2001). We need to develop assessment models that not only allow for the existence of multiple strategies, but allow for students to switch strategies.

Mixed membership models, which have recently become popular in the machine learning community, address this problem. Latent Dirichlet allocation (Blei et al., 2003) and the admixture model (Pritchard et al., 2000) are the two most common versions of the larger class of mixed membership models (Erosheva, 2002; Erosheva et al., 2004).

The basic idea of mixed membership is that each individual can have membership in multiple classes. For example, topic models, such as latent Dirichlet allocation, allow documents to be about multiple topics. The admixture model was built to describe populations where individuals may have genetic heritage from multiple sub-populations. In the context of modeling student knowledge, each class represents a possible solution strategy. Students then have membership in each class according to how much they use that strategy.

The mixed membership model framework also allows us to combine different types of observed student data. For each student i who attempts task j, we observe a feature vector Xij . Features may include the student’s response, the response time, eye tracking information, or any other observed aspect of behavior. The joint distribution of the features Xij depends on the strategy that the student uses to solve the problem.

This mixed membership strategy model allows creates model of individual student knowledge that is consistent with current cognitive psychology research indicating that strategy choice is a critical component of expertise.