Item Response Modeling:: Applications to Large-Scale Assessment of Academic Achievement

Seminar | November 3 | 2-4 p.m. | 2515 Tolman Hall

Speaker/Performer: Xiaohui Zheng, UC Berkeley

Sponsor: Berkeley Evaluation

The call for standards-based reform and educational accountability has led to increased attention to large-scale assessments. Over the past two decades, large-scale assessments have been providing policymakers and educators with timely information about student learning and achievement to facilitate their decisions regarding schools, teachers and students. For large-scale assessments, the outcomes are far from straightforward. There have been great concerns about generalizing and interpreting assessment results, due to a rather large number of observed and unobserved variables co-existing and interacting at different levels of the assessment system. A wide variety of advanced techniques are available for the analysis of large-scale assessments, many of which do not fully capture the complex nature of the data.

This research explores multilevel item response modeling and its application to largescale assessments. Building on the Multidimensional Random Coefficients Multinomial Logit (MRCML) framework and the Generalized Linear Latent and Mixed Model (GLLAMM) framework, three forms of multilevel item response models are presented in three separate studies. Each study addresses a specific measurement issue. The first study proposes a Latent Growth Item Response Model (LG-IRM) for the analysis of longitudinal assessment data. A growth model is incorporated into the item response function from a multidimensional perspective. Instead of using ability scores, the LGIRM provides a direct representation of item responses in the longitudinal model. The second study discusses Multilevel Structural Equation Models (MSEMs) for the relationship between latent variables. MSEMs that combine measurement models with
multilevel regression models are used to explore the effect of the school-level latent variable on the student-level latent variable. The third study investigates between-school Differential Item Functioning (DIF) as well as Differential Facet Functioning (DFF). School-to-school variability is examined in terms of differential functioning of items for students in different schools. An explanatory DFF model that includes covariates is also formulated to predict school effects on the DFF.

Event Contact: dlsmith@berkeley.edu, 510-642-7984