Meta-analysis as a tool for assessing university-wide student learning outcomes

This chapter examines the efficacy of using meta-analysis techniques for the assessment of a university core course requirement that a university delivers through many courses, across several colleges of the university. Meta-analysis is a set of techniques for combining the results of many studies in order to understand the effects of a treatment as it is applied to an entire field. While this technique has been used has been used as a research literature review technique, we believe that meta-analysis procedures can bring both robust and orderly assessment of program effectiveness across courses that meeting a single learning objective, even though the courses themselves may come from divergent fields. Meta-analysis has the advantage of all assessments not needing to be the same because the researcher converts the results of each assessment to a common statistic, an effect size. In a program evaluation context, this means that professors can use their own embedded classroom assessments as their contribution to a university-wide assessment. An institutional researcher could also combine these for a general assessment of student learning. Additionally, because the embedded assessment is a natural part of the class, it is unobtrusive to instruction in the class and invisible to students. The researchers included 10 embedded assessments in the study for a mean gain effect size of .35. From the results of a pilot study, the authors learned that while the technique works, questions about the reliability and validity of the classroom assessments emerged. The authors speculate that institutional researchers can use these techniques to compare the assessment of standards within and among universities.