Task-Level Variation in Elementary Students' Scientific Literacy Performance

Abstract

Scientific literacy is commonly reported as an overall score or as scores for individual competencies. However, this form of reporting may obscure differences in students' performance across tasks designed to measure the same competency. This study aimed to analyze primary school students' scientific literacy performance at the item level and identify patterns of variation across and within three scientific literacy competencies. A descriptive-exploratory design was employed, involving 402 students from three primary schools. The participants completed 10 dichotomously scored items assessing their ability to explain phenomena scientifically, interpret data and evidence scientifically, and evaluate and design scientific inquiry. The data were analyzed at the overall, competency, and item levels using score distributions and proportions of correct responses. This analysis was followed by a conceptual examination of the demands of each task. The results showed that the proportions of correct responses across the three competencies were 64.0%, 74.7%, and 72.6%, respectively. Greater variation was found at the item level, where the proportion of correct responses ranged from 32.4% to 89.7%. Substantial variation was also observed among items assessing the same competency. For example, within the competency of explaining phenomena scientifically, the proportion of correct responses ranged from 32.4% to 89.7% across items. These findings indicate that competency-level scores may not fully capture variations in students' performance across individual tasks. Therefore, scientific literacy assessments should consider not only the competencies being measured but also the characteristics and demands of the tasks used to assess those competencies.