A2i: Assessment to Instruction Design Study
Watch Dr. Carol Connor provide a webinar overview of the science and instructional keys to individualizing instruction to improve children’s literacy skills.
Our project is pursuing an iterative design, development, and testing process in two phases, based upon the principles of design-based research that are used in the Learning Sciences field. (Fishman, 2004; Richey & Klein, 2007). An overarching goal of design-based research is to develop interventions that are responsive to practical needs and that draw from and contribute back to theory (Brown, 1992). Work conducted in this manner, which is akin to engineering design (Collins, 1992; Kelly, 2003) is more likely to lead to innovations that are usable in real-world contexts.
There is strong evidence that working together with school personnel early in the design process helps to reduce problematic or wrong assumptions about classrooms that can lead to designs that work so long as the developers are there to support them, but not on their own and not at large scale (Fishman, Marx, Blumenfeld, Krajcik, & Soloway, 2004). We will also apply the methods of usability evaluation to our work (Nielsen, 1993; Nielsen & Mack, 1994; Rubin, 1994; Squires & Preece, 1999), which allow the developer and researcher to determine that the components of the intervention are operating as intended and that users understand how elements operate. Participatory design is a technique that blends design-based research methods and usability research.
One of the most cogent descriptions of design-based research is by Cobb, Confrey, diSessa, Lehrer, and Schauble (2003), who argue that such research is rooted in “small-t theories”, as opposed to “capital-T Theories” (such as Constructivism). The “small-t theory” guiding our work is: the greater the fidelity with which teachers are able to match individualized instruction for students to recommendations from A2i, the greater will be student literacy learning. In conducting design-based research, the procedure involves iterating between design and testing in order to continually refine the intervention towards the goal established by the theory. A number of sub-theories may also be at play with respect to individual design elements, guiding designers with respect to how particular elements of the intervention function with respect to the goal. For instance, we expect that the planning tools will be instrumental in guiding decisions about individualization, and that in turn there are specific elements within the classroom view (grouping tools, sorting tools, etc.) that we intend as helpful. In design-based research, we would expect that both our design instances and our “small-t theories” about how those designs function would evolve over time, informed both by usability and outcome data generated through our iterative design experiments with the A2i system.
Although it shows potential as an effective way to support teachers’ implementation of effective differentiated reading instruction, in its current form it is not feasible for educational leaders (e.g., literacy coaches, principals, district administrators) and teachers to use A2i without substantial amounts of researcher support. Therefore, the current design and development study aims to develop a usable and scalable version of A2i for use by literacy teachers and educational leaders.
Learn more about A2i by reading this article or visiting Learning Ovations.
Learn more about A2i’s Integrated Online Assessments through the menu bar or clicking here.
References
Al Otaiba, S., Carol M. Connor, Jessica S. Folsom, L. Greulich, J. Meadows and Z. Li (2011). “Assessment Data–Informed Guidance to Individualize Kindergarten Reading Instruction: Findings from a Cluster-Randomized Control Field Trial.” The Elementary School Journal 111(4): 535-560.
Bos, C., N. Mather, R. F. Narr and N. Babur (1999). “Interactive, collaborative professional development in early literacy instruction: Supporting the balancing act.” Learning Disabilities Research and Practice 14(4): 227-238.
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141-178.
Chall, J. S. (1967). Learning to read: The great debate. New York, McGraw-Hill Book Co.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher,, 32(1), 9-13.
Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology (pp. 15-22). Berlin: Springer-Verlag.
Connor, C. M. (2011). Child by Instruction interactions: Language and literacy connections. Handbook on early literacy. S. B. Neuman and D. K. Dickinson. New York, Guilford<strong “mso-bidi-font-weight:=”” normal”=”” style=”color: #868686;”>: 256-275.
Connor, C. M. (2012). Intervening to support reading comprehension development with diverse learners.Unraveling the Behavioral, Neurobiological and Genetic Components of Reading Comprehension: The Dyslexia Foundation and NICHD. B. Miller and L. E. Cutting. Estonia: in press.
Connor, C. M., B. Fishman, E. Crowe, P. Underwood, C. Schatschneider and F. J. Morrison (in press). Third grade teachers’ use of Assessment to Instruction (A2i) software and students’ reading comprehension gains.In press, Technology for literacy achievements for children at risk. O. Korat and A. Shamir. NY, Springer.
Connor, C. M., S. R. Goldman and B. Fishman (in press). Reading and writing technology. Handbook of Research on Educational Communications and Technology. M. Spector, D. Merrill and M. J. Bishop, Association for Educational Communications and Technology.
Connor, C. M. and F. J. Morrison (2012). Knowledge Acquisition in the Classroom: Literacy and Content Area Knowledge Knowledge Development in Early Childhood: How Young Children Build Knowledge and Why It Matters. A. M. Pinkham, T. Kaefer and S. B. Neuman. New York, Guilford Press in press: 220-241.
Connor, C. M., F. J. Morrison, B. Fishman, E. C. Crowe, S. Al Otaiba and C. Schatschneider (in press). “A Longitudinal Cluster-Randomized Control Study on the Accumulating Effects of Individualized Literacy Instruction on Students’ Reading from 1st through 3rd Grade.” Psychological Science.
Connor, C. M., F. J. Morrison, B. Fishman, S. Giuliani, M. Luck, P. Underwood, A. Bayraktar, E. C. Crowe and C. Schatschneider (2011). “Classroom instruction, child X instruction interactions and the impact of differentiating student instruction on third graders’ reading comprehension.” Reading Research Quarterly46(3): 189-221.
Connor, C. M., F. J. Morrison, B. Fishman, C. C. Ponitz, S. Glasney, P. Underwood, S. Piasta, E. Crowe and C. Schatschneider (2009). “The ISI classroom observation system: Examining the literacy instruction provided to individual students.” Educational Researcher 38(2): 85-99.
Connor, C. M., F. J. Morrison, B. J. Fishman, C. Schatschneider and P. Underwood (2007). “THE EARLY YEARS: Algorithm-guided individualized reading instruction.” Science <strong “mso-bidi-font-weight:=”” normal”=”” style=”color: #868686;”>315(5811): 464-465.
Connor, C. M., F. J. Morrison and E. L. Katch (2004). “Beyond the reading wars: The effect of classroom instruction by child interactions on early reading.” Scientific Studies of Reading 8(4): 305-336.
Connor, C. M., F. J. Morrison, C. Schatschneider, J. Toste, E. G. Lundblom, E. Crowe and B. Fishman (2011). “Effective classroom instruction: Implications of child characteristic by instruction interactions on first graders’ word reading achievement.” Journal for Research on Educational Effectiveness 4(3): 173-207.
Connor, C. M., S. B. Piasta, B. Fishman, S. Glasney, C. Schatschneider, E. Crowe, P. Underwood and F. J. Morrison (2009). “Individualizing student instruction precisely: Effects of child by instruction interactions on first graders’ literacy development.” Child Development 80(1): 77-100.
Fishman, B., Marx, R., Blumenfeld, P., Krajcik, J. S., & Soloway, E. (2004). Creating a framework for research on systemic technology innovations. The Journal of the Learning Sciences, 13(1), 43-76.
Kelly, A. E. (2003). Research as design. Educational Researcher, 32(1), 3-4.
Nielsen, J. (1993). Usability engineering. San Francisco: Morgan Kaufmann.
Nielsen, J., & Mack, R. L. (Eds.). (1994). Usability inspection methods. New York: John Wiley & Sons.
Rubin, J. (1994). Handbook of usability testing. New York: Wiley.
Squires, D., & Preece, J. (1999). Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers, 11, 467-483.