The Impact of Technology on Formative Assessment Instruments

The Impact of Technology on Formative Assessment Instruments Evaluating Learners’ Speaking Skills

Student Name:

Student Number

Date:

Bibliography

Article no. 1: Estrada-Araoz, E. G., Sayed, B. T., Niyazova, G. G., & Lami, D. (2023). Comparing the effects of computerized formative assessment vs. computerized dynamic assessment on developing EFL learners’ reading motivation, reading self-concept, autonomy, and self-regulation. Language Testing in Asia, 13(1), 39.

The purpose of the study is to investigate the impact of technology on formative assessment instruments evaluating learners for speaking skills, and this study suits the research topic best. According to (Estrada-Araoz et al. 2023), the effects of computerized dynamic assessment (CDA) and computerized formative assessment (CFA) on reading self-concept, self-motivation, self-regulation, and autonomy among students are investigated in this study. Around 87 Bangladeshi students from a total of 123 Bangladeshi students were selected for the study based on their Preliminary English Test (PET) scores. Students were categorized into three equal groups; two experimental groups were formative and dynamic, and one was a control group comprising 29 students each, using convenience sampling. All three groups completed pre-tests to assess their reading self-concept, autonomy, self-regulation, and motivation before getting different treatments. The first experimental group goes CDA, and the second experimental group goes with CFA. However, the control group experienced traditional assessment. All groups post-test measure the same variables after the treatment. ANOVA test and post hoc Scheffe test allow for the analysis of data, which shows significant differences among the three groups in post-test results. In terms of reading self-concept, autonomy, self-regulation, and reading motivation, the CDA group showed greater performance than the CFA group. The findings highlighted that EFL students significantly enhanced their reading motivation, autonomy, reading self-concept, and self-regulation. A discussion of the implications and limitations of the study, along with the future recommendations, concluded the study.

Article no. 2: Faber, J. M., & Visscher, A. J. (2018). The effects of a digital formative assessment tool on spelling achievement: Results of a randomized experiment. Computers & education, 122, 1-8.

Similarly, another study conducted by Faber et al. (2018) was found relevant to the research topic. A randomized clinical trial was utilized to evaluate the spelling achievement of eight to nine-year-old third-grade students for the evaluation of the impact of the digital formative assessment tool. The sample comprised 39 control schools (n=986) and 30 experimental schools (n=619). The control group employed regular materials and spelling instruction, while the experimental group continued with digital formative assessment tools. Pre and post-test scores of standard achievements, percentage of adaptive assignments, and the tasks completed by a total number of the students were necessary for the data collection process. The outcomes of the study indicated that digital formative assessment tools had a moderate influence on spelling achievement. Several crucial areas for future research were highlighted for further exploration.

Article no. 3: Granberg, C., Palm, T., & Palmberg, B. (2021). A case study of formative assessment practice and the effects on students' self-regulated learning. Studies in Educational Evaluation, 68, 100955

Another study conducted by Granberg et al. (2021) showed relativity with the selected topic. The research study explores the use of formative assessment on students' self-regulated learning (SRL), which one mathematics teacher carries out. The students participating in the study were about 13-14 years of age and belonged to various social and cultural diversities. The two control groups have 18 and 19 students, respectively, whereas the intervention class has 18 students. Nine students from each control group, along with fourteen students from the intervention class, have shown interest and agreed to participate in the study. An impressive suitable SLR effect on students' motivational beliefs was observed in contrast to the two control classes. Exponential improvement in students' SLR behaviour in the classroom was seen through qualitative data. The integration of multiple features of formative assessment was demonstrated by the analysis of teachers' formative assessment applications, which gave factual results on how efficient formative assessment can be arranged with development models of SLR and enhance SLR efficacy.

Article no. 4: Hagos, T., & Andargie, D. (2023). Effects of Technology-Integrated Formative Assessment on Students' Conceptual and Procedural Knowledge in Chemical Equilibrium. Journal of Education and Learning (EduLearn), 17(1), 113-126.

This study was conducted by Hagos, T., & Andargie, D. (2023) and focuses on exploring the student's knowledge and concepts about chemical equilibrium after introducing a technology-integrated formative assessment application. A qualitative method was integrated with the Quasi-experimental pre and post-test approach in this study. About 132 eleventh-grade students from public secondary schools are involved in the study, categorized into three groups: one comparative and two experimental. For the treatment groups and one for the comparative group, random sampling was adopted for the selection of intact classes. A chemical equilibrium conceptual test, class observations, and a procedural test were utilized for the data collection. Inferential statistics like one-way ANOVA MANOVA and Pearson product-moment correlation, along with descriptive statistics like standard and mean deviation, play an essential role in the analysis of the data.

In contrast to traditional methods and formative assessment strategies alone, technology-integrated formative assessment techniques impressively enhance students’ knowledge and concepts for apprehension of chemical equilibrium. The student seems more motivated to learn, and the teacher maintains higher excellence through technology-integrated formative assessment strategies that are acknowledged through classroom observations. Chemical equilibrium understanding among students was significantly improved through these techniques as compared to other approaches.

Article 5: Zheng, C., Wang, L., & Chai, C. S. (2023). Self-assessment first or peer-assessment first: effects of video-based formative practice on learners’ English public speaking anxiety and performance. Computer Assisted Language Learning, 36(4), 806-839.

The purpose of this study is to propose a particular sequence of formative assessment practice in computer-assisted language learning (CALL). Less attention should be paid to the effects of different levels of formative assessment strategies on the cognitive and affective aspects of students' public speaking performance. In this mixed-method study, data were collected from multiple sources to examine and describe the impact of two approaches of integrating video-based formative assessment into the teaching and learning of public speaking among English language learners. There were two intact classes in a 16-week English public speaking course, with 25 students in the self-assessment-initiated (SAI) group and 26 students in the peer-assessment-initiated (PAI) group. After two different sequences of formative practice, both groups were expected to deliver three formal English public speeches. Quantitative and qualitative data analysis revealed that learners in the SAI group had lower public speaking anxiety than the learners in the PAI group, and the learners in the PAI group had better public speaking performance.

Furthermore, the study revealed that SAI formative practice assisted the learners in focusing on their delivery and anxiety in public speaking. In contrast, PAI formative practice may have assisted the learners in the correct usage of the English language. This research thus showed that the order of using the different modalities of assessment was relevant. With the use of video technology and CALL tools, various sequences in getting other types of feedback may influence learners' public speaking anxiety and performance. This study offers more specific insights into the processes of using video technology in formative practice to reduce learners' public speaking anxiety and enhance their performance.

Article no. 6: Song, Y., & Sparks, J. R. (2019). Building a game-enhanced formative assessment to gather evidence about middle school students’ argumentation skills. Educational Technology Research and Development, 67, 1175-1196.

The purpose of this paper is to design and assess a new formative assessment to collect information on middle school students' argumentation abilities. More specifically, the game-enhanced scenario-based assessment used in this study (Seaball—Semester at Sea) consists of a series of argumentation exercises within an elaborate scenario in which students discuss the question of whether junk food should be made available to students. These activities were developed based on argumentation learning progressions (i.e., conjectures about the developmental stages that students move through as they become more elaborate in constructing arguments) used to define the skills, levels, and sequences of the activities. Feedback on performance is also given in the assessment. Researchers carried out a pilot study to assess the performance of students and the effectiveness of the tasks in measuring argumentation skills. Over one hundred middle school students participated in this assessment and other measures of argumentation in a pre/post design. Summary statistics of student achievement in the activities, item indices, and interrelationships are presented. Findings showed that total scores of students were significantly associated with measures of argumentation skills and with students' reading and writing performance on state assessments.

Furthermore, the students obtained higher average scores in a post-test of argumentation skills after they participated in the Seaball activities. Last, the provision of feedback about students' task performance was observed to be helpful for students who scored "Below" or "Approaching" at the proficient level on the state reading and writing test. In the last section, researchers discuss the implications of the current study for the assessment of argumentation and argumentative learning.

Article no. 7: Wang, P. (2015). The effect of dynamic assessment on the listening skills of lower-intermediate EFL learners in Chinese Technical College: A pilot study. Journal of Language Teaching & Research, 6(6).

The purpose of this study is to find out if dynamic assessment can help integrate listening comprehension assessment and instruction and improve students' listening skills. The participants of the study were five second-year English majors from a technical college in an underdeveloped area of a coastal province in China. The assessment used the cake format, where participants were first presented with an audio clip and required to respond to questions and explain how they understood the material. The researcher then stepped in to facilitate the task. The audio material was replayed to the participants, and they were asked to narrate what they had heard. This went on until the listener was able to get the relevant information from the audio material. From the notes taken by the researcher, notes taken by the students, verbal reports by the students, and reflective reports, it was realized that dynamic assessment helps both the researcher and the participants to understand the problems in listening better. The data also showed that the researcher's intervention and mediation aimed at participants' issues supported the building of the mediated learning experience for the participants. Therefore, the integration of listening comprehension assessment and instruction is possible through dynamic assessment and the development of students' listening study at macro and micro levels.

Article no.8:  White, L. (2019). The Impact of Technology on the Implementation of Formative Assessment Lessons in Eighth-Grade Math Classrooms

The purpose of this study was to determine whether there is a statistical difference between the post-lesson assessment results of students who participated in a formative assessment lesson in which the teacher used technology and those of students who participated in a formative assessment lesson in which the teacher did not use technology. The teachers gave the students pre- and post-lesson assessments after the formative assessment lesson enactments. The whole class introduction and discussion were delivered to the control groups by the teacher using technology. The entire class introduction and whole class discussion were done in this experiment without the teacher utilising any form of technology to teach the lesson. This study also sought to determine teacher and student attitudes toward the use of technology in mathematics teaching and learning. The teachers were asked to share their thoughts regarding any consequences stemming from the use of technology during the formative assessment lesson. The students were required to fill out a survey to state their opinions on the incorporation of technology in a formative assessment lesson. The mixed quantitative results also needed to provide a clearer picture of whether technology use in formative assessment lessons affected student performance on post-lesson assessments. The findings of this research may provide doubts about the need to spend money on technology purchases. From the findings of this study, it can be concluded that the integration of technology in a formative assessment lesson does not affect student learning.

Article no. 9: Köroğlu, Z. Ç. (2021). Using digital formative assessment to evaluate EFL learners’ English speaking skills. GIST–Education and Learning Research Journal, (22), 103-123.

A study was conducted by Köroğlu, Z. Ç. (2021) to evaluate English speaking skills. For the evaluation of English speaking skills among 52 upper-intermediate EFL learners, this study compared the traditional speaking tests and digital formative assessments using a mixed-method approach. Qualitative data collection was performed through interviews, and quantitative data was collected through achievement tests. Study findings indicated that participants acknowledged the close integration, preparation time, and refined materials, and their performance increased on digital formative tests. Apart from positive outcomes, some technical complications disturbed the participants which occurred during the digital formative tests. 

Article no.10: Karay, Y., Reiss, B., & Schauber, S. K. (2020). Progress testing anytime and anywhere–Does a mobile-learning approach enhance the utility of a large-scale formative assessment tool?. Medical Teacher, 42(10), 1154-1162.

Another study was performed by Karay et al. (2020) to explore the impact of mobile learning effectiveness among students. The study includes the total number of students (n=1701) from 2-10 semesters divided into two groups. This study evaluated the impact of mobile-formative progress tests on students' responses, validity, and creditability. Students received tests and were allowed to select their desired place and time for performing the test. Data was gathered consisting of test scores, semester performance, student evaluations, and time spent on the test. After the analysis of the findings, it was found that the results did not show a significant difference in the test results among groups. However, groups using the mobile phone often utilized external sources and took more time. The mobile test format was highly favored by the students due to their motivation and acceptance of this approach, but the performance stayed unaffected. 

Article no. 11: Zainuddin, Z., Shujahat, M., Haruna, H., & Chu, S. K. W. (2020). The role of gamified e-quizzes on student learning and engagement: An interactive gamification solution for a formative assessment system. Computers & education, 145, 103729.

This study is performed by Zainuddin et al. (2020), highlighting student engagement and learning enhancement through gamified e-quizzes. The differences in students learning and engagement during science class along with the comparison of three intervention groups utilizing two types of pedagogical methods including gamified instruction with e-quizzes as formative assessments and traditional instruction with paper-based quizzes are explored in this research study. Three applications were employed through gamified instructions; iSpring Leam LMS, socrative, and quizizz. The effects Formative quizzes, personal interviews, and post-questionnaire surveys were used to measure the effects of instructional interventions (n=94). The findings highlight the effectiveness of both paper-based quizzes and gamified e-quizzes in the learning performance of students. For engaging students and enhancing the learning environment in the classroom, incorporating gamified elements as innovative tools plays a vital role in student academic progress.

Article no. 12: McCallum, S., & Milner, M. M. (2021). The effectiveness of formative assessment: student views and staff reflections. Assessment & Evaluation in Higher Education, 46(1), 1-16.

This study was conducted by McCallum & Milner, (2021) which explores the effectiveness of formative e-assessments in first-year courses and focuses on evaluating their efficacy depending upon the staff reflections and students' feedback. Participation of students takes place in these assessments and data gathered through questionnaires have highlighted the effectiveness of e-assessment in students learning and understanding, encouraged them to pursue further studies, and helped them in tracking their progress. The results helped the faculty staff to focus on the benefits of using formative e-assessments to enhance student involvement and allow early development of strategies. The study highlights the advantages of integrating formative assessments in first-year courses.

Article no. 13: Elmahdi, I., Al-Hattami, A., & Fawzi, H. (2018). Using Technology for Formative Assessment to Improve Students' Learning. Turkish Online Journal of Educational Technology-TOJET, 17(2), 182-188.

This paper focuses on the role of the formative assessment technology known as the Plickers, a classroom response system that is a tool used in classroom learning. In this study, a descriptive mixed-method research design was employed. Data was collected from one hundred and sixty-six students from Bahrain Teachers College, University of Bahrain, through the use of a developed self-administered questionnaire. The study established that students hold formative assessment and getting feedback as valuable and this was in line with the application of Plickers. Furthermore, the study revealed that Plickers for formative assessment assists in learning with the following benefits: an increase in students' engagement, time-saving, equal opportunities for all the learners, and fun appeals to learners. The conclusions also call upon the teachers to incorporate technology tools like Plickers in class to assist them in determining the utility of this tool in evaluating their performance as well as the learning achievements of their learners.

Article no. 14: Vásquez, A., Nussbaum, M., Sciarresi, E., Martínez, T., Barahona, C., & Strasser, K. (2017). The impact of the technology used in formative assessment: The case of spelling. Journal of educational computing research, 54(8), 1142-1167.

The purpose of this study is to show how the particular technology addressed to support formative assessment in spelling influences learning outcomes. Observational assessment is an example of a formative assessment that is a set of student-centered practices that do not necessarily yield the best outcomes. Moreover, various technologies are more capable when it comes to performing specific tasks than when it comes to others. The study employed a Design-Based Research method and was conducted in Chile in two phases. The first part of the study identifies a formative assessment approach to teach spelling. In a later step, it becomes imperative to examine the effects of varying technologies on this strategy. This is achieved by comparing two different technologies such as Tablet PCs and the interpersonal computer, The findings of this study show that the Self-Paced Formative Assessment using Tablets is a more effective form of instruction than using an Interpersonal Computer for teaching spelling to the primary school students. This therefore explains how technology affects learning when one has to embrace a formative assessment strategy.

Article no 15: Dalby, D., & Swan, M. (2019). Using digital technology to enhance formative assessment in mathematics classrooms. British journal of educational technology, 50(2), 832-845.

This study examines the contexts within which six mathematics teachers and their classes utilize iPads in formative assessments in two secondary comprehensive schools located in the Midlands of England. In the first instance, a design research methodology is adopted to create and test six lessons, each of which incorporates iPads in formative assessment practices in different ways. Researchers then identified during lesson observation, video analysis, and interviews with the teachers how this technology supports these processes, the roles iPad plays as well as the division of roles for the main actors that include the teacher, technology, and the student. An analytical model is created that describes these interactions and illustrates how the technology both serves communicative purposes and also functionalities that are more actively involved which change the part assumed by the teacher in certain formative processes. The study provides an understanding of how iPad technology can support students' learning through assessment and presents an analytic view that may benefit the following research.