Impact study on early trial versions of Coaching eBooks

Objective of the study

Oxford University Press wanted to find out the benefits and challenges of using prototype (early trial version) Coaching eBooks in order to inform the pre-publication development of this aspect of the Oxford Reading Buddy digital reading service.

Download full impact study

Impact study evaluation findings on early trial versions of the Oxford Reading Buddy Coaching eBooks

[97 page PDF - 2.5MB]

Methodology

Researchers from the School of Education at the University of Southampton conducted a Randomized Controlled Trial following three schools, 322 students and 12 teachers for one term (Summer 2018). The six Year 1 and six Year 5 classrooms were randomly allocated into either the ‘experimental group’, who would use the prototype Coaching eBooks, or the ‘control group’, who would not.

Sources of evidence

  • Researcher-administered questionnaires that assessed students’ reading attitudes before and after using the prototype Coaching eBooks
  • Interviews with teachers
  • Classroom observations
  • eBook usage data supplied by the system

Resources used in the impact study

Seven prototype Coaching eBooks were specially created for the study. The reading level of the books was the assumed ‘average’ Oxford Level for the year group and the prototype contained a small number of the essential features of the proposed Coaching eBooks such as ‘Reading Buddy prompts’ and ‘Reading Buddy questions’, quizzes, feedback, audio and data-reporting.

Findings

Zoom in
Zoom in

Benefits identified by teachers

The Reading Buddy questions helped students reflect on their reading and engage in higher-order thinking skills.

“Reading Buddy questions had the additional: ‘What about this? Have you thought about…?’ I have noticed some children engaging in that and talking it through with themselves.”

School 2, Year 1 teacher

The Coaching eBooks helped engage students with reading by integrating it with ICT.

“Their engagement has been really high with respect to the prototype Coaching eBooks."        

School 2, Year 5 teacher

Students themselves saw the value of the Reading Buddy questions.

“What the children found useful were the questions. So, it is the comprehension-type questions. It is always trying to find a fresh approach to get the children to answer questions about the text. There are lots of approaches to reading and to draw the correct information from the text. Actually, Reading Buddy is supportive as a tool for teachers.”

School 2, Year 5 teacher



“The children will get engaged with higher-order thinking questions. But I think that the benefits would show higher up in the school. If they start using it in Year 1, then they will have it in their system.”

School 2, Year 1 teacher

The Coaching eBooks were enjoyable and therefore helped foster a love of reading.

“The children liked it. They absolutely loved all the stories. They were really upset when I didn’t have time, or the laptops broke down and that happened twice. They were really disappointed. They were engaged ...with the stories.”

School 3, Year 1 teacher

Potential benefits identified

Teachers felt that the Coaching eBooks could help bridge the gap between reading at home and reading at school.

It was thought that the comprehension quiz questions could be useful preparation for National Assessments.

Some teachers commented that a service like this could do two jobs at once: engage students in reading and inform teachers about progress against learning objectives.

“For some pupils, to access it at home will be a massive benefit.”

School 2, Year 1 teacher

Challenges and impact on product development

Challenges identified   Impact on pre-publication development of Oxford Reading Buddy

Many teachers felt that insufficient school ICT equipment prevented them from using the prototype to its full capacity.

Time constraints restricted teachers’ ability to fully integrate and exploit the capabilities of the prototype. Focus on implementing the prototype reduced the number of other effective literacy practices taking place.

Oxford Reading Buddy will provide clear guidance on recommended usage to ensure that schools understand how all aspects of the service can be integrated to ease teacher workload and increase the impact of learning time.

Some Year 1 students found it difficult to access the prototype Coaching eBooks due to forgetting usernames, passwords and not yet possessing sufficient skills in ICT.

Oxford Reading Buddy is much simpler than the prototype and further refinements have been made as a result of the study to all aspects of the student interface to further improve usability for young learners.

A limited capability for the seven prototype Coaching eBooks to address the interests and needs of all students in terms of reading level and motivation.

The finished Oxford Reading Buddy offers a large range and variety of books for students to choose from.

Whereas only one Oxford Level per year group of prototype Coaching eBooks was offered for the study, the finished service matches students with a choice of books at their precise reading level.

Oxford Reading Buddy offers a badge-based reward system to motivate and recognize good reading behaviours and achievements. Enhancements have been made as a result of the study.

Conclusion

The prototype Coaching eBooks were found to provide a number of significant benefits to teachers and students. Most notably, usage of the eBooks was associated with significantly more positive attitudes to reading, particularly in Year 5. The Reading Buddy questions and quizzes were perceived to encourage deeper reflection on reading and to support comprehension. This pre-publication study also revealed challenges with implementation, which has allowed Oxford University Press to address these challenges in the finished Oxford Reading Buddy service.