Don’t shoot! understanding students’ experiences of video-based learning and assessment in the arts

Abstract

This paper explores the benefits and challenges of using video blogging to support reflection and assessment in acting and performance training. A video platform called Acclaim was integrated into the university’s virtual learning environment (Blackboard), enabling students to record and share their performances in an online setting. A key feature of the platform was the ability to add time-based comments on a video, making it possible for tutors and students to discuss and critique specific moments of each performance. Students were also required to record and upload regular video blogs during which they would reflect on their progress. At the end of the first year, a survey was designed to evaluate students’ experiences of the video blogging activity. The findings indicate mixed results: while many students viewed video blogging as a useful learning activity, they also identified a number of challenges that hampered its effectiveness. From this study we conclude that while video blogging presents several notable advantages for the facilitation of performance-based courses, the technology needs to be carefully introduced as part of a broader instructional strategy to maximise the potential benefits for student learning and engagement.

Introduction

The generation of students now entering university have grown up in a world in which digital video is a widely used medium of communication and self-expression (Oblinger and Oblinger 2005). The combination of cheap, high-quality video recording equipment and video-sharing platforms such as YouTube have made user-generated video content a cornerstone of the social media revolution (Kearney and Shuck 2006). Higher Education has not been left untouched by the digital video phenomenon. A growing body of research has emerged around the use of video to extend learning online through instructional approaches such as delivering lectures (Choi and Johnson 2005; Zhang et al. 2006), supporting assessment (Yoo et al. 2009; Schwartz and Hartman 2014) and around more innovative practices such as “flipping the classroom” (Tucker 2012; Berrett 2012; Herreid and Schiller 2013; Bishop and Verleger 2013). However, despite the ease with which comments can be added to videos on platforms such as YouTube and Vimeo, Kearney and Shuck (2006) report a lack of research around how digital video annotation tools can help students reflect on their work and learning. In their study, the authors also raise the question of whether such feedback tools could lead students to scrutinise the authenticity of their learning. Our research aims to explore this gap by investigating the effectiveness of video, and specifically learner-generated videos, in supporting authentic, reflective learning.

A university in the south of the United Kingdom recently introduced a new performing arts course into its portfolio. As much of the assessable work generated by students would consist of performance pieces, a key focus of the course’s teaching and learning strategy was the need to capture and critique these performances in a suitable environment. Common video platforms such as YouTube and Vimeo were considered but discounted as they did not permit comments to be added at specific moments in the performance, and also presented Quality Assurance issues regarding secure storage of student work. A tool called Acclaim was ultimately chosen as it allowed both time-specific comments and secure integration with the University’s Blackboard Virtual Learning Environment. Figure 1 below demonstrates how the Acclaim tool enabled tutors to add time-based comments on students’ work.

Fig. 1

Demonstrating Acclaim. Please click the following link to view the video clip: https://www.youtube.com/watch?v=5ToGiW-8aqo

A further aim of the video-commenting tool was to facilitate interaction between students by enabling them to add formative comments on each other’s videos. The aim of this case study was to investigate the effectiveness of learner-generated video in the form of “video blogging” both as a tool for supporting individual reflection and as a way to enhance student interaction and discussion about their work.

The use of learner-generated video in higher education

Although the effectiveness of video as medium of instruction has been reviewed by a number of publications (see for example Zhang et al. 2006; Choi and Johnson 2005; Yousef et al. 2014), this literature review focuses on studies that examine the pedagogical benefits and challenges of learner-generated videos. By learner-generated videos, we aim to differentiate between videos that were made by teachers for students with the purpose of instruction and those produced by students for teachers with the purpose of assessment. The emphasis on this shift to learner-generated content is significant, as a 2012 report commissioned by Cisco Systems sees learner-generated video as a “powerful tool in the hands of students” (Greenberg and Zanetis 2012, p. 35) and suggests that its use will increase throughout the 21st century. The use of learner-generated video to provide students with opportunities to reflect and provide peer feedback can be understood as supporting a culture of assessment for learning (AfL) (Bloxham and Carver, 2014). In particular, the use of learner-generated video as a pedagogical strategy constitutes a move towards the active learning that Bloxham and Carver highlight as fundamental to such an approach. While it is beyond the scope of this article to undertake a full review of the literature on AfL, learner-generated video blogs can offer valuable opportunities to help students engage in assessment feedback activities of self-regulation and self-judgment as described by Boud and Lawson (2011).

While learner-generated video activities are nascent in many disciplines, they have frequently been employed in teacher education. Studies have reported the benefits of video production as an educational resource in pre-service teacher training (see for example McDrury and Alterio 2001; Tendero 2006). Girod et al. (2007) draw parallels between the phases in video production and the classroom teaching process, so as to emphasise the ways in which video-making enables teachers to reflect on their own practice. A study by Tripp and Rich (2012a) found that some teachers would only recognise the need to improve their teaching practice after watching and reflecting on videos of themselves in action (see also Tripp and Rich (2012b) for an extended review of literature that describes how the process of reflection is facilitated by video analysis). Studies have also shown that using video in teacher education supports the development of empathy and a sense of professional identity (Koc 2011).

But while video has proved to be a valuable tool for supporting reflection in teacher education, surprisingly little research has been undertaken into its pedagogical value as a tool for supporting reflection in other disciplines (Kearney and Shuck 2006). The use of learner-generated video to reflect on practice has been reported in a range of disciplines, including nurse education (Marita et al. 1999; Pereira et al. 2014), anatomical sciences (Doubleday and Wille 2014), mathematics (Borko et al. 2008), surgery (Vaughn et al. 2016) and sports education (Walters et al. 2015). Taylor (2013) has proposed that “vlogging [video blogging] is an ideal medium for producing knowledge and sharing digital stories” precisely because it allows an informal channel where students can discuss and reflect on their own work. But while many of these studies report that students enjoyed and engaged with such video assessments, there were also questions around how to implement teaching strategies and align production exercises with specific learning outcomes. Blomberg et al. (2014) argue that while the use of video remains popular in teacher education, its effectiveness and impact is often motivated by the learning design and educational rationale. In their comparative study, the authors conceptualise video “as a tool to foster learning and not as a pedagogy itself” (p. 458) and recommend embedding video-based learning with a specific instructional strategy to maximise the benefits to learning. In Fig. 2 below, the course leader explains how she intended the comment-based nature of Acclaim to support the development of a more participatory culture of assessment for the course.

Fig. 2

Developing a participatory culture of assessment. Please click the following link to view the video clip: https://www.youtube.com/watch?v=D0L_19uoFg0

While the potential of digital video to enhance what Schön (1983) refers to as ‘reflection on action’ is under-researched, even less is known about the extent to which the ability to record, review and reflect on a performance can help students learn the critical analysis skills that are a key aspect of their development as global citizens (Johnson and Morris 2010; de Andreotti 2014). These skills have been variously identified as report writing, critical thinking, researching, and as oral and written literacies (Ohler 2006). Moreover, the iterative nature of video blogging and its focus on documenting and reflecting on progress means that it can be understood as supporting a pedagogy of experiential learning (Kolb 1984).

Perhaps the closest study to our own was one that examined the role of video based learning in dance education (Leijen et al. 2009). This study used a video annotation tool (DiViDu) to enable students to review and evaluate their own performance through both self-reflection and peer feedback. The authors suggest that reflection is integral to the “professional identity” (ibid, p. 169) of their discipline, because a reflective process enables students and practitioners to accentuate the awareness of their own body, gauge its movement in space and develop necessary physical skills. Following an analysis of semi-structured interviews, the authors concluded that there were pedagogical benefits to the use of DiViDu as a platform to describe and evaluate students’ experience, and relate this experience to multiple perspectives through peer reviewing.

The personal nature of video blogging also suggests that video the potential to support aspects of authentic learning (Donovan et al., 1999). Kearney and Shuck (2006) have presented the learner-generated video production process as a mode of authentic learning, arguing that digital video projects subvert traditional assessment tasks by displacing the initiative to the student. Importantly, their research found that students were clearly motivated by the fact that their work was to be viewed and evaluated by their peers, which also facilitated the critical analysis of their own work.

Theoretical framework and methodology

Drawing from two studies on teacher education, Kearney (2011) proposes a formal pedagogical framework for learner-generated digital storytelling, which can be viewed as a set of guiding principles towards structuring curricula based around learner-generated video activities (for more on the history of digital storytelling as a learning and teaching tool, see Robin 2008). In this framework, Kearney outlines the various stages of video production and post-production, and demonstrates ways in which video-based peer-learning strategies can positively affect comprehension. Arguing that video production and digital storytelling constitute a “valuable, transformative tool for learners in a range of curriculum and discipline contexts” (p. 184), Kearney suggests that platforms for disseminating student videos “need careful consideration to maximise peer learning opportunities” (p. 182). Although not all the features of Kearney’s proposed learning design were applicable to the context of the current study, the following elements of the framework provided a suitable theoretical lens through which to analyse the data:

• “Modelling of exemplary digital stories from relevant contexts,

• Emphasis on support of students’ affective domain, especially for reflection purposes,

• Sharing of perspectives in a ‘mini-conference’ session,

• Mediation of class-based and online discussions stimulated by students’ digital stories,

• More opportunities for students to review and change their work as necessary after teacher-facilitated class discussions and peer feedback” (Kearney 2011, p. 181)

It is important to note that Kearney’s framework was not used to inform the design of the learner-generated video activity in the current case study. As such, the findings resulting from the data analysis can be viewed as an independent test of the above aspects of the framework.

A case study design was deemed to be appropriate as the research aimed to investigate the specific, contemporary phenomenon of video blogging within a situated context at the university (Yin 2013). While the situated nature of a case study design limits the generalisability of the findings, it is hoped that clearly aligning the research with findings from similar studies will permit an evaluation within the growing body of knowledge around the phenomenon of learner-generated video as a tool for learning and reflection.

The researchers’ initial intention was to undertake a qualitative investigation by using a phenomenographic methodology to explore students’ experiences of video blogging. However, the difficulty in arranging interviews resulted in the decision to use a short survey tool to obtain mainly quantitative data. The survey constituted of sixteen questions with multiple choice answers on a Likert scale, with some questions including a space that invited students to elaborate on their response using text. The survey concluded with four open ended questions that invited students to state what they found most useful and most challenging about video blogging, and to suggest improvements (see Table 1 for all questions and the mean values of all answers). More than two thirds (n = 31) of students responded to the survey out of a total of forty-one, giving a response rate of seventy-six percent and providing a comprehensive sample. Questions were wide ranging but focused on three areas:

Table 1 Survey questions and the respective mean values

• the habits of students (the frequency in which they used the platform and responses they gave to other videos; questions 1, 10, 11, 13 and 15),

• their individual level of confidence (whether they felt comfortable talking to the camera or have their work viewed by others; questions 2, 4 and 12), and

• whether they valued video blogging as a significant aspect of their learning process. To explore the latter question, we asked students if creating video blogs improved their acting performance (question 5), whether they reviewed older videos (question 6), whether this helped them understand their learning progress (question 7), and whether the whole process made their subject easier to understand (question 8). We also asked students if they thought video blogging made them more motivated about their subject (question 9).

The open-ended questions at the end of the survey aimed to find out what students found both useful and challenging about video blogging. Students were asked whether the use of video blogs could be improved, and lastly to provide three words that would best summarise their experience of video blogging. These open-ended questions are categorised according to content and are summarised in the next section.

The analysis of the quantitative data involved calculating mean averages, medians and standard deviations of all quantitative questions (questions 2 to 16). These calculations helped identify the strongest opinions in the group as well as the range in which opinions differed. Although the researchers acknowledge that it can be problematic to derive clear meaning from averaging Likert scale responses, it was felt that the specific questions being asked made it possible to infer useful meaning from the averages of each question.

Findings & discussion

Our findings indicate students did not perceive video blogging to be as beneficial to learning as we had initially presumed, with crucial questions testing the validity of the project returning scores ranging between 2.42 and 2.60 as indicated in Table 1 below. Students’ attitudes towards video blogging as an assessment exercise were mixed; while some students found video blogging as fun and entertaining or a way of developing their acting skills, others found it to be a time wasting and uncomfortable activity. Table 1 below indicates the mean values for each question.

The strongest opinion was on the ease-of-use of Acclaim as a platform for video blogging. While the survey demonstrated the highest mean value (3.95) and a relatively small deviation (1.00) in opinion, we think the survey is limited in having follow up questions about video annotation tools in particular, and newer technologies in general. Indeed, the second strongest opinion in the survey was that students were very comfortable in working with new technologies (question 3, mean 3.71). Students who scored 2 and lower on question 3 reported lower levels of satisfaction with the video blogging exercises, suggesting that the more comfortable the students were with new technologies, the more positively they responded to video blogging activities.

Performing in front of a camera was a new experience for many of the students, and a central aspect of the course was to introduce students to the experience of having their work critiqued from multiple points of view. Students reported mixed feelings around whether video blogging helped them feel more confident in talking to a camera, and there was a small increase from how students were comfortable with talking to a camera at the beginning of year to the end (an average increase of 0.61, from 2.84 to 3.45 in responses to questions 2 and 4). However, the most frequent (median) change between these questions was 0. Four students showed a decrease in confidence, and of these one reported “having someone watch my blog” as the most challenging aspect of video blogging.

The data also indicated that the use of video blogging did not enhance the interaction between students on the course, as the lowest average value across all questions was question 15 which asked if students posted comments at others’ videos. This question was of particular importance to the course team because the Acclaim tool was chosen precisely because of its ability to add comments at specific times in the videos, and it was hoped that this ability would enhance the interaction between students and tutors. In Fig. 3 below, the course leader explains the rationale behind her aim of using Acclaim to generate a constructive feedback community.

Fig. 3

Supporting peer feedback. Please click the following link to view the video clip: https://www.youtube.com/watch?v=HnDCUVKNSd8

However, the data indicated that students rarely used this Acclaim to provide feedback to their peers, and their reasons for not doing so were wide ranging. For example, almost a quarter of respondents (n = 7) indicated that they did not feel comfortable writing about other students’ work online, or simply did not find providing online comments to be relevant or appropriate. Four students reported not having the time to add comments, while five did not know what to say or how to comment, suggesting that further support on giving constructive feedback is required. One student reported that there was little encouragement to do so and two others emphasised that they did not comment because nobody else did. Only two students found commenting online useful and reported a good amount of commenting.

The data revealed that students rarely took the time to re-watch the videos they themselves had uploaded (an average score of 1.81 for question 6). Indeed, the data indicated that video blogging did not motivate students to learn more about their subject (question 9, mean average 2.16) and was instead perceived rather as just another task to be completed. This finding was strengthened by the responses to the question regarding whether video blogging was a poor use of their time (mean average of 3.10). As critical reflection is only possible through revisiting and rethinking about one’s own practice, the findings suggest that students require more support to help them reflect effectively on their learning. Students were generally neutral about the pedagogical benefits of video blogging, with most of the key questions on testing its value of video blogging returning scores of between 2.42 and 2.60.

But despite students’ unwillingness to re-watch their own videos, many students found value in watching their performances of their peers (question 14, mean average 2.84). This question shored the highest among those that directly tested the pedagogical benefits of video blogging, although there was also a greater swing of opinions (standard deviation 1.23). In particular, a quarter of respondents (n = 8) used the free text space to indicate that they had found blogging to be a useful way of reflecting on what they had learned. For example, one student wrote, “it develops my ability to evaluate other people’s work and realise what works and what we need to improve”, and another found it valuable to “see the difference between performing the same thing both online and on stage.” In Fig. 4 below, the course leader explains that she witnessed an improvement in students’ ability to reflect on their learning as they created more video blogs.

Fig. 4

Improving reflection. Please click the following link to view the video clip: https://www.youtube.com/watch?v=ed2_3TxR_OA

While there were few negative comments regarding watching other students’ videos, one student found it “pointless [because it consisted of] things we had already discussed in class.” It is also worth noting that the score for how often students watched others videos was slightly lower (question 13, mean average 2.31). The finding that students rarely re-watched their own videos but did regularly watch the videos of their peers supports the theory that while they are using other students’ video blogs to stimulate reflection, they require some assistance to help them see the value in reflecting critically on their own blogs.

In response to the main challenges of video blogging, almost half of respondents (n = 13) indicated that they found ‘organising what to say’ to be the most difficult aspect and over a quarter (n = 9) found video blogging to be too time-consuming. When combined with the finding that students found it valuable to view the blogs of their peers, this suggests that while students did not particularly enjoy the effort required to prepare and create their own video blogs they still found value in video blogging as a learning activity. Several students suggested strategies to combat the challenges of video blogging, and the most notable were to introduce group video blogs and to reduce the frequency at which they had to submit video blogs. As explained in Fig. 5 below, the unexpected findings from the study indicate the need for the use of technology to be informed by a clear instructional strategy.

Fig. 5

The need for a clear instructional strategy. Please click the following link to view the video clip: https://www.youtube.com/watch?v=-uGiH0odicM

The use of video blogs on the course was also intended to provide students with an authentic learning experience. The data analysis indicates that regardless of whether students re-watch their own videos, the act of creating videos of their performances and sharing them with their peers can be understood as supporting authentic learning. While it is evident that students require more support to help them fully appreciate the value of video blogging for professional development, the act of creating a personal video narrative and sharing it in front of other students is consistent with an authentic approach to learning as described by Donovan et al. (1999). Although the course strategy regarding video blogging requires further development to fully realise its potential to support an authentic, student-centred pedagogy, the data indicated that students’ experience of recording and reflecting on their progress could provide a valuable way to help them learn the critical skills they will require as professional actors and performers.

When the findings are interpreted using Kearney’s pedagogical framework for learner-generated digital storytelling, it is possible to identify improvements for subsequent iterations of the video blogging activity that could address some of the issues emerging from the data. These are presented in Table 2 below:

Table 2 Interpretation of findings using Kearney’s pedagogical framework for learner-generated digital storytelling

Lastly, the following visualisation indicates the adjectives that students used in their surveys to describe aspects of the video blogging activity. The size of the word is determined by the frequency of its use and the figure was created through a word cloud software. Just as in the rest of the survey, the results here indicate mixed views: nearly equal number of students defined video blogging as useful and helpful as well as unmotivating and uncomfortable. However, the most common word used to describe the activity was “challenging”, which in itself could be interpreted both negatively and positively. Despite clear indications towards the need to modify certain aspects of video blogging activity, we found “challenging” as an appropriate and productive descriptor for a pedagogical innovation.

Conclusion & recommendations

This case study provides a valuable insight into students’ perceptions of video blogging. Analysing the data using aspects of Kearney’s framework for student-generated digital storytelling (Kearney, 2011) revealed important insights that can be used to enhance subsequent iterations of the video blogging activity, most notably the need to help students both appreciate the value of critically reflecting on their own video blogs and learn how to go about this. The findings support the conclusions of Blomberg et al. (2014) regarding the importance of using learner-generated video as part of a broader instructional strategy if the benefits to student learning are to be maximised. The data analysis also indicated that video blogging has the potential to support an authentic, student-centred pedagogy, although further research would aim to clarify how best to articulate the benefits of video blogging for their professional development.

It was particularly interesting to note that students valued the ability to view the video blogs created by their peers, but were much less inclined to re-watch their own video blogs and use these as a tool to reflect on their learning and progress. This underlines the need to take active measures to help students acquire literacies such as critical thinking and reflection that are integral to their development both as professional practitioners and as global citizens (Johnson and Morris 2010; de Andreotti 2014; Ohler 2006). The findings also suggest that while the Acclaim platform was itself easy to use, students would benefit from exemplars that demonstrate the benefits of commenting on the video blogs created by other students, and an explanation of how to use these critical comments to improve future work.

Although the generalisability of this research is limited, the findings identify several benefits of using learner-generated video to achieve a more student-centred pedagogy. Further research would aim to explore ways of helping students think more critically about the value of video blogging in order to approach the creation of video blogs from a more informed perspective. There is also potential to develop a model for the use of learner-generated video blogs to support critical reflection in disciplines beyond teacher education, and this would be of value in helping educators leverage the full potential of learner-generated video as student-centred instructional practice.