Institution: University College Dublin

Discipline: Film and Television

Authors: Naomi McAreavey

GenAI tool(s) used: ChatGPT

Situation / Context

I first tested this approach in a new module on Shakespeare on Film and Television with 25-30 undergraduate students studying English, Drama and/or Film as part of their BA.

For the final assignment, students were asked to produce an idea for an adaptation of one of Shakespeare’s comedies for film or television and include a sample scene. This could be submitted in written form, as a video or as a storyboard. The assignment tested the students’ creative and critical engagement with one of Shakespeare’s plays, and their ability to address key questions about adapting Shakespeare for a contemporary screen audience.

My (entirely human-created) assignment prompt and template had been available to students from the start of the semester. But I needed a rubric, since I know they can support diverse learners in understanding expectations and setting goals and are particularly important for new and unfamiliar assignment types like this one.

Task / Goal

Creating rubrics can be involved and time-consuming work. I wanted to see if ChatGPT could make the process quicker, easier, and more efficient. I suspected that this was the type of task that ChatGPT would excel at because, as a predictive model, it is good at organizing information in a structured, clear, and consistent way—probably better than me, in fact.

I wanted to involve my students in the process so I allocated class time to the co-creation of assessment criteria that I would then input to ChatGPT. This exercise would give the students an opportunity to review what they had learned in the module so far and begin to prepare for the final assignment. It would also have the secondary benefit of introducing them to GenAI. After we had agreed on a set of criteria through a combination of group work and class discussion, I told them I would use these criteria to create a draft rubric using ChatGPT and bring it for their approval in the following class.

Actions / Implementation

After class, I asked ChatGPT 3.5 to create a rubric using the six criteria the students had created and organised across five levels from excellent to unsatisfactory.

As this was my first time using ChatGPT to create a rubric, there were several stages of refinement before I got a rubric that I was happy to bring back to the students. Along the way, I had to specify that I wanted the rubric in table form. I also made various clarifications to distinguish between different criteria, to refine what we meant by a particular criterion, to consolidate overlapping criteria, to add a criterion that I realised had been overlooked in our discussion, and finally, to simplify the now-bloated rubric.

This was all done outside of class and without direct student involvement, although I did share the process with them in the following class. I explained that I had created the rubric with the help of ChatGPT and I showed how I did it using our criteria and what I already knew about creating rubrics. By showing the process of creating and refining the rubric, I was able to demonstrate the importance of precise prompting and the critical evaluation of outputs. The exercise, therefore, not only helped to create a rubric but also provided an opportunity to talk to the students about the benefits and challenges of using generative AI. Below the process of creation in ChatGPT:

Fig. 1 Instructions in ChatGPT to create the rubric.

Outcomes

The students found the rubric useful. I also found it helpful when grading and providing feedback on the assignments, and I have been able to reuse the rubric since. ChatGPT thus helped me create a useful and adaptable assessment resource quickly and easily.

Using ChatGPT for this task also gave me an opportunity to explore the tool for an important but onerous task. Through the process, I learned how to use the tool more effectively and efficiently, knowledge that I could then share with my students.

As a novice with no experience creating rubrics with ChatGPT, I did not want to confuse my students by demonstrating the process in real-time. I realise now that this was a missed learning opportunity. Now, I would bring the process into the classroom by creating the rubric “live,” with either me or the students leading the process. I learned by doing, so it would be good to give the students this same opportunity to engage with the tools collaboratively in a safe and supported environment.

Reflections

There is no doubt that I was able to create a rubric much quicker and more painlessly using ChatGPT than I could have done manually. I hope the speed and ease of creating rubrics with GenAI might encourage more lecturers to use rubrics as a tool to support and empower learners.

However, they should be aware that creating a rubric with ChatGPT is far from automatic and requires extensive refinement to improve its accuracy, clarity, and precision. Nevertheless, I still found it much easier to refine an unsatisfactory draft created with ChatGPT than to create a new rubric from scratch.

I would advise others interested in creating rubrics with GenAI to:

• Define the levels (e.g., five levels: excellent, very good, good, satisfactory, and unsatisfactory).
• Decide whether to assign points to each level (e.g. from 4 to 0).
• Ensure the criteria are as precise and concise as possible, as ChatGPT tends to use many categories and sub-categories.
• Ask for the rubric to be set out in table form.

I used ChatGPT, but now that we have institutional access to Gemini or Copilot through an educational licence that provides enhanced privacy settings for staff and students, I would use one of these tools instead in the future.

Further Reading

Bolton, C. F. (2006). Rubrics and adult learners: Andragogy and assessment. Assessment Update, 18(3), 5–6.

Panadero, E., Jonsson, A., Pinedo, L., & Fernandez-Castilla, B. (2023). Effects of rubrics on academic performance, self-regulated learning, and self-efficacy: A meta-analytic review. Educational Psychology Review, 35, 113.

Reddy, Y., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435–448.

Author Biography

Dr Naomi McAreavey is a lecturer in Renaissance Literature in UCD School of English, Drama and Film. She is a University for All Faculty Partner who promotes Universal Design for Learning in UCD and a UCD Fellow in Teaching and Academic Development currently working on a project on students as partners. She is also leading a SATLE funded project on the impact of GenAI in Arts and Humanities teaching, learning and assessment.