Institution: Trinity College Dublin

Discipline: Professional Development

Author: Caitríona Ní Shé, Ana Elena Schalk

GenAI tool(s) used: ChatPDF, POE Chatbot

The Special Purpose Certificate (SpCert) in Teaching, Learning and Assessment for Academic Practice is a flexible, research-informed, practice-based programme for Trinity staff involved in teaching and/or supporting teaching and learning. During the academic year 2023-24 three 5 ECTS credit Level 9 modules were run as part of this programme. Perspectives on Teaching and Learning in Higher Education, and Assessment & Feedback in Higher Education were run in Michaelmas term, with 17 and 20 participants completing the respective modules. In Hillary term, 18 participants completed the Curriculum Design in Higher Education module. A total of 55 participants were part of this example. The format of each of the modules is similar, there are generally 6 synchronous (mostly in-person) sessions of 2.5 hours spread over an 8-week period, with learning activities to be completed pre and post sessions. Attendance and engagement with the activities is required as part of successful module completion. An assignment, worth 100%, is due approximately three weeks after the end of the module.

During the modules participants had at least one opportunity to “try out” a GenAI tool and several opportunities to discuss their use within higher education. In total, 55 SpCert participants had the experience of exploring how to use GenAI tools to support their learning process.

Task / Goal

By September 2023 discussion within higher education teaching and learning communities on the use of GenAI broadly focussed on the possibilities that participants were using these tools to assist with assignments. Trinity College Dublin, like many other higher education institutions, had focussed initially on designing approaches to ensuring academic integrity in the light of the release, and widespread use, of ChatGPT3. The main aim of integrating GenAI tools into the SpCert modules was to explore teaching staff’s familiarity with the tools and the possible impacts on their academic practice. We had two main goals:

• Provide the participants with an opportunity to use a GenAI tool to support their learning within the confines of the module’s learning environment
• Enable participants to reflect on this experience, identifying opportunities and challenges of using these tools within their own practice.

We considered the integration of GenAI tools in this way as tipping our toes into the water to gauge the temperature with a view to further immersion in 2024-25. Data was processed in compliance with Data Protection Legislation and Trinity College policies. In order to adhere to copyright, all material uploaded to the GenAI tools were either public domain or openly licensed.

Actions / Implementation

Within the context of the three SpCert modules we used two different GenAI tools.

• ChatPDF

As part of the Assessment and Feedback in Higher Education module participants were asked to pre-read an article on and to discuss the opportunities and challenges of assessment that were identified in this article. Subsequently, in class, the participants uploaded the article to ChatPDF and obtained a summary of the article. They then compared their understanding of the article, from their own pre-read of the article, with that provided in the ChatPDF summary.

In a scheduled in-person session of the Perspectives on Teaching and Learning in Higher Education module participants were asked, in pairs, to upload an article to ChatPDF on learning theories and to ask the bot to describe the four learning theories. They were then asked to select one of the learning theories and skim the article to determine how accurate they considered the summary from ChatPDF.

In the Curriculum Design in Higher Education module participants were encouraged to use ChatPDF, where relevant, when completing pre-reading for a session.

• POE bot

A purposely designed POE bot, where a selection of the learning materials were uploaded, those publicly available or openly licensed, was made available within the Curriculum Design in Higher Education module’s Virtual Learning Environment (VLE). The use of the POE bot was first explored in class and then made available for the remainder of the module. Participants were encouraged to ask the POE bot questions that related to the elements of curriculum design that were under consideration each week.

In all three modules the activities led to discussions on participants experiences of using the tools, and the positives and negatives associated with embedding these types of tools within their own practice.

• GenAI in Teaching & Learning

Throughout the three modules we discussed the opportunities and challenges that the range of GenAI tools available to students might bring to our teaching practice. For example, within the Assessment & Feedback in Higher Education module we focussed on the need to emphasise academic integrity, in light of possible GenAI use, and ensure that our assessment strategies support assessment integrity.

Outcomes

As outlined above, the use of the tools within the modules was discussed within the in-person sessions, in conjunction with their use. For the two modules held in Autumn 2023, most of the participants had not used a tool like this before, however within the Curriculum Design in Higher Education module, held in Spring 2024, half of the participants had previously explored the use of GenAI. Similarly, the Autumn 2023 participants expressed mixed emotions as to both their willingness to use the tools and their effectiveness, whereas the participants from Spring 2024 were more open to the possibilities that correct use of GenAI could benefit both them and their students. This is to be expected as the awareness and use of GenAI has continued to increase since the first release of ChatGPT3 in November 2022.

One of the outcomes of the discussion on ChatPDF was the fact that it identified three rather than four learning theories from one of the uploaded articles. This enabled participants to consider the limitations of GenAI tools. On the other hand, many of the participants found that ChatPDF provided them with material that they would be able to use when framing responses to discussion threads within the SpCert modules, thus useful as a time saving device.

With regard to POE most of the participants considered that they could see value in using such a tool in their own practice. However, amongst the disadvantages is the learning curve necessary to use them effectively, and the possible necessity of asking students to create accounts.

Reflections

• Many academics had not engaged with GenAI with respect to teaching and learning prior to attending this module. This could be for a variety of reasons: absence of an institutional position regarding using (or not) GenAI in teaching and learning, lack of time to try new tools, lack of AI Literacy skills, or simply concerns about the impact of these technologies within Higher Education.
• Many academics were unaware of the power of GenAI tools to both summarise and write.
• Most academics found it very useful to explore GenAI in the context of the modules.
• Some academics are willing to investigate how they might incorporate GenAI into their teaching but would require technical support to do this.
• Having to register for GenAI can inhibit participants willingness to engage with these tools and discourage them to use them with their own students.
• There is considerable work required from the educator in preparing a useful closed bot to support learning.
• It is imperative to gather the activity usage data from the VLE, Blackboard, at the close of the module as retrieving such data from Blackboard is limited to 180 days post module completion.

Next time – future iterations of the SpCert:

It is likely that the dissemination of the Trinity Institutional Statement regarding using GenAI for teaching and learning in the 2024-2025 academic year, could motivate participants to try GenAI in their teaching and learning.

Resources are now available for academic staff to support their efforts to embed these technologies into their practice. Thus, participants might be eager to explore further the use of GenAI in their teaching and learning processes; hence we should take this a step further, i.e. ask them to use one as part of the learning design experience.

As institutionally supported GenAI tools such as Copilot (and other AI tools developed within the institutional tools, i.e., the VLE chatbot and AI for Instructional Design) become available to all teaching staff, we should focus on exploring the potential of these tools.

 

Further Reading

References Used in Class

Bloxham, S. (2014). Assessing assessment: New developments in assessment design, feedback practices and marking in higher education. In H. Fry, S. Ketteridge, & S. Marshall (Eds.), A handbook for teaching and learning in higher education: Enhancing academic practice (4th ed., pp. 107–122). Routledge.

Carlile, O., & Jordan, A. (2005). It works in practice but will it work in theory? The theoretical underpinnings of pedagogy. Emerging Issues in the Practice of University Learning and Teaching, 1, 11–26.

Digital Resources

• Example of the in class activity entailed: the use of a Bot that we developed for this part of the module on POE.

• This AI tutor CDinHE2024Jan has a knowledge base related to the publicly available resources that are used on this module.
• In this task your are asked to use the tutor, as a group at the tables to:
• Investigate what is involved in the curriculum model that you have been given
• Be ready to outline how you might use this model/framework in your practice, and why!
• The AI tutor is easy to use, just click on the link above and keep asking questions!
• Note: Do not upload documents to an AI bot unless the document is publicly available.

Dr Caitríona Ní Shé is an Academic Developer. She has a role in Academic Practice involving the development and delivery of evidence–based professional development for academic staff within Trinity, and contributing to funded projects on educational developments, such as the HCI initiative. Caitríona has worked in higher education across a variety of roles since 2008. Her most recent prior role was as an Education Developer for Professional Development in the National Forum, she managed and evaluated the rollout of the National Forum’s suite of digitally badged professional development courses and supported the upgrading of the Open Courses platform. Prior to this, Caitríona designed, developed and implemented a short course for online educators as part of the #openteach project, led by DCU. In 2021 Caitríona completed her PhD on the use of technology-enhanced resources to support formative assessment in undergraduate Mathematics. Her current work is focused on curriculum design, encompassing aspects such as sustainability in the curriculum.

Dr Ana Elena Schalk is an Academic Developer. Digital Learning Lead. Trinity College. She has been a lecturer, researcher, academic developer, and academic manager for over 30 years. She worked at TU Dublin’s Learning, Teaching and Assessment Team, where she led several strategic learning initiatives and taught on the MSc Education and Post Graduate Certificate in University Learning and Teaching. Prior to joining Tu Dublin, she led a number of institutional, national, and international digital education initiatives in Latin America, working with the Ministries of Education, UNESCO, and the Iberoamerican States Organisation (OEI).

In Trinity, she leads the Digital Learning and GenAI in Teaching Learning and Assessment plan at the Centre for Academic Practice. This plan includes resources to support academic staff, collaboration with other institutional departments and units, and research development. She teaches as part of the SpCert and other professional development initiatives.

She led the creation and implementation of the open course “GenAI in T&L: How to do it right?” (badge), funded by the National Forum to develop AI Literacy skills for academic staff in Higher Education. This initiative is in conjunction with the University of Limerick and UCD.