GenAI in Higher Education: Key Considerations

Billy Kelly

Chair, National Academic Integrity Network.

Introduction

Generative Artificial Intelligence (GenAI) has changed the landscape of higher education (HE) momentously, offering fundamentally new affordances in learning while at the same time dramatically moving the needle on assessment integrity; it is much more than an inflection point on the trajectory of technology in higher education.

In many respects, GenAI has made manifest things that were already amiss in HE: higher education institutions (HEIs) generally have demonstrated sustained innovation, albeit at a slow pace with little fundamental change to teaching and learning methods and the assessment of learning that underpins their accreditation of learning. Those methods have not always changed to reflect technologies and their changed student bodies: more often than not, technological advances have been incorporated into a centuries-old operational model, perhaps with the limited exception of distance and online learning.

HEIs must now grapple with the short-run and long-run implications of Generative Artificial Intelligence (GenAI) for learners and for accreditation while recognising that the capabilities and applications of GenAI and related technologies are growing at a relentless pace. We are entering what Sarah Eaton calls a “postplagiarism” era, where developments such as neurotechnologies will bring ethical and integrity considerations centre stage (Eaton, 2023).

The discourse around GenAI is crucial: talking about GenAI as if it is a single entity doesn’t make sense; it covers and can encompass so many different levels and depths of use, raising differing concerns, among them, ethical and equity issues and inherent biases. Just as smartphones have changed the way we interact with society, so, too, will GenAI change the way that stakeholders of all types will engage with HE. The ubiquitous presence of GenAI, embedded in the everyday technologies that our teachers and learners use, will cast them as co-creators of knowledge and can be expected to impact on the behaviours of all within the HE ecosystem. Furthermore, the apparent expertise of GenAI and how GenAI is discussed in the media and the wider world threatens the social contract that HE has had with wider society: what is the rationale for HE if GenAI can substitute for it at a fraction of the cost?

Need to reimagine

Already we have seen GenAI create a crisis of trust in our assessment methods: the authenticity and integrity of some of HE’s widely used assessment methods have been fundamentally undermined by widely available GenAI tools. This has implications for accreditation of learning and for the policies and procedures that assure that learning.

GenAI technologies will challenge all levels of education to reshape the learning journey of students: what learning should or should not be outsourced to GenAI? To the extent that what happens in primary and post-primary education shapes those students’ patterns of engagement with learning, there are likely implications for student learning in higher education. While the impact of GenAI on earlier stages of learning may have consequences for higher education, what follows focuses just on the HE stage of the learner journey.

Learning

What does learning mean in the context of sophisticated GenAI and ubiquitous and embedded GenAI technologies? For example, if we just took the cognitive domain of Bloom’s taxonomy, what do the levels mean in the presence of ubiquitous GenAI with respect to what we expect as a learning outcome? In the presence of artificial intelligence, what do knowing, understanding, or applying knowledge mean in terms of the ambition that we have for the learner? What should the learning outcomes (LOs) look like? And what should the assessment of these LOs look like? The LOs must take account of artificial intelligence. Given GenAI, what are we looking for in our learners to demonstrate analysis, synthesis, and evaluation? The answers to these questions will reflect individual disciplines and there cannot be a ‘top-down’ solution; it must come from GenAI literate staff on the ground. Resourcing and leadership in GenAI literacy must come from the highest levels in the HEIs and the sector, but it is critical that those at the coalface of learning – where staff and students interact – have current and comprehensive understanding of the capabilities of GenAI and the ethical considerations in its use. This should be supported by professional development at national and institutional levels, e.g., .GenAI for Teaching and Learning: How to do it right? (The National Forum, HEA, 2024). What is key is that engaging with GenAI literacy is not a discretionary choice for those working and learning in HE: as educators, we need to ensure that we (and our learners) understand how, and when, to integrate GenAI into our learning practices and, at the same time, limit the outsourcing of required learning to GenAI and mitigate negative impacts on learning practices; learners need to be equipped with the evaluative skills to guide them in their use of GenAI tools. There is no ‘one size fits all’: what is appropriate will vary across disciplines and stages of learning.

GenAI technologies offer transformational possibilities in addressing the challenges posed by increased participation in HE: personalised learning supported by a GenAI tutor, individualised curricula reflecting diverse student needs, automated grading and feedback, etc. Mollick & Mollick, 2023, point to a creative use of existing GenAI technologies in supporting the student learning journey, while, at the same time, identifying the attendant risks associated with their use. The potential impact of GenAI technologies on student creativity is mixed: its supportive capabilities must be balanced with concerns of negative impacts and, in particular, with the impact of a learner’s ‘creative confidence’ on their effective engagement with GenAI in this area (Habib et al., 2024).

The GenAI landscape in higher education is likely to be populated by bespoke GenAIs, where LLMs are trained and fine-tuned to reflect specific disciplines and stages of learning. This points to a need for professional development for staff in training of LLMs and in the deployment of GenAI technologies.

Used appropriately, GenAI can be an enhancer of the aggregate of learning, raising those at the bottom of the distribution significantly more than those at the top. Mollick (2024) sees GenAI as a great leveller, raising the performance of those at the lowest initial levels, turning “poor performers into good performers” and boosting “the least creative the most”.

Assessment and accreditation

Accreditation of learning is arguably the core function of universities; if they cannot appropriately ensure that learning has taken place, then their authority, reputation, and raison d’être are fatally undermined.

GenAI has been shown to be capable of successfully answering existing sophisticated tests of knowledge and evaluation framed as MCQs (Newton & Xiromeriti, 2024) and of passing professional and other examinations (Varanasi, 2023), (Gimpel, et al., 2024). While GenAI is likely to be integrated into assessment practices, it is its unauthorised use that is problematic, specifically, unauthorised content generation (Foltynek, et al., 2023).

In the short to medium term, much of the focus is likely to be on setting out to secure current versions of assessments (or variants of these) against academic misconduct. This is an understandable response by the HE community but will not be a sustainable position for the longer term due to workload and resourcing issues. The challenge of ensuring the authenticity of students’ learning will, almost certainly, require a more engaged and personal interaction between learner and assessor, whether in proctored assessment or in interrogative interactions with learners such as interactive oral assessment (Ward, et al., 2023). It is worth noting that these approaches necessarily impact on practices related to anonymity of learners in assessment and feedback.

More importantly, Lodge et al. (2023) note the resource intensity of assessment security and identify “a need to target … [resource-intensive assessment methods] … to where it will have the most impact – namely those moments of assessment that provide greater assurance that students who have been awarded the qualification have achieved the program outcomes”.

As GenAI becomes embedded in the learning environment and learning itself is increasingly mediated by GenAI technologies, how concepts such as creativity, innovation, and originality are evaluated and rewarded will create new questions for educators. At the same time, it may confront learners with the need to navigate an uncertain line between what is allowed and what constitutes academic misconduct.

GenAI and hallucination

The apparent capabilities of Large Language Models (LLMs) are tempered by their tendency to ‘hallucinate’ where the LLM may produce plausible but erroneous information and present it as fact. While more recent LLMs hallucinate to a lesser extent, there remains a need to check the credibility and veracity of what has been produced. Detecting hallucinations is likely to present different challenges for learners depending on discipline and stage of learning: there will be disciplines where there are clearly established truths and others where interpretation is key; similarly, where learning is at the boundaries of knowledge, discerning fact from hallucination may be particularly demanding. The capability of learners to meet the challenge of verification may be an important consideration in deciding the stage of learning where learners are exposed to GenAI: for example, final-year undergraduates would be expected to be better placed to substantiate an LLM’s output than a first-year undergraduate.

Disruption to the integrity of qualifications

Without a doubt, the widespread use of GenAI has increased the risks to the integrity of qualifications: these integrity risks extend from pedagogy to assessment. Zero risk does not exist. It follows that institutions should adopt a risk-based approach to integrity, recognising the existence of risks, undertaking an assessment of those risks, and developing strategies to manage and mitigate those risks. The risks to pedagogy and curriculum may not yet be clearly identified but those to assessment are already evident. These include abilities to produce elements of assignments such as essays, reports, computer programming code, etc.

The design of assessment and the learning culture nurtured by institutions will play an important part in abating academic integrity risks but cannot be expected to eliminate those risks. That and the adoption of lessons from situational crime prevention can further mitigate the integrity risks in assessment. In a parallel with opportunities to prevent crime before, during, and after the crime event, Birks & Clare (2023) identify corresponding examples of clear guidance and assessment design (before), monitoring assessment activity through submission of drafts (during), and random interviews based on markedly different scores as between supervised and unsupervised assessments (after). Nonetheless, the efforts of institutions in setting out to detect cases of ‘unauthorised content generation’ will continue to be an essential element in deterring academic misconduct.

Assuring the integrity of qualifications

The rapid growth of GenAI tools and technologies and their incorporation into everyday tools used by staff and students has created blurred lines as between what is seen as acceptable use of GenAI and what has strayed into areas of inappropriate use. From an institutional and learning perspective, identifying that “inappropriate use” is important: is this a misconception on the part of the learner or is it a deliberate attempt to deceive? Depending on the conclusion reached, the institutional response may range from education and additional guidance to the application of misconduct procedures.

While the core of the institutional solution to assurance of integrity may be assessment design and guidance for learners, a necessary accompanying dimension must be the development of robust processes to detect inappropriate use.

It is clear that technological solutions such as GenAI detectors do not, themselves, represent a credible or equitable solution (Perkins, et al., 2024). But AI detectors are likely to be an essential element in institutional processes, “detectors are not all equal, but the best are better than faculty at identifying AI writing” (Bowen & Watson, 2024).

The discourse on the use of AI detectors is almost universally flawed by erroneously treating the results of these detection tools as capable of yielding a binary decision as was the case with using text-matching software to detect cut and paste plagiarism, e.g., Weber et al. (2023). Where detection tools are improperly used to yield binary decisions, the consequences include false accusations of misconduct, and ‘false positives’, with attendant stress on learners.

The development of robust processes to counter the unauthorised use of GenAi is a very real challenge for institutions, a challenge that is likely to involve both technological solutions and human interaction. At best, AI detectors and assignment metadata will raise suspicions which may prompt further investigation by professional staff. An additional problem for institutions is that where past technological solutions, such as text-matching software to detect plagiarism, allowed decision-making to be devolved to those grading assignments, this will not be the case where suspicions of the unauthorised use of GenAI are encountered.

Need for strategic responses by HEIs

Arguably, the disruptive technologies of the last 30 years – the PC, the World Wide Web, and other ICT technologies – have made little fundamental difference to the organisation and structure of learning in higher education. While universities have generally enhanced the reach and standards of higher education, for the most part, those disruptive technologies have been absorbed into the structures that predated them: libraries offer online access to resources and are transformed into learning spaces for students; VLEs allow delivery, management, and assessment of learning; remote learning has become less distinguishable from on-campus learning; and the learning experience of on-campus students is a more hybrid one. The responses of higher education to change have been operational, adapting its traditional model to reflect the affordances and challenges of the new disruptive technologies. A key factor has been the ability of higher education institutions to adapt to and manage the pace of diffusion of new technologies in their learning environments. So far, so good, for the established model of higher education.

But what of Generative Artificial Intelligence? GenAI offers a new set of affordances and challenges. It has the potential to transform pedagogy and student supports while at the same time impugning the validity of widely used means of assessment. We can already see operational responses, most notably in the modification of assessment methods, where institutions seek to address the risks of students outsourcing assessment tasks to GenAI. At best, this puts higher education in the recovery position.

Higher education needs a strategic response to artificial intelligence; operational responses will not be enough. The challenge for higher education is that GenAI is an evolving technology, so the effectiveness of any operational response is likely to be short-lived. Reliance on operational responses places higher education in a war of attrition with an evolving technology, an educational version of trench warfare. What is needed is a strategic response as a failure to do so will condemn higher education to a never-ending cycle of operational responses to this evolving technology, a path to enduring disorder. A strategic response demands a paradigm shift by universities if they are to avoid this perpetual cycle. In addition, the expected evolution of GenAI tools and technologies implies a necessary long-term commitment to resources and to staff development.

In planning for the future, HEIs cannot afford to wait until ethical issues around the use of GenAI are resolved to a universal consensus; the ethical concerns themselves are likely to evolve as GenAI evolves.

As higher education engages with preparing its graduates for a GenAI future, the question is often asked: “What should we teach (and how)?”- a more salient question is, “How will learning happen”? That later question points to the need to factor in the likely behaviours of learners in a GenAI world.

In designing the learning journey, higher education needs to ensure that GenAI enhances learning rather than providing shortcuts that undermine learning skills and the longer-term knowledge essential for foundational knowledge and the development of expertise. Institutions need to consider what parts of learning can be and should be outsourced to GenAI while preserving the goals of higher education and recognising that it is unlikely that one size fits all across disciplines.

Ethical considerations

For education, ethical considerations on the use of GenAI span micro, meso, and macro levels. At a macro level, concerns range from the resource and environmental costs of the development of GenAI technologies, human rights, equity of access, potential inherent biases, and intellectual property matters. At meso, institutional, level ethical considerations need to be central to universities’ engagement with GenAI irrespective of where GenAI is deployed, in curriculum design and delivery, in assessment, in support services, in data usage, etc. These considerations go beyond their use in higher education – HEIs must equip their students with the wherewithal to make ethical decisions in their graduate futures. Additionally, HEIs will also have important thought leadership roles in the appropriate use of GenAI in society. At the micro level, where faculty and learners interact and where GenAI is part of the learning process, the ethical use of GenAI should itself be a fundamental part of that learning process.

References:

Birks, D., & Clare, J. (2023). Linking artificial intelligence-facilitated academic misconduct to existing prevention frameworks. International Journal for Educational Integrity, 19 (1), 20.

Bowen, J. A., & Watson, C. E. (2024). Is it time to turn off AI detectors? Times Higher Education. Retrieved November 7, 2024.

Eaton, S. E. (2023). Postplagiarism: Transdisciplinary ethics and integrity in the age of artificial intelligence and neurotechnology. International Journal for Educational Integrity, 19 (1), 23.

Foltynek, T., Bjelobaba, S., & Glendinning, I. (2023). ENAI recommendations on the ethical use of artificial intelligence in education. International Journal for Educational Integrity, 19(1), 12.

Gimpel, H., Gutheil, N., Mayer, V., Bandtel, M., Büttgen, M., Decker, S., Eymann, T., Feulner, S., Kaya, M. F., Kufner, M., Kühl, N., Lämmermann, L., Mädche, A., Ruiner, C., Schoop, M., & Urbach, N. (2024). (Generative) AI competencies for future-proof graduates: Inspiration for higher education institutions. Hohenheim Discussion Papers in Business, Economics and Social Sciences.

Habib, S., Vogel, T., Xiao, A., & Thorne, E. (2024). How does generative artificial intelligence impact student creativity? Journal of Creativity, 34 (1), 100072.

Lodge, J. M., Howard, S., Bearman, M., Dawson, P., & Associates. (2023). Assessment reform for the age of artificial intelligence. Tertiary Education Quality and Standards Agency. Retrieved November 7, 2024.

Mollick, E. (2024). Co-intelligence: Living and working with AI. Portfolio/Penguin.

Mollick, E. R., & Mollick, L. (2023). Assigning AI: Seven approaches for students, with prompts. The Wharton School Research Paper.

Newton, P., & Xiromeriti, M. (2024). ChatGPT performance on multiple-choice question examinations in higher education: A pragmatic scoping review. Assessment & Evaluation in Higher Education, 1 (1), 1–18.

Perkins, M., Roe, J., & Vu, B. H. (2024). Simple techniques to bypass GenAI text detectors: Implications for inclusive education. International Journal of Educational Technology in Higher Education, 21 (1), 53.

The National Forum (2024). OpenCourses.ie. (n.d.). GenAI for teaching and learning: How to do it right?.

Varanasi, L. (2023). GPT-4 can ace the bar, but it only has a decent chance of passing the CFA exams. Here’s a list of difficult exams the ChatGPT and GPT-4 have passed. Business Insider. Retrieved November 7, 2024.

Ward, M., O’Riordan, F., Logan-Fleming, D., Cooke, D., Concannon-Gibney, T., Efthymiou, M., & Watkins, N. (2023). Interactive oral assessment case studies: An innovative, academically rigorous, authentic assessment approach. Innovations in Education and Teaching International, 1 (1), 1–18.

Weber-Wulff, D., Anohina-Naumeca, A., & Bjelobaba, S. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19 (1), 26.