Institution: University College Cork

Discipline: Philosophy

Authors: Joel Walmsley

GenAI tool(s) used: ChatGPT

 

Situation / Context

In 2023 and 2024, as part of a third-year undergraduate module in Philosophy (PH3035: “Philosophy of Artificial Intelligence”), I required that students use ChatGPT as part of their essay assignment. The module is typically taken by approximately 40 BA students from a number of disciplines (e.g., Philosophy, Digital Humanities), as well as a handful of postgraduate students (including the Higher Diploma in Philosophy). One of the requirements for the module is a 2500-word essay, and I have always given students a choice of essay questions, one of which concerns the “Turing Test.”

We cover this topic in the lectures, having read Turing’s famous 1950 paper “Computing Machinery and Intelligence” together with a variety of secondary literature and in-class discussions. In the last two years, I decided to address the same topic but adjust the way in which students were required to approach it; the appearance of ChatGPT provided a unique opportunity for students to conduct a Turing Test rather than merely read and write about it.

Task / Goal

My objectives here were four-fold. First, since the module itself concerns (the Philosophy of) AI, the essay assignment seemed like a good opportunity to give students some “hands-on” experience interacting with a current GenAI system rather than simply reading about it.

Second, the assignment provided an introduction to the technology itself by enabling students to learn and demonstrate best practices in using it (as well as exploring its strengths and weaknesses) in an environment where I could guide them in doing so responsibly.

Third, I designed the assignment in such a way that completing it would still meet the learning objectives for the module by requiring that students engage carefully with the texts, ideas, and debates that we had examined together in class.

Finally, I hoped that this assignment would also be enjoyable. Here, I was inspired by a recent tweet from Andrew Ng (former head of Google Brain), in which he said, “I wish schools could make homework so joyful that students want to do it themselves, rather than let ChatGPT have all the fun.”

Taken together, these four considerations, I surmised, would significantly reduce students’ ability – or even desire – to use GenAI in a way that would constitute academic dishonesty.

Actions / Implementation

The specific wording of this particular essay question was as follows:

“Conduct a Turing Test (or, rather, play Turing’s ‘imitation game’) with ChatGPT. Given what Turing says in his 1950 paper do you think that ChatGPT passes or fails? How, and why? What kinds—or topics—of conversation would make it more likely to pass, or more likely to fail? What does this show about the Turing Test? What does this show about ChatGPT?”

I designed questions in such a way as to allow students considerable creative freedom in their approach, whilst also ensuring that the essay topics required engagement with the material we had covered. In conjunction with the ordinary course material – which already gave students some familiarity with GenAI in general, and LLMs in particular – I spent a lecture specifically discussing how ChatGPT works, what its strengths and limitations are, and strategies for using it optimally (and honestly) in this kind of assignment. We went through the handout to discuss the ways that ChatGPT might be used productively, the appropriate ways for documenting that process, and my expectations and goals.

Outcomes

The resulting essays were a delight to read, and I was pleasantly surprised both by the enthusiasm that the students displayed, and the creative, insightful, and original ways that they had approached the assignment. Students had conducted Turing Tests by focusing on areas as diverse as humour, slang, gender, neurodiversity, morality, sport, cuisine, and perfumery, and demonstrated exactly the kind of understanding and engagement that I was hoping for.

In their course evaluations—in which I asked specifically about the “unconventional” assignment—students were universally positive. One even commented that the thought of “cheating” had not even crossed their mind because the assignment was so much fun to work on.

Reflections

Having set the assignment this way for two years now, I have had the opportunity to make some changes in light of my experiences. First, I am now much clearer with students about my expectations, and the somewhat novel or unconventional nature of the assignment. I emphasise both that we are all still learning how best to use this technology, and that, for that reason, I am very open-minded about how they can approach the assignment. An example of this includes confusion on students’ parts about how to cite or document their interactions with ChatGPT, so I now stipulate that they must include screenshots either in-text or as a clearly labelled appendix.

The ability to document this kind of process is a valuable skill for them to learn and will surely become even more so in the future. Second, because ChatGPT’s model is continually changing (including the expansion of some of its safety features, and its prohibited responses), I found it useful to experiment with ChatGPT myself before setting the assignment: this is something that will need to be done every year, with tweaks to the assignment introduced. For example, if one asks ChatGPT for an opinion, a feeling, or to report its own mental state, it always gives a “canned” or deflationary response, which reminds the end-user that it is “merely” an AI.

To avoid students getting “stuck” on such things, assignment questions can be tweaked to use this kind of response explicitly, in order to prompt further reflection (see, for example, question 2 in the assignment handout, appended).

Naturally, it must be said that my module lends itself well to this kind of approach by its subject matter concerns the Philosophy of AI. One might worry about the extent to which this particular exercise could be applied in other modules and disciplines. However, I have found that the general approach can be adapted to other contexts and to different topics. ChatGPT is very good at coming up with lists and examples, or proposing hypothetical novel cases, or answering in a particular style or voice, and so could usefully inform a range of different subject matters. For example, in another module of mine, students used ChatGPT to develop (and critique) dialogues with characters from the novel Brave New World and thereby explore the concept of free will with some success.

Several students also mentioned that brief interaction with ChatGPT could be used for brainstorming or for overcoming writer’s block. Care must be taken here, as such usage could limit students’ creativity and perhaps inhibit or bypass engagement with an issue; it may be more appropriate for students only to use ChatGPT after they have made an initial foray into the topic with their own brainstorming. Nonetheless, this is something that could be equally well explored and developed in similar assignments by adopting my approach of facilitating, rather than prohibiting, the use of GenAI.

Digital Resources

UCC Book of Modules Entry for PH3035:

https://ucc-ie-public.courseleaf.com/modules/?details&srcdb=2023&code=PH3035

PH3035 Essay Topics (Appendix):  PH3035-essay-topics Joel W

 

Author Biography

Dr Joel Walmsley is a lecturer and Deputy head in the Department of Philosophy at University College Cork. His research interests include philosophy of mind (especially cognitive science and AI) and philosophy of science. He is the author of Mind: A Historical and Philosophical Introduction (Hackett, 2006), Mind and Machine (Palgrave, 2012) and editor of C.D. Broad: Key Unpublished Writings (Routledge, 2023).