Deepening Students’ Understanding of GenAI Output

Overview

Developing an understanding of how Generative AI (GenAI) works is an ongoing project. That’s because the underlying technology behind GenAI continues to develop at a rapid pace. However, given how ubiquitous GenAI tools have become for students to use as part of their learning experience, it can be valuable to incorporate activities into your class. This then invite students to reflect both on the quality of the output they may be receiving from using GenAI tools as well as an engaged understanding of why GenAI tools may generate the kind of content that the students ultimately see. A literate user of GenAI will not just accept the output of a GenAI system without critical thought or assessment; rather, they will consider how the output meets the needs and expectations of what they’ve been assigned or asked to do.

This page gives you a few suggestions for ways that you can help students develop a deeper understanding of how GenAI works and whether its output is appropriate for meeting the goals and expectations of your class context.

Human vs. GenAI Output Comparison

A simple way to engage students’ critical awareness of what GenAI writing can produce is to ask students to compare and contrast two writing samples: one composed entirely by a human writer and another composed entirely in response to a GenAI prompt. You could provide these samples based on your own work or, alternatively, you could ask students to respond to a prompt by hand in-class (with no GenAI input) and then ask students to respond to that same prompt using a GenAI tool.

No matter how the samples are generated, students could then be asked to read and compare the two samples, considering, for example:

• What were the strengths and limitations of each writing sample?
• What did you find persuasive or compelling about each writing sample?
• How would you describe the tone and writing style of each sample?
• Who do you think would be most drawn to reading each sample? Who would you anticipate would be the target audiences or readers for each sample?

Giving students an opportunity to think critically about what they are producing and how that production changes the readers’ and writers’ experiences can deepen students’ appreciation of what it means to read and write with GenAI.

Information Audit

Many students use GenAI as a research or search tool to deepen their understanding of course content. As such, asking students to verify information that they receive from GenAI can augment their understanding of when GenAI output may be accurate and when it may be “hallucinating” or misrepresenting important citational information.

A simple activity to encourage students to verify source material or information provided by GenAI is to ask students to prompt a large language model to give them a few sources or pieces of information about a research topic they may be investigating for the class. Then, ask students to use Berkeley’s library search tool to cross-check those sources and see what they learn about those sources from using the institution’s library resources. Students can then reflect on:

• How did the results from the LLM shape or change the research investigation?
• What did I learn about the LLM’s recommended sources by looking them up in the Berkeley library database?
• How would I use an LLM for a future research project? What would I do similarly? What would I do differently?

You could also refine this exercise by giving students a research topic to explore with an LLM in advance. That way, all students could be researching the same exact topic and they could compare results and findings from across their various search results.

Prompt Reruns

When GenAI systems are prompted multiple times – even with the same prompts – they may produce different results. As such, invite students to see what happens when they repeatedly enter the same prompt into an LLM. For example, you might ask a large language model to show them a picture of a “student studying in the library” or a “person shopping at the grocery store.” Students can get creative with their prompts. Once they receive this first result, ask students to write down what they observe about it. What do they see? What is foregrounded? What smaller details do they notice?

Then, ask students to rerun the identical prompt several times. Then, ask students to track what they notice in aggregate. What’s consistent across the multiple prompt sessions? What changes?

After they’ve done these multiple prompts, ask them to consider what this has taught them about using the large language model. How would these results change or impact their future usage?