Module 1 -> Lesson 3
evaluating chatBots' output
Last updated
evaluating chatBots' output
Last updated
video by Google for Developers
for hook
Text produced by chatGPT3.5 (included in Activity Guide)
Text produced by BARD (included in Activity Guide)
(if available in your school)
(if available in your school)
Enrichment articles
by Amanda Hetler
by Hannah Wren
by James Manyika
by Alex Hughes
by Arun Padmanabhan
In this lesson, students use the output from popular chatBots powered by large language models (LLMs) to get an overview of common arguments made in favor of and in opposition to facial recognition technology. Students consider the importance of prompts and compare the output from the two chatBots and use what they learn about LLMs in the mini-lesson to make predications about output, craft prompts and consider the similarities and differences in the output. In closing, students consider how chatBots can be used to support them in engaging more deeply in learning rather than as a strategy to blow off assignments.
While this lesson provides students with some general information about how large language models work, the main focus is on evaluating the outputs produced by two different models and critically considering how we use the technology.
This lesson provides sample output from chatGPT and Bard to be used for comparison in the activity guide. However, if you are able to access chatGPT in your setting, you might want to extend this lesson, so that students have the opportunity to experiment with different prompts, follow up prompts to output, and so on. Keep in mind that chatGPT is often down when it is flooded by requests. Bard is currently blocked in NYC public schools.
Students will be able to...
craft a prompt that will elicit desired output from a chatBot.
evaluate and compare texts produced by different chatBots.
respond in writing to text-based questions using evidence from the text and their own ideas.
45-90 minutes (adjust according to your students' needs).
R1 Cite strong and thorough textual evidence to support analysis of what the text says explicitly/ implicitly and make logical inferences; develop questions for deeper understanding and for further exploration.
RH9: Compare and contrast treatments of the same topic in several primary and secondary sources.
W1c: Use precise language and content-specific vocabulary to express the appropriate complexity of the topic.
9-12.DL.1: Type proficiently on a keyboard.
9-12.IC.1 Evaluate the impact of computing technologies on equity, access, and influence in a global society.
The following high-frequency general academic vocabulary terms are crucial to this lesson and future ones:
opponent: a person who disagrees with something and tries to change or stop it
proponent: a person who supports an idea or course of action
advocate: to support or recommend something publicly
prompt: the inputs or queries that a user gives to an LLM GAI, like chatGPT or Bard, in order to elicit a specific response from the model
claim: a statement that something is true although it has not been proved and other people may not agree with or believe it
controversial: causing a lot of angry public discussion and disagreement
potential: that can develop into something or be developed in the future
What concerns do people have about chatGPT (Bard, etc.)?
How are people using chatGPT (Bard, etc.)?
How, if at all, have you used chatGPT (Bard, etc.)?
Facilitate a short discussion with students to get a sense of their experience with chatGPT.
Building on the conversation that emerged from the conversation about the memes, ask students some of the following:
Are chatBots like chatGPT or Bard reliable? Explain your reasoning.
Are chatBots more harmful than they are beneficial? Explain your reasoning.
How do chatBots work?
Provide students with some general information about the training data for GPT3.5:
You might also find useful information about the LLMs powering each chatBot here:
"GPT-3.5: Generative Pre-trained Transformer-3.5 or GPT-3.5 is one of the largest LLMs developed by OpenAI and serves as the backbone for AI chatbot ChatGPT. Boasting an impressive 175 billion parameters, the model can carry out many tasks, such as text generation, translation, and summarization.
Inform students that the training set includes text that was available in 2021 and no later and that chatGPT is not able to access information from the internet.
Provide each student with a digital copy of the the activity via Google Classroom or whatever LMS you use in your classroom. Strategically group students in pairs, triads, or fours to collaboratively read the discuss the prompts and texts in the activity guide. Step by step instructions for the activity are included in the guide.
Check in with students by asking one of the embedded questions in the activity guide that prompts them to consider the ways in which chatBots can be used to support research:
How might you use this output to support independent research of facial recognition technology?
Present students with a few chatGPT memes that you source on your own or take from and ask them to consider the following questions:
Present the video with the following purpose for viewing: What is the relationship between data and an LLMs output?
"The model was trained using text databases from the internet. This included a whopping 570GB of data obtained from books, web texts, Wikipedia, articles and other pieces of writing on the internet. To be even more exact, 300 billion words were fed into the system. ()
LaMDA: Google's Language Model for Dialogue Applications (LaMDA) is the underlying technology behind the newly introduced Bard AI. This language model has undergone training using extensive conversational dialogue data, enabling it to grasp subtle linguistic nuances and engage in open-ended conversations. Google has also developed an advanced iteration called LaMDA 2, which is further refined and equipped to offer recommendations for user queries. LaMDA 2 incorporates Google's Pathways Language Model (PaLM), featuring an impressive parameter count of 540 billion." ()