Introduction

This chapter introduces basic concepts underlying AI, particularly popular language models like ChatGPT. Our goal is not to encourage or discourage the general use of AI, but instead to help you understand its foundational mechanics. We believe this information will become more and more essential to your education and work as AI rapidly becomes integrated into various technologies in our day-to-day life.

AI, Algorithms, & Machine Learning

Older AI relied on humans designing complicated but explicit instructions for how to interpret input and generate output. At their most basic, these instructions, or rules, take the form “if X, then Y.” “If a traffic light is green, then cars can go.” “If text includes the phrase ‘Dear So-and-so’, then it is a letter or email.” “If someone listens to Lana Del Ray, then they may also like Billie Eilish.”

Sophisticated sets of rules governing AI “decisions” and output are called algorithms. You are probably familiar with algorithms in the form of social media feeds or recommendation services. Algorithms influence what you see in your Google search results, on your TikTok “For You” page, and in your Spotify recommendations. They are also the foundation to other widely-used technologies, like image filters or “shortest path” calculations by your GPS. Recent tools like ChatGPT have popularized awareness of AI, but this technology has been in our lives for a long time.

Since the 1950s, AI has become more sophisticated through machine learning: processing extraordinary amounts of data and identifying patterns without explicit human programming. In other words, machine learning allows AI to work with more information, at greater speeds, and with more complex detail as it teaches itself (with human supervision) about the patterns found in its training input (text, images, code, audio, and more).

AI models that independently “learn” at high speeds, and with truly copious amounts of data to “learn” from, do tasks like categorize, write, calculate, summarize, prioritize, etc. in ways that seem closer to what an average human, with an average knowledge of the world around them, could do. As a result, we have seen the advent of new AI tools and rapidly improving capabilities for familiar ones. Increasingly complex AI tools, with mass troves of data, do things like interpret finer details in images, process and produce language for niche situations, and build/run complicated mathematical models, among other tasks.

AI models require guidance from people. Technology which is built on data patterns is only as good as that data and its own programming. We cover this topic in more detail in the next chapter, but AI can overgeneralize, make mistakes, and miss important details. It is common to assume AI is more capable than it actually is, sometimes in ways that can harm our learning, health, relationships, and more.

Generative AI & LLMs

GenAI models learn complicated networks of “rules” that they use to interpret input from a user and generate a unique output. The “unique” part is crucial. In theory, GenAI should not reproduce any of its training data, instead producing a new combination of words, images, and more that are nevertheless still dependent on the training data. (We briefly explore this replication concern and its implication for copyright in the Ethics chapter this week.)

Watch the video below for a short visual overview on generative AI (run time 5:01):

Large Language Models (LLMs) are a type of GenAI focused on language. ChatGPT, currently the most widely used standalone AI tool, is an example of an LLM. One reason for the popularity of LLMs, and skyrocketing public awareness of AI, is their the capacity for natural language, not just in output but in input. In other words, you can “talk” to the AI the same way you would talk to another human being. This feature allows you to use AI without needing to learn computer code or limiting yourself to a set of predetermined options. The only limit to your input (often called a prompt) is your own creativity, while the sophistication of outputs are continually evolving.

ChatGPT and similar tools work as though you are having a conversation with the AI. You type in a prompt, the AI responds, and then you can ask it to complete a new task or edit its previous responses based on further instructions from you. To a certain extent, it remembers your previous input as well as its own responses and can make adjustments without you needing to repeat instructions in their entirety. Again, this feature more closely approximates a real human conversation, making the AI both more accessible and deceptively human-like.

Here’s a screenshot of what a basic ChatGPT prompt and response looks like:

As the response text suggests, you could then type in instructions and have ChatGPT adjust the tone of the text, make it longer, include specific language, and more. This example is basic, but people use conversational LLMs to do any of the following tasks:

• Generating text. People have asked LLMs like ChatGPT to generate emails, essays, scripts, and other text-like outputs like computer code. Whether you should use AI to do these tasks is a topic we cover in the next chapter.
• Summarizing. LLMs can – mostly effectively – summarize longer texts or groups of texts, including documents that you upload.
• Translating. LLMs can serve as reasonably accurate auto-translation tools, with caveats. For example, their accuracy may vary especially when translating less common languages underrepresented in their training data.
• Interpreting and generating tone. In addition to basic text generation, LLMs can “perceive” the tone of your input and follow instructions in generating output to match a specific tone, context, or intended audience.
• Editing. In addition to tone adjustments, LLMs can offer suggestions for basic grammar and spelling edits alongside more significant restructuring improvements for longer texts.
• Providing information…maybe. A major task people frequently try to use LLMs for is answering questions or otherwise providing information. However, as we see below, the function of LLMs is predicting words, not fact-checking, evaluating sources, or often even linking to real web pages. Finding real information, especially for college-level research, is generally not an appropriate use for an LLM!

Training LLMs

AI models are trained on massive datasets of text, art, video, audio, and more. They “learn” rules for interpreting input and generating output through analyzing training data patterns, making guesses, and making corrections under human supervision. Through this process, they improve in their ability to generate “correct” responses.

Important points:

• AI models do not “create” content. Instead, they predict language, art components, etc. based on learned patterns.
• AI companies are often not transparent about their training data. Their use of copyrighted material without creator permission may violate ethical and legal standards. (See the Ethics chapter this week.)

Training data for LLMs include books, web pages, discussion boards, social media, computer code, and more. LLMs “learn” via a transformer that processes text as a sequence, analyzing word frequency in the context of the words and sentences around it. (A generative pre-trained transformer – a GPT – is the basis for the name “ChatGPT”).