Building ChatGPT
Logistics
Class Information
- Lecture
- T / Th 12:30 PM - 1:45 PM
- Location
- 210 DeBart
- Slack
- #cse-30124-sp26
- GitHub
- nd-cse-30124-homeworks
Instructor
- Professor
- Bill Theisen (wtheisen@nd.edu)
- I think many students just call me Theisen (pronounced "Tyson"), which is fine too.
- Office Hours
- T 2:00 PM - 4:00 PM , TR 2:00 PM - 4:00 PM , and by appointment
- Office Location
- 350 Fitz
Teaching Assistants
- Graduate Teaching Assistant
- Tom Lohman (tlohman@nd.edu)
- Senior Teaching Assistant
- Melvin Pineda Miguel (mpinedam@nd.edu)
- Senior Teaching Assistant
- Michael Sorenson (msorenso@nd.edu)
- Senior Teaching Assistant
- Ethan Koran (ekoran@nd.edu)
Office Hours
Schedule
Click on the next to a topic for additional resources. indicates the required resource for the topic.
| Unit | Date | Topics | Assignments |
|---|---|---|---|
| Unit 01 | Tue 01/13 | What is an LLM | |
| Thu 01/15 | Text as Data | ||
| Tue 01/20 | Tokenization 01 | ||
| Thu 01/22 | Tokenization 02 | ||
| Sun 01/25 | Homework 01 | Homework 01 | |
| Tue 01/27 | Tokenization 03 | ||
| Thu 01/29 | Embeddings 01 | ||
| Sun 02/01 | Homework 02 | Homework 02 | |
| Tue 02/03 | Embeddings 02 | ||
| Thu 02/05 | Training Neural Networks | ||
| Sun 02/08 | Homework 03 | Lab 01 Homework 03 | |
| Exam 01 | Tue 02/10 | Review 01 | |
| Thu 02/12 | Exam 01 | Practice Packet 01 Practice Packet 01 Solutions Exam 01 Exam 01 Solutions | |
| Unit 02 | Tue 02/17 | Self-Attention | |
| Thu 02/19 | Multi-Headed Attention | ||
| Sun 02/22 | Homework 04 | Lab 02 Homework 04 | |
| Tue 02/24 | Positional Encodings | ||
| Thu 02/26 | Transformer Blocks | ||
| Sun 03/01 | Homework 05 | Homework 05 | |
| Tue 03/03 | Generative Transformers | ||
| Thu 03/05 | Building nanochat | ||
| Tue 03/10 | Spring Break | ||
| Thu 03/12 | Spring Break | ||
| Tue 03/17 | Training LLMs | ||
| Thu 03/19 | Post-Training | ||
| Sun 03/22 | Homework 06 | Lab 03 Homework 06 | |
| Exam 02 | Tue 03/24 | Review 02 | |
| Thu 03/26 | Exam 02 | Practice Packet 02 Practice Packet 02 Solutions Exam 02 Exam 02 Solutions | |
| Sun 03/29 | Homework 07 | Homework 07 | |
| Unit 03 | Tue 03/31 | Evaluating LLMs | |
| Thu 04/02 | Prompt Engineering | ||
| Tue 04/07 | Explainable AI | ||
| Thu 04/09 | The Alignment Problem | ||
| Sun 04/12 | Homework 08 | Lab 04 Homework 08 | |
| Tue 04/14 | Introduction to Agentic AI | ||
| Thu 04/16 | Agentic AI Workflows | ||
| Sun 04/19 | Homework 09 | Homework 09 | |
| Tue 04/21 | Multimodal Models | ||
| Thu 04/23 | Current LLMs and Future Directions | ||
| Sun 04/26 | Homework 10 | Lab 05 Homework 10 | |
| Exam 03 | Tue 04/28 | Review 03 | |
| Tue 05/05 | Exam 03 | Practice Packet 03 Practice Packet 03 Solutions Exam 03 Exam 03 Solutions |
There's loads of additional resources out there! If you find one that particuarly resonates with you, I'd appreciate it if you were willing to share it with the rest of the class. You'll even have the option to tag it with your name so future students can see who to thank!
Submit any additional resources to this google form: Additional Resources
Requirements
Coursework
| Component | Points |
|---|---|
| Lab Labs | 5 × 5 |
| Homework Homeworks | 10 × 5 |
| Midterm Midterm Exams (10 points for turning in exam practice packet) | 2 × 70 |
| Exam Final Exam (10 points for turning in exam practice packet) | 1 × 85 |
| Total | 300 |
Grading
| Grade | Points | Grade | Points | Grade | Points |
|---|---|---|---|---|---|
| A | 279-300 | A- | 270-278 | ||
| B+ | 260-269 | B | 250-259 | B- | 240-249 |
| C+ | 230-239 | C | 220-229 | C- | 210-219 |
| D | 195-209 | F | 0-194 |
Due Dates
Labs and Homeworks are due by 11:59 PM on the Sunday of the due week.
Practice Packets are due by 12:30 PM (start of class) on the day of the corresponding exam.
Practice Packets and Exams
I do something a little different in this class than I've seen in other ones. Instead of giving you old versions of the exams with which to practice, I will release a "practice packet". If you turn in this practice packet on the day of the exam (either online or paper is fine) then you will receive points on the exam itself for actually studying. The practice packet is graded entirely on completion. There will be an entry in canvas worth 0 points for each practice packet, the points you receive for doing it will be reflected in the exam score itself.
Policies
Participation
Students are expected to attend and contribute regularly in class. This means answering questions in class, participating in discussions, and helping other students.
Foreseeable absences should be discussed with the instructor ahead of time.
Community
Recalling one of the tenets of the Hacker Ethic:
Hackers should be judged by their hacking, not criteria such as degrees, age, race, sex, or position.
Students are expected to be respectful of their fellow classmates and the instructional staff.
Students with Disabilities
Any student who has a documented disability and is registered with Disability Services should speak with the professor as soon as possible regarding accommodations. Students who are not registered should contact the Office of Disabilities.
Academic Honesty
Any academic misconduct in this course is considered a serious offense, and the strongest possible academic penalties will be pursued for such behavior. Students may discuss high-level ideas with other students, but at the time of implementation (i.e. programming), each person must do his/her own work. Use of the Internet as a reference is allowed but directly copying code or other information is cheating. It is cheating to copy, to allow another person to copy, all or part of an exam or a assignment, or to fake program output. It is also a violation of the Undergraduate Academic Code of Honor to observe and then fail to report academic dishonesty. You are responsible for the security and integrity of your own work.
Late Work
In the case of a serious illness or other excused absence, as defined by university policies, coursework submissions will be accepted late by the same number of days as the excused absence.
Otherwise, there is an automatic 25% late penalty for assignments turned in 12 hours pass the specified deadline.
Classroom Recording
This course will be recorded using Zoom and Panopto. This system allows us to automatically record and distribute lectures to you in a secure environment. You can watch these recordings on your computer, tablet, or smartphone. In the course in Sakai, look for the "Panopto" tool on the left hand side of the course.
Because we will be recording in the classroom, your questions and comments may be recorded. Recordings typically only capture the front of the classroom, but if you have any concerns about your voice or image being recorded please speak to me to discuss your concerns. Except for faculty and staff who require access, no content will be shared with individuals outside of your course without your permission.
These recordings are jointly copyrighted by the University of Notre Dame and your instructor. Posting them to other websites (including YouTube, Facebook, SnapChat, etc.) or elsewhere without express, written permission may result in disciplinary action and possible civil prosecution.
CSE Guide to the Honor Code
For the assignments in this class, you are allowed to consult printed and online resources and to discuss the class material with other students. You may also consult AI Tools such as CoPilot or ChatGPT for help explaining concepts, debugging problems, or as a reference. Viewing or consulting solutions, such as those from other students, previous semesters, or generated by AI Tools is never allowed.
Likewise, you may copy small and trivial snippets from books, online sources, and AI Tools as long as you cite them properly. However, you may not copy solutions or significant portions of code from other students or online sources, nor may you generate solutions via AI Tools.
Finally, when preparing for exams in this class, you may not access exams from previous semesters, nor may you look at or copy solutions from other current or former students.
| Resources | Solutions | |
|---|---|---|
| Consulting | Allowed | Not Allowed |
| Copying | Cite | Not Allowed |
See the CSE Guide to the Honor Code for definitions of the above terms and specific examples of what is allowed and not allowed when consulting resources.
If you are unclear about whether certain forms of consultation or common work are acceptable or what the standards for citation are, you responsible for consulting your instructor.
If an instructor sees behavior that is, in his judgement, academically dishonest, he is required to file either an Honor Code Violation Report or a formal report to the College of Engineering Honesty Committee.
Resources
Submit any questions or suggestions to this anonymous google form: Questions and Suggestions
Note: This form is genuinely anonymous but anonymity is a priviledge. Please don't misuse it.
If you're interested in being a TA please apply via this google form: TA Applications
Note: Applications are due by the day of the second exam and will be evaluated shortly after.
Note: TAing for CSE 30124 is quite competitive and usually there are only 1 or 2 open slots a semester (if any), so it may be worth having a backup plan.
One of the benefits of ML/AI being extremely popular is there are many online resources available for learning it. The best teachers of the topics release much of their material avaible for free online. If something in class seemed unclear, you're encouraged to seek out an explanation that makes the most sense to you! If you find one that you really like, please share it with the rest of the class. Below are links to books, blogposts, and lectures that I personally find very useful.