ECTS - Applied Large Language Models
Applied Large Language Models (CMPE454) Course Detail
| Course Name | Course Code | Season | Lecture Hours | Application Hours | Lab Hours | Credit | ECTS |
|---|---|---|---|---|---|---|---|
| Applied Large Language Models | CMPE454 | Area Elective | 3 | 0 | 0 | 3 | 5 |
| Pre-requisite Course(s) |
|---|
| N/A |
| Course Language | English |
|---|---|
| Course Type | Elective Courses |
| Course Level | Natural & Applied Sciences Master's Degree |
| Mode of Delivery | Face To Face |
| Learning and Teaching Strategies | Lecture, Question and Answer, Drill and Practice, Problem Solving. |
| Course Lecturer(s) |
|
| Course Objectives | To equip students with a solid understanding of Generative AI technologies, particularly focusing on Large Language Models (LLMs), Transformer architectures, and widely used development ecosystems such as Hugging Face. Students will explore foundational concepts, hands-on experiments, and real-world applications of LLMs. |
| Course Learning Outcomes |
The students who succeeded in this course;
|
| Course Content | LLMs, Generative AI, tokenization, embeddings, NLP pipelines, attention, Transformer architecture, autoregressive training, LLM scaling and fine-tuning, RLHF, LLM system design, Hugging Face ecosystem, and deployment. |
Weekly Subjects and Releated Preparation Studies
| Week | Subjects | Preparation |
|---|---|---|
| 1 | Course Introduction, Introduction to Generative AI | Course Book – Chapter 1 |
| 2 | Introduction to Hugging Face Tools and Models, Natural Language Processing and Language Models | Natural Language Processing with Transformers, Revised Edition – Chapter 1 |
| 3 | Tokens and Tokenization, Word Embeddings | Course Book – Chapter 2 |
| 4 | Experimenting with Word Embeddings, Tokens, and NLP | Course Book – Chapter 2 |
| 5 | Language Models and Attention | Course Book – Chapter 3 |
| 6 | The Transformer Architecture | Course Book – Chapter 3, Build a Large Language Model – Chapter 4 |
| 7 | Midterm Exam | |
| 8 | Autoregressive Training Transformer LLMs | Lecture Notes, Natural Language Processing with Transformers, Revised Edition – Chapter 10 |
| 9 | LLM Architecture Variants, Scaling Laws in Training LLMs, Training Data for Larger LLMs | Natural Language Processing with Transformers, Revised Edition – Chapter 8, Chapter 11 |
| 10 | Part I: Pre-training, continued pre-training, and task training Part II: Reinforcement Learning with Human Feedback (Chat preparation) | Course Book – Chapter 10, Build a Large Language Model – Chapter 5 |
| 11 | Parameter-Efficient Fine-Tuning Methods | Course Book – Chapter 12 |
| 12 | LLM Components | Lecture Notes, Course Book – Chapter 5 |
| 13 | LLM Compound Systems | Course Book – Chapter 7 |
| 14 | LLM Agents | Course Book – Chapter 7 |
| 15 | Review | |
| 16 | Final Exam |
Sources
| Course Book | 1. Hands-On Large Language Models: Language Understanding and Generation, 1st Edition by Jay Alammar and Maarten Grootendorst, Publisher: O'Reilly Media, Oct. 15, 2024. |
|---|---|
| Other Sources | 2. NVIDIA Deep Learning Institute: https://www.nvidia.com/en-us/training/ |
| 3. Natural Language Processing with Transformers, Revised Edition, by Lewis Tunstall, Leandro von Werra, and Thomas Wolf, Publisher: O'Reilly Media, July. 5, 2022. | |
| 4. Build a Large Language Model (From Scratch), 1st Edition by Sebastian Raschka, Publisher: Manning, Sep., 2024. | |
| 5. Hugging Face web page: https://huggingface.co/ | |
| 6. PyTorch web page: https://pytorch.org/ | |
| 7. TensorFlow web page: https://www.tensorflow.org/ |
Evaluation System
| Requirements | Number | Percentage of Grade |
|---|---|---|
| Attendance/Participation | - | - |
| Laboratory | - | - |
| Application | - | - |
| Field Work | - | - |
| Special Course Internship | - | - |
| Quizzes/Studio Critics | - | - |
| Homework Assignments | 1 | 20 |
| Presentation | - | - |
| Project | - | - |
| Report | - | - |
| Seminar | - | - |
| Midterms Exams/Midterms Jury | 1 | 35 |
| Final Exam/Final Jury | 1 | 45 |
| Toplam | 3 | 100 |
| Percentage of Semester Work | 55 |
|---|---|
| Percentage of Final Work | 45 |
| Total | 100 |
Course Category
| Core Courses | X |
|---|---|
| Major Area Courses | |
| Supportive Courses | |
| Media and Managment Skills Courses | |
| Transferable Skill Courses |
The Relation Between Course Learning Competencies and Program Qualifications
| # | Program Qualifications / Competencies | Level of Contribution | ||||
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | ||
| 1 | Gains the ability to apply advanced computing and/or information knowledge in solving software engineering problems. | |||||
| 2 | Develops solutions using different technologies, software architectures and life-cycle approaches. | |||||
| 3 | Gains the ability to design, implement, and evaluate a software system, component, process, or program using modern techniques and engineering tools for software engineering practices. | |||||
| 4 | Gains ability to gather/acquire, analyze, interpret data and make decisions to understand software requirements. | |||||
| 5 | Gains skills of effective oral and written communication and critical thinking about a wide range of issues arising in the context of working constructively on software projects. | |||||
| 6 | Gains the ability to access information to follow current developments in science and technology, conducts scientific research in the field of software engineering, and conducts a project. | |||||
| 7 | Acquires an understanding of professional, legal, ethical and social issues and responsibilities related to Software Engineering. | |||||
| 8 | Acquires project and risk management skills and gains awareness of the importance of entrepreneurship, innovation, and sustainable development, as well as international standards and methodologies. | |||||
| 9 | Understands the impact of Software Engineering solutions in a global, environmental, societal and legal context while making decisions. | |||||
| 10 | Gains awareness of the development, adoption, and ongoing support for the use of excellence standards in software engineering practices. | |||||
ECTS/Workload Table
| Activities | Number | Duration (Hours) | Total Workload |
|---|---|---|---|
| Course Hours (Including Exam Week: 16 x Total Hours) | 16 | 3 | 48 |
| Laboratory | |||
| Application | |||
| Special Course Internship | |||
| Field Work | |||
| Study Hours Out of Class | 16 | 2 | 32 |
| Presentation/Seminar Prepration | |||
| Project | |||
| Report | |||
| Homework Assignments | 1 | 18 | 18 |
| Quizzes/Studio Critics | |||
| Prepration of Midterm Exams/Midterm Jury | 1 | 12 | 12 |
| Prepration of Final Exams/Final Jury | 1 | 15 | 15 |
| Total Workload | 125 | ||