ECTS - Applied Large Language Models

Applied Large Language Models (CMPE454) Course Detail

Course Name Course Code Season Lecture Hours Application Hours Lab Hours Credit ECTS
Applied Large Language Models CMPE454 Area Elective 3 0 0 3 5
Pre-requisite Course(s)
N/A
Course Language English
Course Type Elective Courses
Course Level Natural & Applied Sciences Master's Degree
Mode of Delivery Face To Face
Learning and Teaching Strategies Lecture, Question and Answer, Drill and Practice, Problem Solving.
Course Coordinator
Course Lecturer(s)
  • Asst. Prof. Dr. Arda Sezen
Course Assistants
Course Objectives To equip students with a solid understanding of Generative AI technologies, particularly focusing on Large Language Models (LLMs), Transformer architectures, and widely used development ecosystems such as Hugging Face. Students will explore foundational concepts, hands-on experiments, and real-world applications of LLMs.
Course Learning Outcomes The students who succeeded in this course;
  • Understand the foundations and impact of Generative AI and LLMs.
  • Gain practical experience with Hugging Face tools, NLP pipelines, and tokenization.
  • Analyze the Transformer architecture and its role in autoregressive training.
  • Examine scaling laws, pre-training approaches, and reinforcement learning with human feedback.
  • Apply parameter-efficient fine-tuning and integration of LLM components and systems.
  • Explore the use of LLM agents and compound systems in real-world tasks.
Course Content LLMs, Generative AI, tokenization, embeddings, NLP pipelines, attention, Transformer architecture, autoregressive training, LLM scaling and fine-tuning, RLHF, LLM system design, Hugging Face ecosystem, and deployment.

Weekly Subjects and Releated Preparation Studies

Week Subjects Preparation
1 Course Introduction, Introduction to Generative AI Course Book – Chapter 1
2 Introduction to Hugging Face Tools and Models, Natural Language Processing and Language Models Natural Language Processing with Transformers, Revised Edition – Chapter 1
3 Tokens and Tokenization, Word Embeddings Course Book – Chapter 2
4 Experimenting with Word Embeddings, Tokens, and NLP Course Book – Chapter 2
5 Language Models and Attention Course Book – Chapter 3
6 The Transformer Architecture Course Book – Chapter 3, Build a Large Language Model – Chapter 4
7 Midterm Exam
8 Autoregressive Training Transformer LLMs Lecture Notes, Natural Language Processing with Transformers, Revised Edition – Chapter 10
9 LLM Architecture Variants, Scaling Laws in Training LLMs, Training Data for Larger LLMs Natural Language Processing with Transformers, Revised Edition – Chapter 8, Chapter 11
10 Part I: Pre-training, continued pre-training, and task training Part II: Reinforcement Learning with Human Feedback (Chat preparation) Course Book – Chapter 10, Build a Large Language Model – Chapter 5
11 Parameter-Efficient Fine-Tuning Methods Course Book – Chapter 12
12 LLM Components Lecture Notes, Course Book – Chapter 5
13 LLM Compound Systems Course Book – Chapter 7
14 LLM Agents Course Book – Chapter 7
15 Review
16 Final Exam

Sources

Course Book 1. Hands-On Large Language Models: Language Understanding and Generation, 1st Edition by Jay Alammar and Maarten Grootendorst, Publisher: O'Reilly Media, Oct. 15, 2024.
Other Sources 2. NVIDIA Deep Learning Institute: https://www.nvidia.com/en-us/training/
3. Natural Language Processing with Transformers, Revised Edition, by Lewis Tunstall, Leandro von Werra, and Thomas Wolf, Publisher: O'Reilly Media, July. 5, 2022.
4. Build a Large Language Model (From Scratch), 1st Edition by Sebastian Raschka, Publisher: Manning, Sep., 2024.
5. Hugging Face web page: https://huggingface.co/
6. PyTorch web page: https://pytorch.org/
7. TensorFlow web page: https://www.tensorflow.org/

Evaluation System

Requirements Number Percentage of Grade
Attendance/Participation - -
Laboratory - -
Application - -
Field Work - -
Special Course Internship - -
Quizzes/Studio Critics - -
Homework Assignments 1 20
Presentation - -
Project - -
Report - -
Seminar - -
Midterms Exams/Midterms Jury 1 35
Final Exam/Final Jury 1 45
Toplam 3 100
Percentage of Semester Work 55
Percentage of Final Work 45
Total 100

Course Category

Core Courses X
Major Area Courses
Supportive Courses
Media and Managment Skills Courses
Transferable Skill Courses

The Relation Between Course Learning Competencies and Program Qualifications

# Program Qualifications / Competencies Level of Contribution
1 2 3 4 5
1 Gains the ability to apply advanced computing and/or information knowledge in solving software engineering problems.
2 Develops solutions using different technologies, software architectures and life-cycle approaches.
3 Gains the ability to design, implement, and evaluate a software system, component, process, or program using modern techniques and engineering tools for software engineering practices.
4 Gains ability to gather/acquire, analyze, interpret data and make decisions to understand software requirements.
5 Gains skills of effective oral and written communication and critical thinking about a wide range of issues arising in the context of working constructively on software projects.
6 Gains the ability to access information to follow current developments in science and technology, conducts scientific research in the field of software engineering, and conducts a project.
7 Acquires an understanding of professional, legal, ethical and social issues and responsibilities related to Software Engineering.
8 Acquires project and risk management skills and gains awareness of the importance of entrepreneurship, innovation, and sustainable development, as well as international standards and methodologies.
9 Understands the impact of Software Engineering solutions in a global, environmental, societal and legal context while making decisions.
10 Gains awareness of the development, adoption, and ongoing support for the use of excellence standards in software engineering practices.

ECTS/Workload Table

Activities Number Duration (Hours) Total Workload
Course Hours (Including Exam Week: 16 x Total Hours) 16 3 48
Laboratory
Application
Special Course Internship
Field Work
Study Hours Out of Class 16 2 32
Presentation/Seminar Prepration
Project
Report
Homework Assignments 1 18 18
Quizzes/Studio Critics
Prepration of Midterm Exams/Midterm Jury 1 12 12
Prepration of Final Exams/Final Jury 1 15 15
Total Workload 125