Applied Large Language Models (CMPE454) Course Detail

Course Name	Course Code	Season	Lecture Hours	Application Hours	Lab Hours	Credit	ECTS
Applied Large Language Models	CMPE454	Area Elective	3	0	0	3	5

Pre-requisite Course(s)
N/A

Course Language	English
Course Type	Elective Courses
Course Level	Bachelor’s Degree (First Cycle)
Mode of Delivery	Face To Face
Learning and Teaching Strategies	Lecture, Question and Answer, Drill and Practice, Problem Solving.
Course Coordinator
Course Lecturer(s)	Asst. Prof. Dr. Arda Sezen
Course Assistants
Course Objectives	To equip students with a solid understanding of Generative AI technologies, particularly focusing on Large Language Models (LLMs), Transformer architectures, and widely used development ecosystems such as Hugging Face. Students will explore foundational concepts, hands-on experiments, and real-world applications of LLMs.
Course Learning Outcomes	The students who succeeded in this course; Understand the foundations and impact of Generative AI and LLMs. Gain practical experience with Hugging Face tools, NLP pipelines, and tokenization. Analyze the Transformer architecture and its role in autoregressive training. Examine scaling laws, pre-training approaches, and reinforcement learning with human feedback. Apply parameter-efficient fine-tuning and integration of LLM components and systems. Explore the use of LLM agents and compound systems in real-world tasks.
Course Content	LLMs, Generative AI, tokenization, embeddings, NLP pipelines, attention, Transformer architecture, autoregressive training, LLM scaling and fine-tuning, RLHF, LLM system design, Hugging Face ecosystem, and deployment.

Weekly Subjects and Releated Preparation Studies

Week	Subjects	Preparation
1	Course Introduction, Introduction to Generative AI	Course Book – Chapter 1
2	Introduction to Hugging Face Tools and Models, Natural Language Processing and Language Models	Natural Language Processing with Transformers, Revised Edition – Chapter 1
3	Tokens and Tokenization, Word Embeddings	Course Book – Chapter 2
4	Experimenting with Word Embeddings, Tokens, and NLP	Course Book – Chapter 2
5	Language Models and Attention	Course Book – Chapter 3
6	The Transformer Architecture	Course Book – Chapter 3, Build a Large Language Model – Chapter 4
7	Midterm Exam
8	Autoregressive Training Transformer LLMs	Lecture Notes, Natural Language Processing with Transformers, Revised Edition – Chapter 10
9	LLM Architecture Variants, Scaling Laws in Training LLMs, Training Data for Larger LLMs	Natural Language Processing with Transformers, Revised Edition – Chapter 8, Chapter 11
10	Part I: Pre-training, continued pre-training, and task training Part II: Reinforcement Learning with Human Feedback (Chat preparation)	Course Book – Chapter 10, Build a Large Language Model – Chapter 5
11	Parameter-Efficient Fine-Tuning Methods	Course Book – Chapter 12
12	LLM Components	Lecture Notes, Course Book – Chapter 5
13	LLM Compound Systems	Course Book – Chapter 7
14	LLM Agents	Course Book – Chapter 7
15	Review
16	Final Exam

Sources

Course Book	1. Hands-On Large Language Models: Language Understanding and Generation, 1st Edition by Jay Alammar and Maarten Grootendorst, Publisher: O'Reilly Media, Oct. 15, 2024.
Other Sources	2. NVIDIA Deep Learning Institute: https://www.nvidia.com/en-us/training/
	3. Natural Language Processing with Transformers, Revised Edition, by Lewis Tunstall, Leandro von Werra, and Thomas Wolf, Publisher: O'Reilly Media, July. 5, 2022.
	4. Build a Large Language Model (From Scratch), 1st Edition by Sebastian Raschka, Publisher: Manning, Sep., 2024.
	5. Hugging Face web page: https://huggingface.co/
	6. PyTorch web page: https://pytorch.org/
	7. TensorFlow web page: https://www.tensorflow.org/

Evaluation System

Requirements	Number	Percentage of Grade
Attendance/Participation	-	-
Laboratory	-	-
Application	-	-
Field Work	-	-
Special Course Internship	-	-
Quizzes/Studio Critics	-	-
Homework Assignments	1	20
Presentation	-	-
Project	-	-
Report	-	-
Seminar	-	-
Midterms Exams/Midterms Jury	1	35
Final Exam/Final Jury	1	45
Toplam	3	100

Percentage of Semester Work	55
Percentage of Final Work	45
Total	100

Course Category

Core Courses	X
Major Area Courses
Supportive Courses
Media and Managment Skills Courses
Transferable Skill Courses

The Relation Between Course Learning Competencies and Program Qualifications

#	Program Qualifications / Competencies	Level of Contribution
#	Program Qualifications / Competencies	1	2	3	4	5
1	Gains adequate knowledge in mathematics, science, and subjects specific to the software engineering discipline; acquires the ability to apply theoretical and practical knowledge of these areas to complex engineering problems.
2	Gains the ability to identify, define, formulate, and solve complex engineering problems; selects and applies proper analysis and modeling techniques for this purpose.
3	Develops the ability to design a complex system, process, device, or product under realistic constraints and conditions to meet specific requirements; applies modern design methods for this purpose.
4	Demonstrates the ability to select, and utilize modern techniques and tools essential for the analysis and determination of complex problems in software engineering applications; uses information technologies effectively.
5	Develops the ability to design experiments, gather data, analyze, and interpret results for the investigation of complex engineering problems or research topics specific to the software engineering discipline.
6	Demonstrates the ability to work effectively both individually and in disciplinary and interdisciplinary teams in fields related to software engineering.
7	Demonstrates the ability to communicate effectively in Turkish, both orally and in writing; to write effective reports and understand written reports, to prepare design and production reports, to deliver effective presentations, and to give and receive clear and understandable instructions.
8	Gains knowledge of at least one foreign language; acquires the ability to write effective reports and understand written reports, prepare design and production reports, deliver effective presentations, and give and receive clear and understandable instructions.
9	Acquires an awareness of the necessity of lifelong learning; the ability to access information, follow developments in science and technology, and continuously improve oneself.
10	Acts in accordance with ethical principles and possesses knowledge of professional and ethical responsibilities.
11	Knows the standards used in software engineering practices.
12	Knows about business practices such as project management, risk management and change management.
13	Gains awareness about entrepreneurship and innovation.
14	Gains knowledge on sustainable development.
15	Has knowledge about the universal and societal impacts of software engineering practices on health, environment, and safety, as well as the contemporary issues reflected in the field of engineering.
16	Acquires awareness of the legal consequences of engineering solutions.
17	Applies knowledge and skills in identifying user needs, developing user-focused solutions and improving user experience.
18	Gains the ability to apply engineering approaches in the development of software systems by carrying out analysis, design, implementation, verification, validation, and maintenance processes.

ECTS/Workload Table

Activities	Number	Duration (Hours)	Total Workload
Course Hours (Including Exam Week: 16 x Total Hours)	16	3	48
Laboratory
Application
Special Course Internship
Field Work
Study Hours Out of Class	16	2	32
Presentation/Seminar Prepration
Project
Report
Homework Assignments	1	18	18
Quizzes/Studio Critics
Prepration of Midterm Exams/Midterm Jury	1	12	12
Prepration of Final Exams/Final Jury	1	15	15
Total Workload			125