Applied Large Language Models (CMPE454) Course Detail

Course Name	Course Code	Season	Lecture Hours	Application Hours	Lab Hours	Credit	ECTS
Applied Large Language Models	CMPE454	Area Elective	3	0	0	3	5

Pre-requisite Course(s)
N/A

Course Language	English
Course Type	Elective Courses
Course Level	Bachelor’s Degree (First Cycle)
Mode of Delivery	Face To Face
Learning and Teaching Strategies	Lecture, Question and Answer, Drill and Practice, Problem Solving.
Course Coordinator
Course Lecturer(s)	Asst. Prof. Dr. Arda Sezen
Course Assistants
Course Objectives	To equip students with a solid understanding of Generative AI technologies, particularly focusing on Large Language Models (LLMs), Transformer architectures, and widely used development ecosystems such as Hugging Face. Students will explore foundational concepts, hands-on experiments, and real-world applications of LLMs.
Course Learning Outcomes	The students who succeeded in this course; Understand the foundations and impact of Generative AI and LLMs. Gain practical experience with Hugging Face tools, NLP pipelines, and tokenization. Analyze the Transformer architecture and its role in autoregressive training. Examine scaling laws, pre-training approaches, and reinforcement learning with human feedback. Apply parameter-efficient fine-tuning and integration of LLM components and systems. Explore the use of LLM agents and compound systems in real-world tasks.
Course Content	LLMs, Generative AI, tokenization, embeddings, NLP pipelines, attention, Transformer architecture, autoregressive training, LLM scaling and fine-tuning, RLHF, LLM system design, Hugging Face ecosystem, and deployment.

Weekly Subjects and Releated Preparation Studies

Week	Subjects	Preparation
1	Course Introduction, Introduction to Generative AI	Course Book – Chapter 1
2	Introduction to Hugging Face Tools and Models, Natural Language Processing and Language Models	Natural Language Processing with Transformers, Revised Edition – Chapter 1
3	Tokens and Tokenization, Word Embeddings	Course Book – Chapter 2
4	Experimenting with Word Embeddings, Tokens, and NLP	Course Book – Chapter 2
5	Language Models and Attention	Course Book – Chapter 3
6	The Transformer Architecture	Course Book – Chapter 3, Build a Large Language Model – Chapter 4
7	Midterm Exam
8	Autoregressive Training Transformer LLMs	Lecture Notes, Natural Language Processing with Transformers, Revised Edition – Chapter 10
9	LLM Architecture Variants, Scaling Laws in Training LLMs, Training Data for Larger LLMs	Natural Language Processing with Transformers, Revised Edition – Chapter 8, Chapter 11
10	Part I: Pre-training, continued pre-training, and task training Part II: Reinforcement Learning with Human Feedback (Chat preparation)	Course Book – Chapter 10, Build a Large Language Model – Chapter 5
11	Parameter-Efficient Fine-Tuning Methods	Course Book – Chapter 12
12	LLM Components	Lecture Notes, Course Book – Chapter 5
13	LLM Compound Systems	Course Book – Chapter 7
14	LLM Agents	Course Book – Chapter 7
15	Review
16	Final Exam

Sources

Course Book	1. Hands-On Large Language Models: Language Understanding and Generation, 1st Edition by Jay Alammar and Maarten Grootendorst, Publisher: O'Reilly Media, Oct. 15, 2024.
Other Sources	2. NVIDIA Deep Learning Institute: https://www.nvidia.com/en-us/training/
	3. Natural Language Processing with Transformers, Revised Edition, by Lewis Tunstall, Leandro von Werra, and Thomas Wolf, Publisher: O'Reilly Media, July. 5, 2022.
	4. Build a Large Language Model (From Scratch), 1st Edition by Sebastian Raschka, Publisher: Manning, Sep., 2024.
	5. Hugging Face web page: https://huggingface.co/
	6. PyTorch web page: https://pytorch.org/
	7. TensorFlow web page: https://www.tensorflow.org/

Evaluation System

Requirements	Number	Percentage of Grade
Attendance/Participation	-	-
Laboratory	-	-
Application	-	-
Field Work	-	-
Special Course Internship	-	-
Quizzes/Studio Critics	-	-
Homework Assignments	1	20
Presentation	-	-
Project	-	-
Report	-	-
Seminar	-	-
Midterms Exams/Midterms Jury	1	35
Final Exam/Final Jury	1	45
Toplam	3	100

Percentage of Semester Work	55
Percentage of Final Work	45
Total	100

Course Category

Core Courses	X
Major Area Courses
Supportive Courses
Media and Managment Skills Courses
Transferable Skill Courses

The Relation Between Course Learning Competencies and Program Qualifications

#	Program Qualifications / Competencies	Level of Contribution
#	Program Qualifications / Competencies	1	2	3	4	5
1	Gain sufficient knowledge in mathematics, science and computing; be able to use theoretical and applied knowledge in these areas to solve engineering problems related to information systems.
2	To be able to identify, define, formulate and solve complex engineering problems; to be able to select and apply appropriate analysis and modeling methods for this purpose.
3	Designs a complex system, process, device or product under realistic constraints and conditions to meet specific requirements; applies modern design methods for this purpose.
4	To be able to develop, select and use modern techniques and tools required for the analysis and solution of complex problems encountered in information systems engineering applications; to be able to use information technologies effectively.
5	Designs and conducts experiments, collects data, analyzes and interprets results to investigate complex engineering problems or research topics specific to the discipline of information systems engineering.
6	Can work effectively in disciplinary and multidisciplinary teams; can work individually.
7	a. Communicates effectively both orally and in writing; writes effective reports and understands written reports, prepares design and production reports, makes effective presentations, gives and receives clear and understandable instructions. b. Knows at least one foreign language.
8	To be aware of the necessity of lifelong learning; to be able to access information, to be able to follow developments in science and technology and to be able to renew himself/herself continuously.
9	a. Acts in accordance with the principles of ethics, gains awareness of professional and ethical responsibility. b. Gains knowledge about the standards used in information systems engineering applications.
10	a. Gains knowledge about business life practices such as project management, risk management and change management. b. Gains awareness about entrepreneurship and innovation. c. Gains knowledge about sustainable development.
11	a. To be able to acquire knowledge about the universal and social effects of information systems engineering applications on health, environment and safety and the problems of the era reflected in the field of engineering. b. Gains awareness of the legal consequences of engineering solutions.

ECTS/Workload Table

Activities	Number	Duration (Hours)	Total Workload
Course Hours (Including Exam Week: 16 x Total Hours)	16	3	48
Laboratory
Application
Special Course Internship
Field Work
Study Hours Out of Class	16	2	32
Presentation/Seminar Prepration
Project
Report
Homework Assignments	1	18	18
Quizzes/Studio Critics
Prepration of Midterm Exams/Midterm Jury	1	12	12
Prepration of Final Exams/Final Jury	1	15	15
Total Workload			125