LLM Foundations

We will kickoff the course with a foundational understanding of AI, machine learning, and deep learning. We'll understand the intuition behind neural networks and train a simple neural network in Python for a prediction task. This module should give you a concrete sense of how these networks learn.

We'll learn how text is converted into a numerical format that these models can understand. We will also learn about different tokenization methods and implement a custom tokenizer in Python.

We will learn how words and tokens are represented as dense vectors, capturing semantic meaning. We will train a neural network to predict the token in a sequence, using these embeddings.

Attention is a core concept of LLMs. We will break down the "Attention is All You Need" paper, understand the intuition behind attention mechanism, and implement it in Python.

We will integrate everything we've learned in the previous modules to build a simplified transformer model with multi-head attention. We will train it to predict the next token in a text, giving you a practical understanding of how LLMs generate text. We will conclude the course with a summary of the key concepts and provide resources and guidance for you to continue your learning journey in the field of LLMs.