DAY #1
Date: 2026-06-27
"We are drowning in data, but starved for wisdom" - E.O. Wilson.
Artificial Intelligence
Intelligence - It comes from data.
Artificial - Because man made algorithms used to unlock the potential of this data.
For masses AI means LLM chatbot mostly. It responses for your queries like human does and understands context as your conversation grows. But, how this thing exactly happens under the hood ?
When can you answer any question ? When you know the answer or when your mind connects the dots from your past experiences. Same happens in LLM. LLM has trained on huge corpus of text data and from that corpus it learns the patterns and relationships between words.
To start - "LLM Basics By Andrej Karpathy" is great way to start.
Key points to remember from this pdf : - Two stages - Pre-training and Fine-tuning - In pre-training : we compress Tbs of data into billions of parameters , like in llama-2 70B model, 10 Tb of internet knowledge is compressed into 10gb parameter file. - In fine tuning : people write quality Q/A responses and then fintune base model with this data and obtain assistant model.
Satrted watching Karpathy's micro grad videos. Ended up in 3blue1brown calculus videos. Reason is - there should be no rush in this. And explore as much as possible in basics.
Backpropagation
Karpathy says that every AI training process has backpropagation at its core. The only difference is that, at a higher level, there is more advanced mathematics involved for improving performance, but the underlying logic remains the same.
He explains backpropagation by showing a chain of equations. In the end, we have a final output, let’s say x, and we started with inputs a and b. Along the way, we performed different arithmetic operations such as addition, multiplication, division, and others, creating many intermediate variables. Eventually, these operations lead to the final output x.
Backpropagation means finding the derivative of x with respect to every variable involved in the chain. In other words, we calculate how much x changes when each individual variable changes.
That process of tracing backward through the chain and calculating these gradients is called backpropagation.
Now I am watching : https://www.youtube.com/watch?v=SmZmBKc7Lrs&list=PLgtmMKe4spCPsxyMpg-sxf3EcbsFYlzPK&index=1
Part of the Learning AI series.