> ## Content Index
> Fetch the complete content index at: https://www.anup.io/llms.txt
> Use this file to discover other available public pages before exploring further.

# Deep Dive into LLMs like ChatGPT by Andrej Karpathy - TL;DW version
- URL: https://www.anup.io/deep-dive-into-llms-like-chatgpt/
- Published: 2025-02-09T16:56:03.000Z
- Updated: 2025-08-31T13:53:56.000Z
- Author: Anup Jadhav
- Tags: Machine Learning, LLM, AI Engineering, AI Talks

I think everything that [Andrej Karpathy](https://karpathy.ai/?ref=anup.io) shares on [X](https://x.com/karpathy?ref=anup.io) or [YouTube](https://www.youtube.com/@AndrejKarpathy?ref=anup.io) is a goldmine of information, and his latest video, [Deep Dive into LLMs like ChatGPT](https://www.youtube.com/watch?v=7xTGNNLPyMI&ref=anup.io), is no exception. It provides an in-depth breakdown of the key mechanisms behind LLMs. At a whopping 3 hours and 30 minutes, it covers a lot of ground, and I’d highly recommend watching it. But in case you haven’t watched it because it’s too long, here’s a TL;DW version.

### **TL;DW (Too Long; Didn’t Watch)**

- **LLMs are trained in two main stages:** pretraining (learning from vast datasets) and post-training (fine-tuning via human feedback).
- **Pre-training involves "downloading and processing the internet,"** using datasets like Hugging Face’s FineWeb (44TB, 15 trillion tokens).
- **Post-training refines the model** with Supervised Fine-Tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF).
- **LLMs suffer from hallucinations** but mitigations include model interrogation, web search integration, and structured reasoning techniques.
- **Prompt engineering plays a crucial role** in optimising LLM outputs and in-context learning allows adaptation without retraining.
- **LLMs still have challenges** like struggles with counting, spelling, and brittleness in certain scenarios, but improvements are ongoing.

### **Key Takeaways from the Talk**

#### **1\. The Foundation: Pretraining on the Internet**

The journey of building an LLM begins with pre-training on massive text datasets. Karpathy describes this step as effectively "downloading and preprocessing the internet." Large-scale datasets such as Hugging Face’s FineWeb (44TB, 15 trillion tokens) play a crucial role in training models to predict the next word in a sentence—a technique known as self-supervised learning.

Key processes in pretraining include:

- Downloading or using an existing large-scale dataset and pre-processing the data as follows:  
![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/ddaff43a-7da4-441e-8b01-7aec5bcd62e9_1150x492-1.png)
- **Tokenisation** – Converting raw text into token sequences.

![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/aec5bf39-7e28-4fb9-8e71-22e370844ffb_1226x848.png)

- **Neural Network Training** – Teaching models to generate coherent text based on input sequences.

![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/75dff731-35f1-433c-8d04-d599c89f62df_1228x1434.png)

- Inference

![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/4a239e9b-1435-4572-8d17-e6008196c693_1226x730.png)

However, a base model at this stage is essentially an "internet document simulator"—good at predicting the next token but not optimised for useful conversations.

I really like the idea of “The psychology of the base model” as described here:

![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/dd123e20-3d5b-4394-9fae-65c809231ceb_1238x408.png)

#### **2\. Post-Training: Making LLMs More Useful**

Once a base model is trained, it undergoes two crucial post-training steps:

- **Supervised Fine-Tuning (SFT)**: This involves feeding the model high-quality instruction-response pairs to refine its outputs and align them with human expectations.

![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/28570523-727b-4821-9d92-8a7004dfbd2a_609x570.png)

![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/d8a846b0-3478-4bc7-b810-a89a44d678e1_619x580.png)

- **Reinforcement Learning with Human Feedback (RLHF)**: Karpathy likens this stage to a student practising problems after learning theory. RLHF helps shape the model’s behaviour, making it more aligned with human preferences, reducing harmful responses, and improving coherence.

![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/f4036818-04fc-4348-8057-43f438aced3e_573x470.png)

![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/6cf3ca6c-9de9-45af-9e21-f8d146b58543_573x538.png)

#### **3\. Addressing Hallucinations**

A significant challenge in LLMs is hallucination—where models confidently generate incorrect or nonsensical information. Karpathy highlights mitigation techniques such as:

- **Model Interrogation** – Teaching models to recognise gaps in their knowledge.
- **Tool Use & Web Search** – Allowing LLMs to query external sources to verify facts.
- **Encouraging Reasoning** – Asking models to "think step-by-step" before answering complex questions.

#### **4\. Prompt Engineering & In-Context Learning**

Karpathy dives into prompt engineering, explaining how well-structured prompts can drastically improve model outputs. He also discusses in-context learning, where LLMs learn from the structure of prompts without needing weight adjustments—making them highly adaptable.

#### **5\. The Future of LLMs & Open Challenges**

Despite their remarkable capabilities, Karpathy acknowledges that LLMs still face fundamental limitations, such as:

- Struggles with counting and spelling.
- Brittleness in certain use cases (Swiss cheese model of capabilities).
- The challenge of extending structured reasoning to open-ended creative tasks.

Yet, the field is advancing rapidly, and we’re only scratching the surface of what’s possible with AI-driven reasoning and problem-solving.

I don’t think LLMs were compared to Swiss Cheese before (or at least I’ve not come across this analogy before) but I like it.

![](https://storage.ghost.io/c/b1/a4/b1a432f8-3f63-4839-8020-97b63b6056f7/content/images/2025/08/14b63ac2-38ae-436f-8e93-756101ac6d44_529x265.png)

### **Why You Should Watch the Talk**

Karpathy’s presentation provides a non-technical, clear, and engaging deep dive into how LLMs are trained, fine-tuned, and improved over time. Whether you’re interested in building AI applications, understanding AI alignment, or just fascinated by the inner workings of ChatGPT, this talk is an essential watch.