In the last lesson, you learned about the first stage of AI training:
Pretraining.
That is the stage where the model reads massive amounts of text and learns language patterns.
But after pretraining, something important is still missing.
The model may know:
grammar
writing patterns
coding structures
factual associations
But it still does not know how to behave like a useful assistant.
In fact, a raw pretrained model can feel surprisingly strange.
It may:
This is where the next phase becomes important.
After pretraining, AI companies begin shaping the model into something more usable.
This stage is called:
Fine-tuning
If pretraining is like general education, fine-tuning is like job training.
It is the phase where a general language model becomes:
In this lesson, we are going to unpack:
what fine-tuning actually is
how human feedback changes AI behavior
why different AI assistants feel different
what RLHF means
and why “helpful behavior” is trained, not natural
By the end, you should stop thinking:
“The AI naturally behaves this way.”
and start understanding:
“Its behavior was intentionally shaped through training.”
The Difference Between a Base Model and an Assistant
Let’s start with an important distinction.
A base model is simply a model that completed pretraining.
It learned patterns from massive text datasets.
But it has not yet been carefully trained for human interaction.
You can think of it like this:
A base model has:
But it does not yet have:
polished behavior
conversational structure
safety tuning
helpful formatting
response discipline
We can explain it with the analogy of a fresh graduate entering a workplace.
The graduate may be intelligent.
But they still need training before becoming effective in a real job.
The same thing happens with AI.
What Is Fine-Tuning?
Fine-tuning is additional training performed after pretraining.
During fine-tuning, humans provide examples of:
The model studies these examples and adjusts its behavior accordingly.
For example, trainers may teach the AI:
how to answer politely
how to structure explanations
how to summarize clearly
how to avoid harmful responses
how to follow instructions carefully
Over time, the AI becomes more aligned with what humans consider helpful.
A Simple Analogy: Job Training
Imagine hiring someone fresh out of university.
They may know:
But they still need workplace training.
A hospital trains doctors differently from how a law firm trains lawyers.
The same thing happens with AI.
A general model becomes specialized through additional instruction and feedback.
This is why the lesson is called:
“The Job Training Phase.”
Because the model is essentially being trained for a role.
What Actually Happens During Fine-Tuning?
During fine-tuning, the model sees examples like this:
User Prompt
Explain photosynthesis simply.
Preferred Response
Photosynthesis is the process plants use to turn sunlight into energy...
The model studies thousands or millions of examples like this.
Gradually, it learns:
preferred wording
useful structures
response style
formatting habits
conversational behavior
This is why many AI assistants naturally:
Those behaviors were trained into them.
What Is RLHF?
Now we arrive at one of the most important ideas in modern AI alignment.
RLHF stands for:
Reinforcement Learning from Human Feedback
The name sounds intimidating, but the core idea is actually simple.
Humans compare AI responses and choose which one feels better.
The system then learns from those preferences.
How RLHF Works
Imagine the AI gives two answers to the same question.
Response A
Response B
clearer
structured
more helpful
Humans choose Response B.
The model then adjusts itself slightly toward that style of response.
This process repeats millions of times.
Gradually, the AI becomes more aligned with human preferences.
Why AI Assistants Feel Different
This explains something many people notice quickly.
ChatGPT, Claude, and Gemini do not feel identical.
That is because they were trained differently.
Different companies:
prioritize different behaviors
use different training examples
apply different safety philosophies
optimize for different user experiences
Your source highlights this clearly.
For example:
ChatGPT
Often feels:
energetic
structured
action-oriented
list-heavy
Claude
Often feels:
cautious
reflective
careful with uncertainty
more paragraph-based
Gemini
Often feels:
exploratory
conversational
flexible
creative
These differences are not accidental.
They come from:
training decisions
feedback systems
fine-tuning priorities
In other words:
AI personality is engineered.
Fine-Tuning Shapes Behavior, Not Intelligence
This distinction matters a lot.
Fine-tuning can shape:
tone
helpfulness
formatting
conversational style
refusal behavior
But it does not magically create true understanding.
For example:
Your source explains this directly:
Fine-tuning teaches behavior, not real knowledge.
That is why AI can still:
even when sounding highly confident.
Why Human Feedback Matters So Much
Without human feedback, many AI systems feel awkward or difficult to use.
They may:
overcomplicate answers
ignore instructions
produce chaotic outputs
behave inconsistently
Human feedback teaches the model:
what users prefer
what feels clear
what feels useful
what feels trustworthy
This is one reason modern AI assistants feel surprisingly conversational.
Their communication style was heavily shaped by human evaluation.
But Human Feedback Also Introduces Bias
This is important to understand.
Human feedback is not perfectly objective.
Humans have:
cultural assumptions
preferences
blind spots
political views
communication biases
So when humans train AI systems, some of those biases can influence the model.
This means:
AI behavior is not neutral
AI reflects training choices
“helpfulness” is partly subjective
That is why different AI systems sometimes respond differently to the same request.
Why AI Can Feel Trustworthy Even When Wrong
One of the most important consequences of fine-tuning is this:
The AI becomes better at sounding trustworthy.
That does not necessarily mean it became more accurate.
This distinction is critical.
A well fine-tuned model may:
sound confident
communicate clearly
appear intelligent
feel persuasive
while still producing:
false facts
weak reasoning
fabricated information
Fine-tuning improves presentation.
It does not eliminate errors.
The Main Training Stages (Simple Summary)
At this point, it helps to zoom out and see the bigger picture.
Stage 1 — Pretraining
The AI learns language patterns from massive datasets.
Like general education.
Stage 2 — Supervised Fine-Tuning
Humans show the model examples of preferred responses.
Like guided workplace training.
Stage 3 — RLHF
Humans rank outputs and reinforce preferred behavior.
Like performance feedback.
Stage 4 — Deployment
The AI becomes available to users.
Companies continue refining the system over time.
Common Beginner Mistakes
Mistake 1: Thinking AI behavior is natural
Most assistant behavior was intentionally trained.
Mistake 2: Confusing politeness with intelligence
A polite answer can still be wrong.
Mistake 3: Thinking all AI systems are the same
Different training methods create different personalities and behaviors.
Mistake 4: Assuming fine-tuning solves everything
Fine-tuning improves behavior.
It does not solve all reasoning or factual problems.
Mental Model
Here is the clearest way to think about this phase:
Pretraining
teaches the AI:
how language works
Fine-tuning
teaches the AI:
how to behave around humans
That distinction is extremely important.
Practice Thinking
Think carefully through these questions:
Why might a raw base model behave strangely without fine-tuning?
Why do different AI assistants feel different?
Why can a helpful-sounding response still be inaccurate?
What kinds of human preferences might influence AI behavior?
Why does RLHF make AI feel more conversational?
Take your time with these.
These are foundational AI literacy questions.
Key Takeaways
Fine-tuning happens after pretraining
A base model learns language patterns but not polished assistant behavior
Fine-tuning teaches preferred response styles
RLHF uses human feedback to shape behavior
Different AI assistants feel different because they were trained differently
Fine-tuning improves helpfulness, not true understanding
Human feedback strongly shapes AI personality and behavior
What’s Next
At this stage, the AI has:
But another important question remains:
Why does the AI sometimes refuse requests?
Why can the same prompt produce:
That takes us into:
In the next lesson, we will explore the hidden instruction systems working behind every AI conversation.
