LLM Fluency Scale Explained
Key Points
- The video introduces a model‑agnostic “LLM fluency scale” to help users gauge their AI proficiency, noting that most people fall below level 5.
- Level 1 (basic beginner) covers typical users who employ tools like ChatGPT or Copilot for simple tasks such as rewriting emails or editing documents.
- Levels 3‑5 focus on developing a mental model of how large language models work—understanding token prediction, reasoning limits, and the importance of context retrieval.
- The presenter offers practical resources, including a comprehensive assessment prompt and a 90‑day custom development plan, to guide individuals from their current level toward their personal AI goals.
Sections
- Model‑Agnostic AI Fluency Scale - The speaker introduces a model‑independent framework that grades users' LLM proficiency on a ten‑point scale and provides assessment prompts and a 90‑day development plan to guide improvement.
- Thinking Backwards for Prompt Engineering - The speaker explains that grasping AI mental models lets users focus on the desired output instead of the prompt itself, fostering intuitive, outcome‑driven prompt engineering across beginner to advanced skill levels.
- Teaching as a Learning Catalyst - The speaker emphasizes that teaching forces clarity, reveals knowledge gaps, and by documenting and sharing AI insights—through curricula, workshops, or media—one both scales influence and deepens personal understanding.
- Mapping Career Fluency to AI Agents - The speaker urges professionals to treat AI skill development as a fast‑moving train, build mental models of career fluency levels, and proactively map emerging AI‑agent competencies onto those levels to stay aligned with future demands.
Full Transcript
# LLM Fluency Scale Explained **Source:** [https://www.youtube.com/watch?v=DdlMoRSojtE](https://www.youtube.com/watch?v=DdlMoRSojtE) **Duration:** 00:14:28 ## Summary - The video introduces a model‑agnostic “LLM fluency scale” to help users gauge their AI proficiency, noting that most people fall below level 5. - Level 1 (basic beginner) covers typical users who employ tools like ChatGPT or Copilot for simple tasks such as rewriting emails or editing documents. - Levels 3‑5 focus on developing a mental model of how large language models work—understanding token prediction, reasoning limits, and the importance of context retrieval. - The presenter offers practical resources, including a comprehensive assessment prompt and a 90‑day custom development plan, to guide individuals from their current level toward their personal AI goals. ## Sections - [00:00:00](https://www.youtube.com/watch?v=DdlMoRSojtE&t=0s) **Model‑Agnostic AI Fluency Scale** - The speaker introduces a model‑independent framework that grades users' LLM proficiency on a ten‑point scale and provides assessment prompts and a 90‑day development plan to guide improvement. - [00:03:19](https://www.youtube.com/watch?v=DdlMoRSojtE&t=199s) **Thinking Backwards for Prompt Engineering** - The speaker explains that grasping AI mental models lets users focus on the desired output instead of the prompt itself, fostering intuitive, outcome‑driven prompt engineering across beginner to advanced skill levels. - [00:07:57](https://www.youtube.com/watch?v=DdlMoRSojtE&t=477s) **Teaching as a Learning Catalyst** - The speaker emphasizes that teaching forces clarity, reveals knowledge gaps, and by documenting and sharing AI insights—through curricula, workshops, or media—one both scales influence and deepens personal understanding. - [00:11:48](https://www.youtube.com/watch?v=DdlMoRSojtE&t=708s) **Mapping Career Fluency to AI Agents** - The speaker urges professionals to treat AI skill development as a fast‑moving train, build mental models of career fluency levels, and proactively map emerging AI‑agent competencies onto those levels to stay aligned with future demands. ## Full Transcript
You know, one of the questions that I
get the most is, "How do I level up an
in AI?" And how do I know that I'm doing
good or at least improving? There hasn't
really been a comprehensive approach
that is agnostic of models, that doesn't
care if you're a Chad GPT user or a
Copilot user or a Claude user. It just
focuses on the principles and your level
of understanding and helps you to scale
it up. That's what I'm doing here. In
this video, I'm going to walk you
through how to tell roughly where you
are in the overall LLM fluency scale.
Spoiler alert, most people end up below
five. This is a tough scale. Don't be
afraid. Then I'm going to walk you
through what it looks like to actually
improve. Now, you're going to have a lot
more to dig into here. In the post, I
have a comprehensive assessment prompt.
I also have a prompt for building
yourself a 90-day custom development
plan for wherever you're at. So, there's
lots to dig into, but first, let's
understand the levels and what they
mean, and I'm not going to do all 10
because people just don't stick around
for that. I'm going to give you some
blocks, right? Like a one to a three, a
three to a five, that kind of thing. And
we're going to go pretty quick here. So,
number one, level one, basic beginner
level. Most people are here. That's your
default. If you are a chat GPT user, a
co-pilot user, if you're the kind of
person who uses these AI tools to
rewrite emails, to adjust a document,
you're probably in this one to three
area. And I just want to emphasize
again, this is not a bad or good thing.
This is just helping you understand
where you are so that you can figure out
where you want to go and where your
goals are. Not everyone has to be a 10,
right? Like that's not the point. The
point is to understand your level and
what your goals are and make sure you
are equipped to get there. And that's
what I'm all about. Let's jump ahead to
number three to five here. What does
that look like? So 3 to five, you are
starting to build a mental model for AI.
And this is why this scale is so
important. By the way, no one talks like
this. Like people tend to give you
specific skill sets and and I can do
that, too. I'm going to talk about some
of the specific skill sets you
demonstrate, but you need an overarching
perspective on the fluency and
competency assessment that you're
looking for at this level. And 3 to 5 is
all about building mental models. You
are starting to understand how LLMs
actually work, what they do when they
reason, what they do when they don't
reason. You're starting to understand
that LLM don't truly know things, that
they're not programmed. You're
understanding what next token prediction
looks like. You have the beginnings of a
mental model of what AI can do. Now, one
of the things that is more important
these days than it used to be is
understanding context retrieval. It used
to be that if I gave you those like
understand how LLMs work lessons, that
would be enough. But now, as AI has
gotten more powerful, you actually do
need to understand the ability to
retrieve a larger piece of context and
work with it. Because to be honest,
these AIs can take booksized prompts
now, right? Book-sized context windows.
And so, you need to understand how that
works a little bit and have a mental
model for that, too. I hasten to add,
none of this means you can build an AI
system. None of this means that you can
build a context window like a rag system
or a memory system. If that's all above
your head, you're still firmly at 3 to
five if you have the mental model down.
The last piece I want to call out from a
mental model perspective is that this
conceptual understanding of AI is going
to naturally lead to you thinking
backwards from outcomes. You're going to
stop asking what should I tell the AI
and at this stage you're going to start
asking what is the output that I need?
because the mental models are going to
inform your understanding of how it
creates the outputs and you're naturally
going to start to say, "Okay, I get a
sense of how the sausage is made,
right?" And so this is the output I
want. I can work back in my head. And
this is how you start to get to what I
would call intuitive prompt engineering.
You're not reading from a book. You're
not trying to copy a prompt necessarily.
Maybe you do, maybe you don't sometimes.
And even if you do, you know how to
massage it and tailor it a little bit.
Or you can write it yourself, but you
know how to get to the outcome you want.
So many people are here. I would say
what I just described with sort of the
level one and two where you're just
basic users of co-pilot or a basic user
of shed GPT plus this level with
understanding LLM and how they work with
mental models and kind of going from
there. That's almost almost everybody
right like if you want to talk about
8020 80% are right there. Now what goes
on above those levels? I'm going to make
this as accessible as possible. And I'm
going to give you a sense of whether you
need to go farther or not based on your
goals. So from 5 to 7, you really are
probably going to be working with AI on
a professional basis very seriously. So
if you get above five, if you get above
this mental model session, there are
some patterns that start to come through
that you just don't see at a lower
fluency level. And I'm going to name a
few of them, but you're going to get the
idea. The overall approach is systemat
system systematization. You are using
systems thinking moving from a five to a
seven. And that applies to AI because
you take it very seriously. So a person
between a five and a seven is going to
be thinking in auditable patterns with
AI. They're going to be thinking in
terms of usually do this and they're
going to move that over to this is the
sequence I follow. I get a predictable
result. I know how to get the
predictable result and I can start to
systematize it in a way that others can
do it too. You see that difference? It's
not just an intuition at that point.
It's actually a understanding of how the
system works so that you can predict and
move with it. Another example of
systematic thinking is building for
prompt yield. So prompt yield is like
what is your quality output per unit of
prompting. So if you're prompting
inefficiently, you might take 10
iterations to get one usable output. But
if you're prompting efficiently, you
might do one or two prompts and get 98%
of the way there and move on. And that
is part of why, by the way, I emphasize
the kinds of prompts I emphasize in my
posts. I think it's really important to
value the tokens, value the time that we
are taking with AI so that we can go on
to other things. And it is much much
much more efficient to just do the
prompt correctly and just get the right
answer. And someone who is building and
thinking in systems is able to
understand that and also able to move
from a casual intuitive I think this is
the right prompt to get to this output
to a systematic this is the yield I get
on this prompt. I think this prompt can
be modified in these three ways and I'm
going to get a much more efficient
output and then they make the change and
they measure it and they see it. These
kinds of people think in feedback loops,
right? Like you're thinking in terms of
my systems working to make me more
effective at AI. doesn't necessarily
mean that you have to have tons of
tools, but in my experience, most people
at this stage will have a prompt
library. They will have five to seven
tools they're working with regularly in
the AI space. They will have preferences
for specific work tasks that are
associated with those tools and they
will be seen by their teammates as a
peer collaborator and peer leader who
can help the team put in place systems
that matter. So far so good. You notice
by the way these are not job specific. I
am not giving you the fluency levels for
engineers and then the fluency levels
for for PMS. Do you know why? It's
because I have a strong conviction that
AI is a generalist skill set and we are
probably teaching it wrong if we dive
too deep into verticals without that
generalist conceptual foundation. And we
really haven't had that. And that's what
I'm setting out to do here. I think it's
great if you understand how to build
with Langsmith as a developer, but I
don't think that's the only kind of AI
learning and grounding you need. And I
think that we're missing this piece
here. This sort of general approach to
skill sets and fluency. And I think
having a common understanding here will
be helpful. Let's jump to 7 to 9. What
does it look like? Really, at this
point, you've mastered systems thinking.
You understand how LLMs work. You are a
teacher and you are a trailblazer. And
so you need to start thinking about who
you can teach with your skill set and
how your teaching drives your own
learning. So I will say for me teaching
has been super helpful in driving
clarity and revealing gaps in my own own
understanding that I have to
relentlessly close. Most teachers will
tell you regardless of subject that
that's true. Try to be if you're at this
level a documentarian. And what I mean
by that is the more you document about
what you're learning and what you're
thinking and how you're growing, the
more you're able to scale your influence
and teach others. And it's not about
growing influence. It's about being able
to communicate really clearly things
that are net new in the space that you
can then understand how to teach others
in a way that is accessible for their
level. And so that might look for you
like setting up the AI training
curriculum at work. It might look for
you like leading a group of developers
through through their first AI build. It
might look for you kind of like what I
do here where you're on YouTube or
you're on Substack and you're kind of
like talking through what it means to
grow and learn AI. There are lots of
ways to do this, but the systematic
thinking doesn't go away. And so you're
not just thinking in team systems or
personal systems. You are often doing
something that is public that many, many
others can use. So you might be building
a clawed projects that others can use.
You might be building a little vibe
prompted tool that others can use in
order to understand their own level of
fluency. Similar to what I've done with
the prompts in this in this piece
actually. But your goal is to pull the
impossible problems into the realm of
the possible. That's what that
innovation piece looks like. someone who
is teaching, who is learning, who is
growing, they should be helping to pull
forward things that were previously
deemed very difficult to do with AI
because they are helping to discover AI
capabilities. And by the way, that
understanding that AI capabilities are
not all documented and you can discover
them and you can put them to new uses is
a great example of what teaching and
innovation and the relationship between
systems thinking and deep understanding
of LLMs is all about. People who
understand LLM deeply know that LLMs are
not all discovered. We do not ship an AI
and OpenAI knows all about it. We ship
an AI and we all collectively discover
the capabilities it has because it's
more accurate to say these systems are
grown than to say that they are
programmed. And so we're all discovering
together what grew. And that is part of
the job at level seven to nine is to
start to innovate and understand where
to push farther on LLM capability and
why it matters and then be able to turn
around and teach that back and really
grow the practice. So
I don't give tens. There's not going to
be a 10 here. I think one of the things
that I want to call out is that you
should understand
that wherever you are, your competitive
reality is shifting. We are in October
of 2025. We are not too far away from
the end of the year in 2026. You need to
think of your baseline as shifting into
the new year such that like the whole
population is going to grow into one to
three in the next year and there will be
a much larger part of the population
growing into that sort of 3 to five area
and there will be many more people who
are pushing themselves up the skill
ladder from there. What I'm saying isn't
here to sort of panic you. Your goal may
not be to be a teacher or an instructor.
Maybe your goal is to be a systems
thinker. I don't know. or maybe you're
perfectly happy just understanding how
LLMs work. But regardless, I want you to
recognize that the skills required at
each stage are sort of evolving as you
go. My best advice, and I think I've
said this other times, is think of it as
a moving train and it's never going to
go slower than it's going right now. So,
hop aboard and get yourself going as
quick as you can in a way that you feel
comfortable that feels aligned with your
goals. And so just as I said that three
to five sort of people learning systems
thinking and all of that uh are are
starting to develop mental models of AI,
develop a mental model of your career
path a little bit. Have a sense in your
job family of what is the level of
fluency that would be useful. And then
here's the extra credit. I want to give
this to you because come back to 2026.
You're going to want this. Think about
the corresponding skill sets that will
emerge and map them onto this fluency
chart. Let me give you an example of
that. That can feel really abstract.
Think about AI agents. We just had on
October 6th a launch of a new kind of
agent framework from OpenAI. Well, think
about what the fluency types map onto
that, right? Like how do those map onto
it? Well, systems thinkers are going to
think about how you build not just one
agent but multiple agents, how you
sustain them within an org. Innovators
are going to think about new things you
can do with agents. People just
understanding LLMs are going to think
about what is an intuitive way to get a
task done that helps me to express my
understanding of LLMs and get real work
accomplished. And people who are just
starting out are going to scratch their
heads and say this this agent thing
looks really hard. But you can you can
map that whole technical launch onto
this capability assessment and you can
do that with other launches that are
coming forward too. And so this is not
meant to be an October 2025 artifact and
we're done. It is actually meant to be a
living breathing framework that helps
you make sense of your own skill level
relative to where we are with AI that
you can come back to again and again. So
there you go. That is my evergreen AI
fluency assessment. And as far as I
know, we haven't really talked about
stuff like this before, or certainly not
in this way. I hope you enjoyed it. I
hope it's helpful, and I'd love to hear
where you're at, and most of all, where
you want to get to. That's one of the
things I was really excited about for
this particular piece is I wanted to put
together a sense of the ladder, for lack
of a better term. It's really not a
ladder, but the sense of the jungle gem
of AI and a sense of where people can
go. And then here where all of you want
to go and start to craft some prompts
that help out and all of that. So yeah,
drop a note.