Beyond Tools: Building AI Fluency
Key Points
- AI should be viewed as a multi‑dimensional competency set rather than a single skill tied to any one tool.
- Current certifications that focus on using a specific platform (e.g., OpenAI, Gemini) do not equate to genuine AI fluency, especially as we move into a rapidly evolving multimodal model landscape.
- Most existing courses treat AI as a job‑specific add‑on or merely a tool proficiency, overlooking the need for a broader, cross‑functional AI skill set.
- There are five core AI capabilities that every professional should develop, with “AI strategy”—understanding how to incorporate AI as a team member and align it with market and workflow goals—being the first and essential one.
- Strategic AI thinking is no longer the sole domain of executives; anyone involved in product, engineering, design, or other functions must be able to plan and deploy AI effectively within their teams.
Sections
- Beyond Tools: AI Fluency - The speaker argues that true AI competence involves mastering a suite of interrelated skills rather than merely earning tool certifications, emphasizing a holistic, multi‑dimensional approach to stay fluent in an ever‑expanding multimodel landscape.
- Mastering Prompting and AI Integration - The speaker stresses that prompting is a nuanced, teachable skill across models and contexts, highlights the new Hey Presto tool, and calls for teaching AI‑native workflow design and rigorous output evaluation.
- Designing Trustworthy AI Products - The speaker asserts that AI ethics is fundamentally product design aimed at building trust, and stresses the importance of cultivating a balanced skill set across all key LLM competencies instead of over‑relying on prompting alone.
- Filmmaker Embraces AI Workflows - A veteran Pennsylvania filmmaker describes how his fascination with AI transformed his video editing process, led to building new workflows, and ultimately connected him with the host through shared AI guides and prompts.
- AI Fluency Assessment Overview - The speakers explain a multi‑section AI fluency score, how it ranks users among peers, and react to seeing their ranking place them in the top percentile.
- Adaptive AI Workshop Module - The speakers review Module 1’s free, AI‑graded curriculum, which tasks the learner with designing a two‑hour workshop for non‑technical users on a core AI workflow and uses the resulting quiz to personalize subsequent modules.
- AI Community Prototype Launch - The speakers outline a recently rolled‑out AI prototype for a private Substack community, noting its sign‑up count, added social leaderboard and profile features, and upcoming UI fixes.
- AI-Powered Leaderboard Demo - The speakers walk through a community app’s homepage featuring a leaderboard and friend search, discuss evaluation‑only entries, share personal anecdotes about bug hunting, and explain how they assembled the system using multiple AI tools like Claude and Codeex.
- Navigating AI Misinterpretations and Clarifications - The speaker describes how AI models like Gemini and Claude can briefly produce incorrect or unsettling outputs, prompting users to cancel and then use iterative clarification methods (e.g., Whisper Flow) to steer the model toward the intended meaning.
- Celebrating Substack Community Benefits - The speaker promotes a Black‑Friday discount for Substack members, lauds the diverse, high‑level contributors who make the community thrive, and reflects on its organic many‑to‑many interaction.
Full Transcript
# Beyond Tools: Building AI Fluency **Source:** [https://www.youtube.com/watch?v=HDVG8RKYX9s](https://www.youtube.com/watch?v=HDVG8RKYX9s) **Duration:** 00:35:12 ## Summary - AI should be viewed as a multi‑dimensional competency set rather than a single skill tied to any one tool. - Current certifications that focus on using a specific platform (e.g., OpenAI, Gemini) do not equate to genuine AI fluency, especially as we move into a rapidly evolving multimodal model landscape. - Most existing courses treat AI as a job‑specific add‑on or merely a tool proficiency, overlooking the need for a broader, cross‑functional AI skill set. - There are five core AI capabilities that every professional should develop, with “AI strategy”—understanding how to incorporate AI as a team member and align it with market and workflow goals—being the first and essential one. - Strategic AI thinking is no longer the sole domain of executives; anyone involved in product, engineering, design, or other functions must be able to plan and deploy AI effectively within their teams. ## Sections - [00:00:00](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=0s) **Beyond Tools: AI Fluency** - The speaker argues that true AI competence involves mastering a suite of interrelated skills rather than merely earning tool certifications, emphasizing a holistic, multi‑dimensional approach to stay fluent in an ever‑expanding multimodel landscape. - [00:03:10](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=190s) **Mastering Prompting and AI Integration** - The speaker stresses that prompting is a nuanced, teachable skill across models and contexts, highlights the new Hey Presto tool, and calls for teaching AI‑native workflow design and rigorous output evaluation. - [00:06:28](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=388s) **Designing Trustworthy AI Products** - The speaker asserts that AI ethics is fundamentally product design aimed at building trust, and stresses the importance of cultivating a balanced skill set across all key LLM competencies instead of over‑relying on prompting alone. - [00:11:51](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=711s) **Filmmaker Embraces AI Workflows** - A veteran Pennsylvania filmmaker describes how his fascination with AI transformed his video editing process, led to building new workflows, and ultimately connected him with the host through shared AI guides and prompts. - [00:15:01](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=901s) **AI Fluency Assessment Overview** - The speakers explain a multi‑section AI fluency score, how it ranks users among peers, and react to seeing their ranking place them in the top percentile. - [00:18:11](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=1091s) **Adaptive AI Workshop Module** - The speakers review Module 1’s free, AI‑graded curriculum, which tasks the learner with designing a two‑hour workshop for non‑technical users on a core AI workflow and uses the resulting quiz to personalize subsequent modules. - [00:21:14](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=1274s) **AI Community Prototype Launch** - The speakers outline a recently rolled‑out AI prototype for a private Substack community, noting its sign‑up count, added social leaderboard and profile features, and upcoming UI fixes. - [00:24:23](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=1463s) **AI-Powered Leaderboard Demo** - The speakers walk through a community app’s homepage featuring a leaderboard and friend search, discuss evaluation‑only entries, share personal anecdotes about bug hunting, and explain how they assembled the system using multiple AI tools like Claude and Codeex. - [00:28:03](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=1683s) **Navigating AI Misinterpretations and Clarifications** - The speaker describes how AI models like Gemini and Claude can briefly produce incorrect or unsettling outputs, prompting users to cancel and then use iterative clarification methods (e.g., Whisper Flow) to steer the model toward the intended meaning. - [00:33:05](https://www.youtube.com/watch?v=HDVG8RKYX9s&t=1985s) **Celebrating Substack Community Benefits** - The speaker promotes a Black‑Friday discount for Substack members, lauds the diverse, high‑level contributors who make the community thrive, and reflects on its organic many‑to‑many interaction. ## Full Transcript
How do I learn AI? That is the topic of
the video this week. I want to talk
specifically about how we move from an
idea that AI is a single competency to
the idea that AI is a related group of
competencies that we can all stand to
get better in. And I want to talk about
a tool that I'm using to measure that.
But let's talk about how we think about
AI as a multi-dimensional skill set
first because I think if you don't
believe me on that, none of the rest of
it is going to make sense. And I have to
be honest here. I think most of the
courses out there are not doing any of
us any favors because they view tool
competency as equivalent to AI
competency. So as an example, if you get
an open AI certification, it certifies
you for use in the tool and they may
tell you it certifies you for AI, but
any given organization, Claw, Gemini,
OpenAI, I they're all make great models
and they're doing good work on the
training part for the tool. But we
shouldn't mistake the tool certification
for an understanding of artificial
intelligence, especially in a world that
is multimodel. If you have gathered
anything from what I've been talking
about the last month or two with the
launch of Gemini 3, with the launch of
Opus 4.5, we are living in a multimodel
world. Chad GBT 5.1, right? Every week a
new model comes along. Grock 4.1 last
week. It's a multi-model world and we
need to be ready for AI fluency that
scales as models continue to proliferate
and to grow and to evolve. And so if
you're looking at the elements of a
multi-dimensional skill set, you have to
look above the level of the tool. I
don't know very many courses that think
that way. And the ones that do tend to
look at a particular job family and say,
are you certified in AI for engineering?
Are you going to be focused on AI for
product management? whatever it is. And
they don't tend to think about AI as a
new core skill set above the level of
the tool that we all need to be good at.
Regardless of job family, there are
foundational AI skills we all need to
know. You're going to be asking me,
Nate, what are those skills? I'm going
to suggest to you that there are five
critical skills that we need to get
better at, that we need to grow in, that
we need to have in our growth kit for
AI, regardless of what job family we're
in. and we will use them. I'm they're
all practical. Number one is a sense of
AI strategy. Strategy is not just for
executives anymore. Strategy is for all
of us because AI puts artificial
intelligence as a team member on all of
our teams. We have to have the strategy
to know how to deploy that AI
intelligence correctly. We have to have
the strategic insight if we're looking
at products to know what is the right
product that moves my workflow forward.
If we're in any of the building spaces,
Videoding, engineering, I don't care
what it is, you have to have a strategic
understanding of the market and how AI
fits into that market. It is not
something you can outsource to just one
executive part of the business anymore.
Prompting, that's another
multi-dimensional skill set. You need to
know not just how do I prompt for a
particular task, but how do I evolve and
change and think about that prompting as
it shifts, right? How do I think about
prompting Gemini differently from chat
GPT? How do I think about prompting for
a deck differently than a doc? I
released a prompt tool because of this
gap just this week called Hey Presto,
and it's designed to help you figure out
how to form intent through prompting.
But prompting is a skill set that is
above the level of the tool. You can use
any tool. I don't care if it's Hey
presto or any of the other hundred tools
out there on prompting badly or you can
use them well. And that is your skill
set that drives that. And that is
something that we haven't got a good
handle on how to teach yet because it's
so new. The third critical skill set
that I want to talk about is how you
think about integrating AI into your
workflow. I talk so much about AI being
useless if it lives off to the side.
What does it mean if AI is deeply
connected and tied in and integrated
into your workflows? How do you design
workflows that are AI native? So,
they're integrated in that is also a
learnable skill set that we need to
practice and teach on, but very few
people teach on it. Critical evaluation,
that's another one. How do you evaluate
the output of an AI critically with good
taste, with good judgment, so that you
can say with confidence and authority,
this is good and this is terrible. I was
working on a little fun test today
between Gemini 3 and Chad GPT 5.1 Pro
and Claude and I was asking them to
write me a creative story because one of
the interesting bars for LLMs, even if
you're using them in a business context,
is how they form narrative. And so I
asked them and I gave them the same
prompt for a story. I had to use my
taste to figure out what the best answer
there was. I had to read through the
three different stories that they made
me to figure out which one was the
highest quality. I'm still digesting it.
Don't ask me. I'm still thinking it
through. So this is a skill that we need
to develop and it's a skill we need to
develop that we can use in a wide range
of context for numeric data, for decks,
for docs, etc. The final big piece of
skills that I don't think we talk enough
about is the ethics piece. And people
often sort of roll their eyes and say
ethics is for ethics officers. And I say
the ethical choices we're all facing are
actually so multi-dimensional and so
woven into our work that we probably
need to have an understanding regardless
of where the lines should be. I'll give
you a few examples. One is from Nano
Banana Pro. It can make passport
pictures. Now, now the simplest ethic
thing is don't fake a passport, right?
Like that's a really easy one. But a
complicated one is how do you think
about installing guardrails in systems
so that your system is not vulnerable to
a model change in output generation
capability that would enable something
like that to get through. That's system
design and ethics. You can also think
about in the same vein, how do you
change the way you have security
policies so that you have in-person or
physical asset policies where needed to
make sure that you aren't being spoofed
by something like a fake passport photo?
Or you can start to think about it a
different way. How do you build trust in
your product experiences? How do you
ensure that people trust what you are
making and that you are one of the good
guys if your model can be used so many
different ways? Where where's the right
line for guard rails? And so I am firmly
convinced that the question of how LLMs
ought to act, which we traditionally
call ethics, is really a question of
product design to build trust. And we
are all in the business of either
designing products that use AI or using
products in such a way that we want to
build trust with others. I would argue
that the way we use products to build
reports for sales or the way we use them
to build white papers can either be
trust building or not and that is again
it falls under that banner of ethics. So
I've given you sort of a painted picture
across strategy and prompting and uh
integration into workflows critical
evaluation fix and I've done that for a
reason. We constantly misjudge ourselves
when we don't understand the core skill
sets that we need to learn because we
tend to overindex on one of them while
ignoring the others. I will tell you
frankly of those five most of what I
hear is unprompting and I think that's
frankly an impoverished view of the rich
LLM skill set that we need to develop.
Most AI fluent users tend to be strong
in only one or two pillars and they tend
to be weaker in the rest. If you don't
measure across all fives, you don't get
a real picture of how someone thinks,
how they work, how they reason through
ambiguity. So where am I going? Why does
this matter? Real AI work blends all of
these skills, the judgment, the
synthesis, the workflow design, the
rapid learning, and the ability to in
interrogate models into a overall
application on a particular task that
delivers value. If you can only prompt,
but you can't evaluate output, that's
dangerous. If you're great at strategy,
but you can't operationalize a workflow,
it's not going to work out. You get the
idea. This is where I want to point you
to a tool that I did not build. This is
a tool that one of my Substack readers
built with my permission using a prompt
set from a very popular uh post that I
made about evaluating AI fluency. He's
done a fantastic job. It's called AI
Cred. I'll show it to you in a few
minutes. We'll talk with him a little
bit. I am sharing this with you because
this is a way for me to start to give
you tools to understand AI fluency to
test yourself on AI fluency. It's a
little bit fun. There's a little
leaderboard um but most importantly to
give you resources to grow that I
haven't seen collected and customized in
any other way. And so the idea with AI
credit is it does go across that full
multi-dimensional skill set. Strategy,
prompting, integration, critical
evaluation, ethics, workflow design,
synthesis. It's a very comprehensive
assessment of your AI skill set. It's
one you can retake and grow in. It is a
hard test to do well in. That is on
purpose. I believe the leaderboard on AI
cred right now, nobody scores above an
8.9 out of 10. No one has hit nine yet.
So maybe you're going to be the first
one. It is really tough. And when you
answer honestly and when you see your
actual score across these different
multi-dimensional skill set, it empowers
you. It gives you options to succeed
because it's going to come back with
custom resources that I spent a lot of
time picking out and tailoring so that
the recommendation you get is tied to
the specific gaps in your skill set that
allow you to improve. I don't want to
give you a one-sizefits-all course. I've
been asked for a long time, Nate, can
you give me just a course on AI so I can
scale up? And because the answer is
custom, because it involves this
multi-dimensional skill set, I can't
just point you in good conscious at one
tool and say, "Learn that tool." I can't
just point you at one particular course
and say, "That's just going to make it
work." I want to point you at a thousand
courses, right? And point at a bunch of
resources to read, but only in
bite-sized chunks that are tied to your
particular gaps. And that's where AI
cred comes in because it custom assesses
you for your unique skill sets and it's
consistent. So you can reassess yourself
in a month or in two months or in three
months and say where am I at? Have I
made progress? Did the resources I
actually dug into and learn change the
way I work and think? Because that's the
other piece I see is that so often when
we talk about AI fluency, it is a piece
of paper we staple to the wall. it is
not getting into our head and into our
hands and onto the keys. And this
measures whether that outcome is
actually. So with that, I am going to
transition over. I'm going to start
chatting with Jonathan who did so much
of the heavy lifting to help to put this
together. I'd love to hear his story of
the build, share it with you guys, share
AI cred a little bit, and that's what
the second half of this video is going
to be about. Let's enjoy. All right, I
am here with Jonathan. Uh really the guy
who built AI Cred. Uh, I want the second
half of this video to be all about sort
of one, how how this came to be, how you
make the build decisions to build a tool
like this and then of course what the
tool does. I'd love to get into the
tool, show it on screen, demo it a
little bit, um, and talk about it. Uh,
so that everyone knows like, hey, this
is what it is, right? It might be for
you, it might not be for you, but either
way, you know, about AI. So, Jonathan,
maybe introduce yourself and, uh, let's
jump into it.
>> Yeah. So, uh, I don't know. I'm a I'm a
filmmaker out of northeastern
Pennsylvania, uh, Wilsbury, Scranton
area. I I've been running a production
company for the last 20 years and uh
and I got started with AI like everyone
else and just
I got a little bit addicted and uh as I
was building workflows out for my video
editing and uh I don't know I did some
really cool stuff and I I was fascinated
with what AI was giving me the ability
to do and I couldn't stop talking about
it. tried to find other people to talk
about it and you know ended up following
you and uh you know that's the that's
the super a bridged version of the
story. So
>> yeah, we'll we'll take it. We'll take
it. We're heading into Thanksgiving.
We'll take the a bridged version. Um so
how about a guy cred? Like what what
brought you into that? I know that we've
been working together on that one for a
bit, but what was what was that sparked
that for you?
Um,
so Nate, I I gotta tell you, you keep on
making these videos and these guides and
these these prompts and uh and all these
notes over here, but every third post
you make could be an app. And um because
and I I think one of the things we're
most aligned on is uh is really trying
to share our skills and everything like
that. And when you when you made that
post
uh about those prompts determining your
AI fluency, I'm like, man, that's what
people need. We need, you know, you need
to identify where you're at and you have
to, you know, and that getting that
foundation is what's going to help you
um move forward. Uh
>> yeah, that that makes a lot of sense. I
think in a sense my my goal has always
been to put out good stuff and let it
sort of percolate through the internet.
And I think one of the things I'm really
excited about is that sometimes stuff
really lands. And it feels like with AI
cred,
the the vision goes well past Substack.
You don't have to be a Substack person
for this to get the idea that fluency
matters, that learning AI matters. Um,
and it's something that should be
evergreen and available and all of us
should be able to just dig in on our own
pace and do.
>> Yeah. No, I agree. It's it's not even
about the tools you use like whatever
model like we've been stressing the the
core principles that you're always that
you're always pushing like the the
fluency really does
matter regardless of tool tools how to
you know learning how to communicate
with AI. I mean honestly it helps you
learn how to communicate with people as
well which is super cool. Um
>> well actually I would love to hear and
see a little bit about AI cred. Maybe
you can throw it up on the screen. Walk
us through what it looks like and then
sort of toward the second half would
love to talk a little bit about like
using AI to build AI like how you
actually built it and put it together. I
think that's a nice little uh touch we
can do afterward.
So right now what we're looking at is
the dashboard. Um, I went ahead and I
got my I'm I'm currently just not not to
brag or anything, but I'm I'm number one
on the leaderboard.
>> We'll see how long that lasts.
>> Yeah. Um, so, um, you you can see your
score up here. And this is going to be
pretty different tomorrow, but, uh,
right now you can go in, you can take
your assessment, and so
you see your fluency score. we have you
know your your sectional breakdown. So
the fluenc the the evaluation works in
six different sections. It talk you know
it's the introduction and report you
know talks about what you've done where
you know what you're experienced in your
tech technical fundamentals
uh different use cases that you use AI
for prompt engineering skills your
strategic thinking and then you know how
you apply things practically and uh
man uh I just switched over
>> really breakdown isn't it? Yeah, it's
insane. Especially I I switched it over
to Opus 4.5 yesterday. It just
>> oh my god, it it it just gives some
crazy stuff. It gives you a competitive
context.
>> Um
that competitive context is that about
like you in relation to your peers.
>> Yeah, pretty much just um you know how
you know for example mine reads uh
Jonathan operates in the top 5% of AI
practitioners. his combination as a
system thinking documented workflows and
active frontier uh experimented
experiment I I can't read uh places him
well above most professionals including
blah blah blah so uh I'm feeling a
little self-conscious reading that right
now because it really talks
>> uh the brutal truth is where I feel a
little less
>> you know so
>> um so you've mastered personal AI
fluency and built an impressive and
built impressive systems, but uh you're
still operating as a solo practitioner.
Your knowledge transfer has reached
maybe 12 people through one-on-one work,
and that's not scale. Now, here's the
thing. This is where it's wrong. Well,
it's hopefully it's going to be wrong
tomorrow. Uh your your assessment app is
in the right direction, but it's still
MVP with no revenue validation. Your
technical depth is genuine asset, but
also a blind spot, you know,
>> but then you can retake it, right?
That's what I've been saying is with AI
cred, you can go back and retake your
score and you can kind of see how you're
growing. And so that's an example for
you where you could you could be
growing.
>> And here's the thing. Um, and I don't
know if you've seen this part. Um, and
my god, I I hope it
uh let's see. I I Okay, here we go.
Start your training plan. I have not
clicked this button yet.
>> Oh, we're
>> on my profile. So,
um, yeah, it's going to take a minute.
So, what it's doing right now is it's
actually building a training plan. And,
um, your original prompt told it all to
build it at once. I I'm a little bit of
a psycho, so I wanted each maj.
So, uh, module two, three, four, five,
they won't be generated without the
context of the p previous module.
>> Okay. So, let's look at the module
content in module one. What is it
suggesting for you?
>> So, we're going to hop into module
content. And this is all live. Thank god
it works. Uh,
>> this is why we're launching it because
it actually works. I know it's the
builder's nervousness here, but this is
good.
>> Yeah. Yeah, for real. Um, so yeah, focus
area gives you a hand on hands-on
exercise. Um,
let me make that larger for everyone.
>> Yeah. Yeah.
um you know it wants me to design and ex
execute a two-hour workshop for group of
five non-technical users teaching one
core AI workflow then we could take a
progress check and this is AI graded so
I could you know it actually gives you a
quiz and
>> oh this module and in your case because
your work challenge or your growth
challenge is scaling your impact to
others the exercise is about that
>> exactly yeah and uh and then when you
finish that module and you take the
quiz. Uh, and this is all free. You
know, the evaluation cost of credit, but
all these modules, they're entirely
free. You know, you you pay to get free
education.
>> Y
>> So, uh,
yeah. Uh, you could take your progress.
you're getting that whole learning plan
that evolves with you because this is
really cool because it basically like
you'll take the progress check and
depending on how you respond, it's going
to customize module 2 to what I
particular need.
And that was honestly one of the things
that took me forever to build in um
>> because when you get done with your
learning path, it doesn't end there. Uh
you take a reassessment
which builds on the context of your
first conversation,
>> all your training modules,
>> and then it reassesses you and then it
regenerates a brand new training, you
know, learning path and
>> it's it's
super rad. Um
>> and tell me about like and just the way
you're designing it. Like there's
obviously hands-on exercises. What other
things are there? resources, videos,
like how does it like start to pull all
that together?
>> Well, what we're doing right now is
right now I have a basic uh you know a
basic database.
>> Um I would like to call it a rag, but
it's not a rag yet. That's I'm being you
know it's just it's just my sequel but
uh
uh it it pulls from honestly almost
entirely you and training
>> you know AI training modules uh I have
you know there's many many more versions
to come there's little you know there's
I I have a road map that's 10 miles long
but
>> um yeah that's that's where it pulls
from a lot of the training that you've
done I What are you posting? Six six
videos and four posts a day lately.
>> It's I also age in dog ears, Jonathan, I
will tell you.
>> No,
>> the beard grows ever wider.
>> Um, and just to show off a couple more
things that I really like. Uh, uh, so a
lot of people really like, you know,
well, we ran we ran a prototype of this,
uh, two months ago
>> uh, in in your private substack. Yeah.
Um, so for anybody who doesn't know,
Nate Substack has this whole group of
people like
>> hundreds of people that are like
obsessed with AI. So that's the place to
be.
>> Um, we we ran a prototype and had just
over a hundred people sign up for it and
they were all dying to see the other
people's scores like so
>> begging to and that's why we added the
social piece, the leaderboard piece.
>> Yep. So, we have
this profile page, which please please
ignore the This is going to be fixed
tomorrow. I promise.
>> You You can barely see the text.
>> The lime green there. Yeah.
>> Yeah. Yes, that'll be fixed tomorrow.
>> No, it's authentically built, right?
Like it's not like it's a bunch of VC
money here.
>> Yeah. Yeah. So basically, you know, you
could add add your social links, you
could add your own bio, but it it it
ranks you right here. Yeah.
>> Um uh it actually tells you what
position you are in the leaderboard. Uh
I have this, you know, this wingman
summary. A lot of people are
>> well me in particular, I don't like
bragging about myself. So if your public
profile or if you make your profile
public, you get an automatic
augment.
>> Brag about yourself. If the AI just is
your wingman and talks about you for
you.
>> Exactly.
>> You can share your links here. Um and uh
of course you can take a look at the
leaderboard which again I just not going
to brag. I'm number one.
>> Somebody's got to be Jonathan. That's my
ask to the world. Go be Jonathan.
>> Then we have uh we have a few other
users. My wife by the way 5.5. I got to
tell 5.5 is a phenomenal score.
>> Yeah. This isn't a hard test, isn't it?
Like I I I made this one brutal.
>> Th This is So you you you see these
numbers like 8.7, 8.9, 8.6. These are
some test users that have like
>> hardcore been in AI for
>> Yeah.
>> You know, since the day Chat GPT was
released, you know, they
>> uh and they're they're crazy. Michael
Dion's a good developer friend of mine.
Um and I just met him, so he's he's good
people. But uh yeah uh and my wife like
I was shocked that she got a you know
the average person gets like a 1.4 to a
two.
>> Yeah.
>> Uh somebody who's been using chat GBT
and stuff just casually and stuff like
that and actually trying a little bit is
more around a three. 5.5 is ridiculous.
>> It's really Yeah. Yeah. No. And I'm sure
that like people will come at me but
like when I wrote the fluency algorithm
I was like I don't care about grade
inflation. I know it's a thing. We're
not doing it here. If you earn your way
to whatever score, it's a real score.
It's hard. You should be proud.
>> Yeah, for real. And it it's absolutely
true. I I um So,
uh last couple things. Uh this is the
homepage
>> right now. We have the leaderboard right
there on the homepage. The top 10 people
are going to be here
>> on there. That's right. That's the
challenge.
>> Yep. And uh and then you can search for
your friends. Um
so
>> oh you can search for I didn't even know
this. This is so cool. So you can see
>> so the only people who show up on the
leaderboard are people who have actually
gone through the evaluation. So here
this is this is my son's page. He has
not gone through the evaluation. Uh but
he was
>> his page. Yeah.
>> Yeah. He sent me so many errors and bug
fixes and he's, you know, uh,
>> I'm the god of finding bugs. I love
this.
>> Yeah. So, and of course he linked to his
TikTok here.
>> Yeah. So, coming back to the build
story, how did you build this with AI?
Tell us a little bit about that. You can
pull the screen down if you want. We can
just chat.
>> Um, or you can show what you're
building. Like that's also all totally
cool.
Well,
I used every tool known to man.
So, so I started
um I started working uh
it's just using clawed code and uh and
then jumping from system to system to
system and uh you know finding out what
work like for example codecs like I I
hate using codeex but I love using it to
review what Claude did because Claude
is a little bit silly sometimes. And
>> tell me more about that. A lot of people
are really keen for the like hands-on
Claude versus Codeex comparison. I think
you have an opinion there. Dig into that
for me.
>> I have a strong opinion there. Like I
get um so uh Codeex Codeex straight up
ignores you when you ask it to use
tools. Period. Okay. Um, and yeah, like
well, I mean, you do have to prompt it
in a certain way, but you have to do it
every single time. Like, hey, use this
tool to do this job. By the way, you do
have access to this tool. This is how
you use this tool. And all that has to
be in your prompt. And that drives me
insane. So, I just use Claude, you know,
I just use codeex for code review
because it's phenomenal at that. It
it'll go through
>> Yeah. that, you know, it it'll help me
plan. It'll help me check the plans.
it'll find all the faults in the plans.
But then I have Claude execute cuz you
know Codeex is lazy about execution in
in my opinion, you know, and you could
prompt your way out of that. But I don't
know. I just I I just really like the
experience with Claude. I had a few days
with Gemini 3 last week that blew me
away and then Opus 4.5 came out and I I
haven't even thought about use, you
know, using Gemini 3 again. Uh because
it has its quirks. Gemini 3 scares the
hell out of me. Um, you know, when you
when you're watching it
uh its thought chain, you know, the it
it starts off it always starts off like
speaking in the first person for some
reason, like repeating your prompt in
the first person like, hey, I'm really
focused on this and I'm really focused
on this and then it'll say the wrong
thing
>> and then it'll dig into saying the wrong
thing
>> and you're like desperately trying to
pull it out of the mud in your head and
you're like and then maybe it
selforrects but you don't know.
>> Yeah. Then then you right as you go to
click cancel it's like
>> it it'll it'll be like no no no I'm
wrong I should be doing this. They're
like oh okay thank god
>> because it was just about to touch
something you didn't want it to touch.
And uh
>> I think that happens with AI more than
we realize. And Gemini made a very bold
choice to expose that because I think
that behind the scenes there is a lot of
what I would call temporary
misinterpretation. Like with Claude for
example, I often see if it's if it's
streaming the train of thought, it will
say something absolutely nasty to me.
Like I'm thinking about the user's
underdescribed prompt or something awful
like that
>> and I'm like, "What do you mean? This
thing is like a 50line prompt and it's
because it hasn't opened the prompt up
and it's just kind of like thinking
>> and then eventually it gets into it and
opens it out."
>> Yeah. I I'll do my standard like, you
know, here's a list of things, you know,
I'll just, you know, I'll just use
whisper flow and just talk to it for
like five minutes and then I'll do the
standard like, hey, please explain to me
what you think I'm talking about and
wait for me to confirm or or clarify.
And it will it'll do that whole thing.
It'll make this huge list and it'll be
like and I'll clarify or whatever. But
if it's right, I'll I'll tell it, "Yeah,
please proceed." It be like in the first
chain of thought is like what is the
user talking about? This is super vague.
And I'm like what?
>> And yet from there we get to you're
absolutely right with claude
>> every time. Yeah. As soon as you see
you're absolutely right, you should
clear your context immediately. Like
that go away. It just it's it's the sign
to run away.
>> That's a bad omen.
>> Okay. So you're using cloud code, you're
using codeex. Uh any other vibe coding
tools? Uh the prototype of this app I
built in lovable. Um and
then of course of course the minute Nate
and I talked he's he's like
I want it in next.js because it was in
view and uh
>> so Ben no longer has any lovable
heritage in it.
>> Yeah, we did pull it back out. Yeah.
Yeah. the uh he made me refactor the
whole thing.
>> We did. We had to ref I'm sorry about
that. We had to refactor it. Um but this
does call out something really cool I
don't think people realize. One of the
things that's cool in the age of AI is
that it's easier to build things with
fewer meetings. So this is the first
time Jonathan and I are having a meeting
and we are launching tomorrow.
>> Yeah. Usually it's me posting like
document after document after document
and Nate saying,
>> "Yeah, this is fine. I hate this. That's
good. And then I'll hear from him two
days later, you know.
>> Yeah. Yeah. I'm like I'm like one of
those slow inference LLMs. I eventually
prickle back around hopefully with a
high quality response.
I I always ladder back to sort of why we
do this and and I think one of the
things that
really excites me about AI cred is
we haven't had any kind of product space
where we can have a conversation about
what overall fluency feels like and I am
sure that AI cred will continue to
evolve as folks give us feedback. One of
the things I do want to call out on that
note is that Jonathan and I are going to
be opening up a Slack channel into our
work Slack. So if you sign up for AI
cred, you are going to be invited to
join us in the work Slack and give us
feedback directly. Um, so you can just
ping us and say, "Hey, I got this weird
response and I want this fixed or I have
this cool idea for the leaderboard or
whatever it is. Maybe it's a bug and we
can just get right on fixing it." uh
because I think that one of the things
we want to model is that AI tooling
evolves and so we build AI cred but
we're building it for a space that's
evolving and so AI cred will evolve to
keep pace with how learning continues to
need to grow in the age of AI. Yeah,
there just to double down on that,
definitely join the Slack. I I I
implemented this whole bug report
system. You could use that. I did rate
limited at two per hour. I don't want
people spamming me, but
um but definitely join the Slack. Uh
this this is going to like oh man, in
two months you're going to hear all
kinds of updates that are implementing
really really cool features. There
there's
a lot of stuff that I don't want to talk
about now because it probably, you know,
you never know if it'll actually happen
and I want to get your open
>> there there's so many cool things
coming. This is just going to constantly
improve and and yeah, we definitely need
people's input. Like, you know, my
12-year-old son is the reason the
profile pages are actually going to look
good. So
>> everyone like
>> yeah your feedback is super important.
>> Um and of course uh for folks this came
out of the substack community. So we
want to give back to the Substack
community. So if you are a member of the
Substack you're going to get a screaming
Black Friday discount on this. Um and
that's because this is one of the
products that got born out of the chat,
right? Like we want to make sure that it
feels like it's part of the community.
Um
>> don't join the Substack. Sorry D. Don't
join the Substack to get a discount on
this. Join the Substack because like
genuinely
the people the people that I've met in
this space, the people that have helped
me like that.
>> Shoot. We have Shank who's
>> Yeah.
>> who who's
a huge head engineer. I I don't even
know. I can't even pronounce some of
their titles. like these people are like
top level
engineers and software developers and
people who run actually large companies
and you know you know Nate's Christmas
tree farmer from earlier you know like
there are so so many just genuinely
absolutely amazing people um and then
there's me but yeah um join anyway
>> yeah no I
>> it was a weird attempt at
self-deprecation but Yeah, I don't know.
The community is a reason. Like I and I
I am genuinely shocked. Like it's
something I really appreciate is that
this community has just grown up
organically and now I just watch the
chat and I'm like it's not me answering
everything, right? It's people
responding to each other which is always
what you want with a community because
it's like this many to many connection
and it feels really strong. So I've just
I love that.
>> Yeah. Yeah. I I totally agree. And uh
definitely
sign up for AI cred, get your fluency. I
want to see I have an 8.9 right now. I
I'm pretty sure you can't get more than
a nine.
>> First evaluation. So it's is it might be
a little tough to beat me, but
>> h someone needs to go beat John. I want
to see the leaderboard tomorrow. All
right. Thank you, Jonathan. This has
been great.
>> Same. All right. Cheers.