When Smarter Bots Aren’t Enough
Key Points
- The rapid advances in AI are driven mainly by ever‑larger pre‑training datasets and improved inference reasoning introduced with the 01 model in late 2024, but these gains are still largely narrow and domain‑specific.
- Despite massive data consumption and billions of user interactions, the finite quality of available data and concerns over token “learning” value are prompting companies like Anthropic to restrict first‑party model access.
- Even if pre‑training and inference challenges were solved, the current approach still falls short of handling tasks that require long‑term intent, broad contextual awareness, and the ability to track multiple, simultaneously changing variables.
- CEOs’ promises of fully autonomous agents capable of creating full‑length movies or replacing professional colleagues hinge on achieving generalized, not just incremental, intelligence—a milestone that remains elusive.
Sections
- Smarter Bots, Still Narrow Capabilities - The speaker questions whether continual improvements in large‑scale pre‑training and inference can fulfill ambitious promises of fully autonomous creative and professional agents, arguing that current AI progress remains narrowly focused and constrained by limited high‑quality data.
- AI Limits in Multi‑Reward Marketing - The speaker argues that even highly intelligent AI cannot handle marketing tasks that involve evolving, partial-information environments with multiple, conflicting long‑term rewards, requiring human intuition and fuzzy reasoning to balance outcomes.
- Beyond Pre‑training: Jagged AGI Future - The speaker urges consideration of missing technical breakthroughs beyond data and inference, arguing for a “jagged” scenario where some AGI advances arrive while others lag, and asks what obstacles still block true artificial general intelligence.
Full Transcript
# When Smarter Bots Aren’t Enough **Source:** [https://www.youtube.com/watch?v=5PasrHSrato](https://www.youtube.com/watch?v=5PasrHSrato) **Duration:** 00:09:06 ## Summary - The rapid advances in AI are driven mainly by ever‑larger pre‑training datasets and improved inference reasoning introduced with the 01 model in late 2024, but these gains are still largely narrow and domain‑specific. - Despite massive data consumption and billions of user interactions, the finite quality of available data and concerns over token “learning” value are prompting companies like Anthropic to restrict first‑party model access. - Even if pre‑training and inference challenges were solved, the current approach still falls short of handling tasks that require long‑term intent, broad contextual awareness, and the ability to track multiple, simultaneously changing variables. - CEOs’ promises of fully autonomous agents capable of creating full‑length movies or replacing professional colleagues hinge on achieving generalized, not just incremental, intelligence—a milestone that remains elusive. ## Sections - [00:00:00](https://www.youtube.com/watch?v=5PasrHSrato&t=0s) **Smarter Bots, Still Narrow Capabilities** - The speaker questions whether continual improvements in large‑scale pre‑training and inference can fulfill ambitious promises of fully autonomous creative and professional agents, arguing that current AI progress remains narrowly focused and constrained by limited high‑quality data. - [00:04:25](https://www.youtube.com/watch?v=5PasrHSrato&t=265s) **AI Limits in Multi‑Reward Marketing** - The speaker argues that even highly intelligent AI cannot handle marketing tasks that involve evolving, partial-information environments with multiple, conflicting long‑term rewards, requiring human intuition and fuzzy reasoning to balance outcomes. - [00:08:14](https://www.youtube.com/watch?v=5PasrHSrato&t=494s) **Beyond Pre‑training: Jagged AGI Future** - The speaker urges consideration of missing technical breakthroughs beyond data and inference, arguing for a “jagged” scenario where some AGI advances arrive while others lag, and asks what obstacles still block true artificial general intelligence. ## Full Transcript
What if the bots kept getting smarter,
the AI kept getting smarter, but it
didn't matter anymore? That is the
question that has been keeping me up at
night. And I want to talk about it.
There's lots of ways we can talk about
this. The simplest way is to say this.
The bots have been getting smarter
because of two key things. One is very
large pre-training data sets and the
other is smart inferencing which was
introduced in late 2024 with the 01
model. Now lots of people have it. So
here's the question. If we just have
pre-training and we just have inference
and reasoning, is that enough to make
all of these big promises that the CEOs
are making come true? Are is it enough
for us to make fulllength movies? Is it
enough for us to have agents at work
that are just like our professional
colleagues and can do all of our work
for
us? And increasingly, as we see the
successive generations of these bots, we
see 03 come out, we see Gemini 2.5 Pro
come out, what I see is that these bots
are getting smarter, but they're getting
smarter in what I would call narrow
ways. They're smarter at specific
things, but they're not generally smart
in ways that will enable us to do this
generalized work if we just keep getting
better at inference or pre-training
data, which by the way has its own
questions because, of course, there's
not infinite data in the world. We've
used a lot of it. The remainder may not
be as high a quality. There's there's
questions about it. Now, you can say,
"Chat GPT has access to a lot of data
from usage now. They have almost a
billion users. they can use that to
refine their models. That by the way is
why Enthropic has decided to cut model
access to Windsurf as much as they can
because Windsurf was purchased by OpenAI
and Enthropic has basically said those
tokens could now be used for learning by
OpenAI. We don't want that. Uh you will
have to get thirdparty access to your
cloud models. We're not providing
firstparty access anymore because those
learning tokens are like gold right now.
Okay, fine. So let's say just for a
second we solve any questions around
pre-training. We have the data which may
be true like we may be able to scale for
a bit longer anyway. Uh and let's also
say uh that we have reasoning figured
out we can get better and better at
inference. Even then is that really
enough for doing tasks that require
months of intent? Is it really enough
for what I would call widen changing
context understanding where you are
aware of a very broad work context or
personal context and two or three
elements are changing at once during a
day or a week and you can track all of
that context change and the fuzzy logic
implications. Like for example, you are
trying to hit a sales target and you are
aware of the three or four things in
product and in finance and in customer
success that are all affecting how your
deals are coming together and you are
able to process all of that and then
package that up in a way that is useful
on conversations with
prospects. Humans are really good at
that
stuff. AI, even really smart AI, is not
as good as it needs to be. And part of
why is that when you have widely
changing contexts like that, you need an
AI that sort of learns from experiences
that it has in the field after
deployment, not just from experiences
that it has in
pre-training. Chad GPT is making a nod
at that with memory, but memory is not
close to being at a point where you
could say it adaptively learns on the
fly to these very wide context changes,
then can track it at high fidelity. It
just isn't. And I think that I made a
video like a few months ago that I think
was vaguely popular that basically said
we have a memory problem. I would go
broader than that. We have a context
awareness and adaptability
problem. We also have a problem with
intent over time where these things have
to have goals. We also have a major
problem with how we handle tacet
knowledge in the workplace which I've
talked about extensively on other
videos. How do you handle knowledge that
is never spoken because there are social
consequences to speaking that for
humans? The AI never sees it. It's
invisible. What do you do with that?
Even if you have infinitely smart AI,
that won't help. How do you handle tasks
that evolve at the edges to be
successful? A great example of this um
is if you are trying to do marketing,
you have multiple different rewards
you're optimizing for. or it's not just
one clear reward like you have with code
where it runs or it doesn't. Uh the
relationship between those different
rewards in the funnel is unclear and
varies dramatically by
business. Once you optimize for one of
them, you risk deoptimizing for another
one and you have to keep your eye on the
end result with a business and like the
value and the long-term customer value
you're driving that you don't see for
months if if not years. Some deal cycles
take
years. And so marketers have to adapt to
this extremely changeable partial
information environment and also the
novelty that customers are looking for
and new tactics that
emerge. AI is not good at that. That's a
that's an adaptable context problem. It
is also a change the task at the edges
and tweak it in ways that enable you to
account for multiple partial rewards
that the human is accounting for with
some kind of fuzzy logic and intuition.
and what we call intuition. We don't
really have intuition for
AI. The AI may feel intuitive sometimes,
but at the end of the day, it is coming
to a conclusion based on the
reinforcement learning it's had, based
on the previous interactions with you,
if it has some kind of memory, and based
on inference. That's what you got.
And so I do think that we are
underestimating the number of technical
breakthroughs that we would have to have
to really get into a place where these
visions come true. And the critical
thing is because we're not talking about
it because we're only talking for the
most part about how amazing A is at
inference AI is at inference and at
pre-training data which is great. It
does magical things that the the rocks
have begun to think. I'm not
complaining.
But even if we had all that and even if
we had smarter and smarter AI, if we
don't solve the problems I'm describing
with adaptable context, intent over
time, the ability to sort of make
optimizations across multiple partial
rewards, we're going to be in trouble
from a perspective of wanting all of
this AI magic to come true. Now, I am
not clear that most of us actually want
that future. So I I am under no
illusions, but that is the goal that a
hundred billion dollars in capital is
chasing right now. And from a
riskmanagement perspective, they all
think the other one might have a
breakthrough. And so they've got to keep
chasing it because if someone gets that
breakthrough, it's going to be lots and
lots of money on the table. And so
that's why all of that money has like
coalesed and is chasing that goal.
But no one is reporting on or talking
about the other breakthroughs and the
kinds of breakthroughs we need to
actually get to this vision of a fully
functional AI
colleague. And if we don't talk about
it, one, we're sort of not putting
sunlight on AI companies and model
makers and what they're working on. And
I think we should be. It's a very
important initiative. We should be
talking about what they do. Uh, and two,
if we can't name the stuff, we can't
describe what we want or do not want as
users, as builders, as
consumers because we can't understand
it. And so I I actually would love us to
be able to have a conversation where we
are able to say these are the things
that are standing in the way of this
broader vision. This is what I would be
interested in. This is what I would not
be interested in. This is where I want
to go. This is where I don't want to go.
This is the kind of product I want to
build. This is the kind I don't want to
build. Having a specific preference
makes a big difference.
And I am begging us to think beyond
pre-training data and inference and have
a conversation about the larger gaps
because I think there is a real chance
that we will live in a world where we
get really smart models at inference and
pre-training but those other technical
breakthroughs are not inevitable and
maybe we don't get to them for a while
for a decade for two decades for three
decades ever. We don't know. They're not
inevitable. And so I want us to think
more about a jagged future. What does it
look like when we have some
breakthroughs toward artificial general
intelligence, but they don't all
materialize or at least not in the same
timeline? I don't know. You tell me.
What is standing in the way of AGI?