OpenAI Diminishing Returns Claim Sparks Defense
Key Points
- A report by *The Information* claimed OpenAI’s mid‑training “20 % finished” model (rumored to be GPT‑4.5) showed only marginal improvements, suggesting diminishing returns on larger language models.
- OpenAI’s leadership, including the VP of product, and many external AI experts publicly disputed the claim, saying the article confused raw model scaling with the reasoning abilities demonstrated by the upcoming GPT‑4o model.
- GPT‑4o’s “reasoning tokens” enable it to iteratively explore, correct, and refine its answers at inference time—a capability that markedly outperforms earlier models like GPT‑4 on complex, under‑specified tasks such as finance calculations.
- The controversy underscores a broader industry question: while raw scaling may hit diminishing returns, value can still be gained by improving inference‑time reasoning, a strategy OpenAI has emphasized in its roadmap.
Full Transcript
# OpenAI Diminishing Returns Claim Sparks Defense **Source:** [https://www.youtube.com/watch?v=Ey811OUpX70](https://www.youtube.com/watch?v=Ey811OUpX70) **Duration:** 00:07:38 ## Summary - A report by *The Information* claimed OpenAI’s mid‑training “20 % finished” model (rumored to be GPT‑4.5) showed only marginal improvements, suggesting diminishing returns on larger language models. - OpenAI’s leadership, including the VP of product, and many external AI experts publicly disputed the claim, saying the article confused raw model scaling with the reasoning abilities demonstrated by the upcoming GPT‑4o model. - GPT‑4o’s “reasoning tokens” enable it to iteratively explore, correct, and refine its answers at inference time—a capability that markedly outperforms earlier models like GPT‑4 on complex, under‑specified tasks such as finance calculations. - The controversy underscores a broader industry question: while raw scaling may hit diminishing returns, value can still be gained by improving inference‑time reasoning, a strategy OpenAI has emphasized in its roadmap. ## Sections - [00:00:00](https://www.youtube.com/watch?v=Ey811OUpX70&t=0s) **Industry Defends OpenAI Amid Leak** - Experts rebuff a report that OpenAI’s next model (tentatively GPT‑4.5) shows only marginal gains, arguing the claim confuses large‑language‑model scaling with true reasoning capabilities. ## Full Transcript
we have three really big pieces of AI
news today number one over the weekend a
publication called the information broke
the news that open AI was seeing
diminishing returns on their work with
AI and this was based apparently on
Anonymous conversations with openai
employees and specifically on the
results of the 20% finished model
training run for the 01 model the
speculation is the 20% Mark is roughly
where GPT 4.5 would be and what the
leakers suggested is that this was only
like marginally better not necessarily
better at all at code
Etc well a whole bunch of people and not
just from open AI spent the weekend
pushing back I mean you expect the
leadership of open a to push back right
the VP of product basically said this is
not true without directly addressing it
and what I thought was interesting is it
wasn't just the open AI top brass it was
also a bunch of other folks in the
industry who really are only thinking
about what is the correct solution for
artificial general intelligence and do
we hit a wall here that would prevent us
getting there and so the information
broke earlier in the weekend and by
Sunday it was evident
that most of the people who were
well-versed in Ai and who are not at
open AI were coming to open ai's defense
and essentially saying you know what the
information got this one wrong and the
reason they got it wrong is that they're
confusing large language models with the
reasoning that 01 has simply because 01
presents as a reasoning model so when
we're using one preview we type it into
the same chatbot window but it does
reasoning at the time of inference so
when you type it in it can go and
explore different chains of thought and
reverse and change itself reverse and
change itself reverse and change itself
and use what are called reasoning tokens
to eventually get to the correct answer
if you ever want to see how that works
I'll give you a really simple prompt for
that just ask it to do a fairly
complicated discount flow analysis in
finance or ask it to do a uh compound
annual growth return in finance it knows
those formulas but if you don't give it
too much information it has to infer a
lot and so it will and you will actually
see it sometimes correct itself and
you'll see it eventually lay out a
really clear methodology for how it
solves the problem if you give that same
problem to 40 it will not work nearly as
well in fact I did that last night and
what I saw was clearly an actual
rational response from 01 and clearly
much more simple next token prediction
from
40 and so I think what the open ai's vpa
product said got it right at the end of
the day the question of whether an llm
is hitting a diminishing rate of return
on training on data is
different than whether or not you are
getting more value out of reasoning and
open AI has been telegraphing for months
that they a lot of the value in scaling
intelligence as scaling at the time of
inference or scaling
reasoning so we will see but it just
looks like the information got very very
excited about reporting something that
they thought was scandalous and
surprising and rushed it to print a
little bit too much because it gave the
impression that open AI was out of juice
and the reality is open AI is just at
the beginning of releasing something
interesting and I actually think that is
one of the things that makes this leak
seem a little bit malicious 01 the full
01 model is rumored to be released this
week now I've been doing this for a bit
I've been seeing these rumors come out
so that's not that surprising that
they're rumoring it this week maybe it
will happen this week maybe not but it's
supposed to happen by the end of the
year we're coming up into the teeth of
the end of the
year and it is very interesting that
they would choose to play for clicks in
this way right now all right I've said
enough about that number two the US has
ordered
tsmc to Halt export of roughly speaking
AI chips to China if you want to dig
into the details it's uh chips below the
7 nanometer architecture specifically
going for gpus used in AI
applications and you might wonder I
wondered how on Earth is the United
States government able to restrict
exports from Taiwan to China it doesn't
make sense well it turns out that we
have a special set of rules called the E
the export
administrative something I'm going to
have to look this up a second it's
the uh
export
administrative rules that are authorized
by the export control Reform Act of 2018
I did my research on this um and what
basically what it does
is it allows you to if you are the US
government regulate the export from
other countries of things that are
produced using US tech and in this case
the chips have us design input a lot of
the sort of routine in tsmc is you get
the chip designed from the US and then
you manufacture it in Taiwan on that
basis of the intellectual property and
the chips the Chamber of com the US
Department of Commerce is a able to
regulate export of the chips from Taiwan
and that's what they're doing and the
reason they did it and this is not as
widely
reported is that a chip that had gone to
a Chinese customer which was legal at
the time ended up in a huahe device
which is not legal because the US has
put huah on a export restrictions list
and so basically seeing that like
there's some unlicensed transfer of
these chips inside China the Department
of Commerce basically said no just stop
exporting to China until we figure this
out and so that's what as of today tsmc
is doing now
separately people are wondering how
tariffs if they come will affect AI chip
production and one of the plays for a
lot of other materials is that people
ask themselves can we do final assembly
in the
US in this case Taiwan has sent a very
clear signal that you won't be able to
because it is said explicitly that they
have regulations on the books in Taiwan
that say that advanced chip
manufacturing capabilities in this case
the current top generation is 2
nanometer that they will not be able to
make them elsewhere because they have
regulations saying that capability stays
on Taiwanese soil so we will see we have
some competing
regulations and in the meantime I'm
hoping that these weird photos that open
AI employees continue to post about the
Orion Nebula end up meaning something
right like surely they can't just post
pictures of the stars and not release
models we will see