AI Safety Trends and France’s $100 B Fund
Key Points
- Experts offered mixed opinions on AI safety over time, with some noting it’s becoming safer, especially due to growing open‑source initiatives.
- This episode of *Mixture of Experts* will discuss test‑time scaling, Sam Altman’s latest blog post, and Anthropic’s new Economic Index.
- The Paris AI Action Summit, organized by the French government, gathered civil society, industry, and policymakers to develop AI standards and guidelines.
- President Macron announced a €100 billion AI fund—supplemented by private investors like Iliad’s €4 billion—to bolster sovereign European AI infrastructure.
- Pleias, co‑founded by Anastasia Stasenko, will leverage this funding to help build and scale European AI capabilities.
Sections
- AI Safety Trends and Perspectives - Panelists quickly debate whether AI is getting safer over time, and the host previews upcoming topics—including test‑time scaling, Sam Altman’s blog, Anthropic’s Economic Index—and highlights the Paris AI Action Summit.
- GPU Limits, Open AI Funding - The speakers discuss GPU shortages and environmental concerns while introducing the "Current AI" public‑good foundation, which aims to raise billions to support open‑data, open‑source AI models for broader, responsible applications.
- AI Safety, Democratization, and FOMO - The speaker argues that leaders’ fear of missing out fuels rapid AI rollout, making safety increasingly complex as powerful models become more democratized, offering both advantages and unforeseen harms while regulatory frameworks lag.
- Building Foundations for Factual AI - The speaker stresses prioritizing curated, multilingual databases and small retrieval‑augmented models to ensure safe, factual AI, noting current deficiencies in toxicity classifiers and the lack of AGI ambitions.
- Rapid Replication of Test‑Time Compute Trick - The speaker notes that the swift, low‑cost recreation of a test‑time compute method—leveraging longer chain‑of‑thought generation, rejection sampling, and a single weight token—is anticipated rather than surprising.
- Prioritizing Quality Over Quantity - The speakers contend that focusing on smaller, high‑quality, representative datasets—rather than massive noisy corpora—reduces computational waste and drives more effective model progress.
- Legal Showdown of AI Models - The speakers imagine a future courtroom where litigants use tiny versus gigantic AI models—favoring a pluralistic, capitalistic AI landscape over a single governing AGI—before transitioning to discuss Sam Altman’s recent “Three Observations” blog post.
- Debating Hype vs. Realism - A dialogue where one speaker questions exaggerated claims about a technology’s impact, while another expresses enthusiastic optimism about its world‑changing potential.
- AI as Emancipatory Tool vs AGI Ambitions - The speaker argues that AI assistants can democratize writing and coding for marginalized users, warning that the pursuit of AGI diverts focus from this empowerment and won’t solve looming ecological crises.
- Rethinking AI Scaling and Energy - The speaker argues that AI advancement should prioritize test‑time compute and consumer‑grade hardware, challenging the notion that massive data centers or nuclear power are required for scaling.
- Anthropic Finds AI Use Still Tech‑Centric - Anthropic’s analysis shows that about 36% of AI assistant usage remains concentrated in software development and technical writing, defying expectations of broader consumer applications.
- AI’s Coding Dominance Over AGI - The speakers argue that despite hype about AGI, current powerful AI systems like Claude are predominantly valued for coding assistance, making usage data appear skewed toward that niche.
- AI as Platform, Not Just Tool - The speaker argues that AI should evolve from isolated tools to integrated platform signals, emphasizing the need to discover more practical use‑cases and educate users before it becomes mainstream.
Full Transcript
# AI Safety Trends and France’s $100 B Fund **Source:** [https://www.youtube.com/watch?v=KNNi-enb2B4](https://www.youtube.com/watch?v=KNNi-enb2B4) **Duration:** 00:39:45 ## Summary - Experts offered mixed opinions on AI safety over time, with some noting it’s becoming safer, especially due to growing open‑source initiatives. - This episode of *Mixture of Experts* will discuss test‑time scaling, Sam Altman’s latest blog post, and Anthropic’s new Economic Index. - The Paris AI Action Summit, organized by the French government, gathered civil society, industry, and policymakers to develop AI standards and guidelines. - President Macron announced a €100 billion AI fund—supplemented by private investors like Iliad’s €4 billion—to bolster sovereign European AI infrastructure. - Pleias, co‑founded by Anastasia Stasenko, will leverage this funding to help build and scale European AI capabilities. ## Sections - [00:00:00](https://www.youtube.com/watch?v=KNNi-enb2B4&t=0s) **AI Safety Trends and Perspectives** - Panelists quickly debate whether AI is getting safer over time, and the host previews upcoming topics—including test‑time scaling, Sam Altman’s blog, Anthropic’s Economic Index—and highlights the Paris AI Action Summit. - [00:03:09](https://www.youtube.com/watch?v=KNNi-enb2B4&t=189s) **GPU Limits, Open AI Funding** - The speakers discuss GPU shortages and environmental concerns while introducing the "Current AI" public‑good foundation, which aims to raise billions to support open‑data, open‑source AI models for broader, responsible applications. - [00:06:11](https://www.youtube.com/watch?v=KNNi-enb2B4&t=371s) **AI Safety, Democratization, and FOMO** - The speaker argues that leaders’ fear of missing out fuels rapid AI rollout, making safety increasingly complex as powerful models become more democratized, offering both advantages and unforeseen harms while regulatory frameworks lag. - [00:09:16](https://www.youtube.com/watch?v=KNNi-enb2B4&t=556s) **Building Foundations for Factual AI** - The speaker stresses prioritizing curated, multilingual databases and small retrieval‑augmented models to ensure safe, factual AI, noting current deficiencies in toxicity classifiers and the lack of AGI ambitions. - [00:12:20](https://www.youtube.com/watch?v=KNNi-enb2B4&t=740s) **Rapid Replication of Test‑Time Compute Trick** - The speaker notes that the swift, low‑cost recreation of a test‑time compute method—leveraging longer chain‑of‑thought generation, rejection sampling, and a single weight token—is anticipated rather than surprising. - [00:15:30](https://www.youtube.com/watch?v=KNNi-enb2B4&t=930s) **Prioritizing Quality Over Quantity** - The speakers contend that focusing on smaller, high‑quality, representative datasets—rather than massive noisy corpora—reduces computational waste and drives more effective model progress. - [00:18:36](https://www.youtube.com/watch?v=KNNi-enb2B4&t=1116s) **Legal Showdown of AI Models** - The speakers imagine a future courtroom where litigants use tiny versus gigantic AI models—favoring a pluralistic, capitalistic AI landscape over a single governing AGI—before transitioning to discuss Sam Altman’s recent “Three Observations” blog post. - [00:21:49](https://www.youtube.com/watch?v=KNNi-enb2B4&t=1309s) **Debating Hype vs. Realism** - A dialogue where one speaker questions exaggerated claims about a technology’s impact, while another expresses enthusiastic optimism about its world‑changing potential. - [00:24:52](https://www.youtube.com/watch?v=KNNi-enb2B4&t=1492s) **AI as Emancipatory Tool vs AGI Ambitions** - The speaker argues that AI assistants can democratize writing and coding for marginalized users, warning that the pursuit of AGI diverts focus from this empowerment and won’t solve looming ecological crises. - [00:28:00](https://www.youtube.com/watch?v=KNNi-enb2B4&t=1680s) **Rethinking AI Scaling and Energy** - The speaker argues that AI advancement should prioritize test‑time compute and consumer‑grade hardware, challenging the notion that massive data centers or nuclear power are required for scaling. - [00:31:08](https://www.youtube.com/watch?v=KNNi-enb2B4&t=1868s) **Anthropic Finds AI Use Still Tech‑Centric** - Anthropic’s analysis shows that about 36% of AI assistant usage remains concentrated in software development and technical writing, defying expectations of broader consumer applications. - [00:34:35](https://www.youtube.com/watch?v=KNNi-enb2B4&t=2075s) **AI’s Coding Dominance Over AGI** - The speakers argue that despite hype about AGI, current powerful AI systems like Claude are predominantly valued for coding assistance, making usage data appear skewed toward that niche. - [00:37:50](https://www.youtube.com/watch?v=KNNi-enb2B4&t=2270s) **AI as Platform, Not Just Tool** - The speaker argues that AI should evolve from isolated tools to integrated platform signals, emphasizing the need to discover more practical use‑cases and educate users before it becomes mainstream. ## Full Transcript
Globally speaking, is AI getting
more or less safe with time?
Marina Danilevsky is a
Senior Research Scientist.
Marina, welcome back to the show.
What do you think?
Both.
Okay, all right, we'll get into that.
Uh, Chris Hay is a Distinguished Engineer
and CTO of Customer Transformation.
Uh, Chris, what do you think?
More or less safe with time?
I think it's getting more safe with time,
so I'm, I'm pretty pleased with that.
Um, and joining us for the very first
time is Anastasia Stasenko, who is
the CEO and co-founder of pleias.
Uh, Anastasia, more or less safe?
Uh, it's definitely getting safer
because it's getting more open source.
All right, great.
More to dive into.
All that and more on today's Mixture of Experts.
Greetings from Paris.
I'm Tim Hwang, and welcome
to Mixture of Experts.
Each week, MoE is the place to tune
into to hear how leading researchers,
engineers, entrepreneurs, technologists,
and many more are thinking about the
latest trends in artificial intelligence.
As always, we have a lot to
cover, way too much to cover.
Um, we're going to talk
about, uh, test time scaling.
We're going to talk about a
new blog post from Sam Altman.
We'll talk about Anthropic's new Economic Index.
But first, uh, given that I'm in Paris
and Anastasia is in Paris as well, we want,
to talk about the Paris AI Action Summit.
So this is the latest of addition of a series
of summits that governments been, have been
holding, uh, around AI in the last few years.
Uh, this year is hosted
by the French government.
Um, and it collects representatives
from civil society, governments,
companies and, and more, to talk
about and set sort of standards and
guidelines around the development of AI.
And there are some really big
announcements that we wanna get into.
Anastasia, it's really
exciting to have you on the show.
In part because you were sort of
directly involved in some of these events.
I understand basically that
Macron has announced a enormous, I
think it's like a 100 billion
dollars fund, to support AI and
support AI specifically in France.
And, I know pleias is, is part of
that, but if you want to tell us just a
little bit more about, how you got
involved in and what you're going to
be doing with this new fund.
Yes, of course.
Well, so first of all, there actually have
been multiple announcements about investment.
We do love announcing investment in France.
We do hope that the real action will follow.
However, what's important to say is
that 109 billion investment fund is
actually not only of France investing.
This is, an international
and, have quite, there are
some private companies involved.
For example, Iliad Group, which is a
free, mobile company, basically,
giving over 4 billion, et cetera.
And this fund, has the objective
to really focus on sovereign
European AI infrastructure, right?
So this is one part, of this and
it's true that we have been, we have
been listening, we have been hearing for
a long time that Europe is lagging behind
in terms of AI infrastructure, right?
We don't have, enough GPUs to basically
neither train, frontier models,
nor to then run inference, for
actually scaled AI applications, right?
I'm not sure that this is entirely true, and
I'm not even sure that we should be really going
into scaling the AI infrastructure in the world
where we actually have ecological imperatives
kind of haunting us at the same time.
another big one was the announcement
of the Current AI, which is,
actually AI for public good foundation.
Yeah, I know and this is like a very
particular part of pleais' work, right?
Because I know you guys are specifically
working on sort of open source
and open data models, ultimately.
Yes, totally and we have been, we have
actually trained the world first models,
on exclusively open data, open in the strong
sense of this word without copyrighted
material with permissive licenses only and
it's true that opening data and creating
this open data infrastructure is something
that is important to us, but actually is
important to larger AI communities, which
is of today actually cannot advance as
fast or work with as many applications for
the good of the communities, which are
Even think about more resource languages,
think about more specific applications
without actually being supported by
the initiative, such as Current AI.
So this foundation with already 400
million secured for the first year
aims to actually raise over $2 billion
for their five year, five year run.
At least as of now.
And yeah, we are very happy to be
part of this from the very beginning.
We have signed the open letter with 10
other industry leaders, such as Mistral,
ALEPH ALPHA, Hugging Face, et cetera, et cetera.
So it's all very exciting.
And I'm most particularly excited about
data finally not being like kind of
the forgotten piece, of this AI,
AI hype and like not only hype, but
like, AI development in general.
For sure.
Yeah.
And I think it's something that we talk a
lot about on the show is just how much like
the data piece often gets lost, even though.
Arguably, it's like the most
important part of the whole thing.
One of the themes I did want to
pick up on,you know, Politico
did this interesting article.
The headline I'll just read here was,
quote, "How the world stopped worrying
and loved to, learned to love AI.""
And sort of the argument of the article
was that, you know, the last few
summits had really focused on, like,
safety and security around the model.
But in contrast, I think this year,
there was just a lot more kind of lean
forward, like, we just need to deploy
this faster and better and bigger.
Than ever before and I guess Marina,
maybe I'll call on you because I know in
response to the first question you said,
well, is it getting safer or less safe?
Well, maybe a little bit of both.
I'm curious about kind of what you meant
by that and how you think about it.
Um, particularly in this context where it
feels like at least kind of world leaders
are like rah, rah, in a way that maybe
they haven't been so much in the past.
World leaders have FOMO.
So they're saying, well,
safety is all well and good.
Everybody else is working on this, so
we better go ahead and work on this too.
And why I say both is because there are
aspects in which the safety is getting better.
There are aspects in which the power of
the models makes it easier, again, to do
potentially more deeply nuanced and misleading.
Tasks. And so they're, the safety is
just sort of getting, I think, more complicated.
So less the model is going to teach you to build
a bomb and more the model can still potentially
over time, depending on how it's used,
have, maybe not effects that you intend.
I mean, always when you have
technology, you let it out and you
can't always control where it goes.
It's going to go where it goes.
So this has always been to me, the
flip coin of the democratization of AI.
Having it in more hands is going
to be always at the same time.
Good.
And bad.
And I guess, Chris, this is a good
chance to kind of bring you in as well.
I know you were playing the voice of kind
of optimism here, saying, well, overall,
things are getting, in fact, safer,
and you know, I guess I had maybe one
way of nuancing this question a little
bit is to talk about kind of like at what
layers it's getting more or less safe.
It seems to me at least kind of on the
regulatory side, there's been more of a push
certainly to kind of say, well, look, we
don't want to restrict to this technology.
Let's leave it more open.
Which I think certainly
some people think is like,
less safe, right?
But I think there's also people
saying, look, our techniques around
safety are getting a lot stronger as well.
Is that kind of how you think about it?
Like the argument for why it's safer?
I think so.
I mean, if I think of the models that, if we even
just go back two years ago, right, think of like
the GPT, version 3 at that point, or if
you think of the early open source models, like.
The kind of remember LlaMA one, et cetera.
Now, if we started to think about that,
I mean, come on, it's like those models
were terrible in comparison to today's
models, today's models are much more
safer, they come back with better answers.
They hallucinate less.
And then if we think about things from a stack
perspective, we've now got the guard models,
we were talking about that last week, you
know, the idea of being able to reduce bias,
how we train the models are a lot better.
So I think in general, we're thinking about
this a lot more now, that's not, that
we are not going to be much safer in the
future and we haven't got a long way to go.
And we can't do bad things with models today.
Of course you can, but then the way I like
to think about it is, my friend used
to write test code for missiles and he
was like the worst programmer I ever met.
So I'm like, would I even want an old Llama
model doing that versus him writing that code? I, you know what?
I'm like, I think actually maybe it's safer.
Yeah, it's fine.
It's fine.
Anastasia, as, as kind of a
model developer in the space, how do,
how's play us thinking about safety?
Do you feel it's kind of like distinctive
a little bit from what you see elsewhere?
I know kind of philosophically, you're
very, you know, kind of focused on
open, but kind of curious about
your, if you're also trying to kind of,
blaze a new trail in the safety space as well.
For us, we, we don't develop,
conversational models.
We don't develop, chatbots.
We do really specialize, in the models for
both data processing and data processing until
it goes to the retrieval augmented generation.
So basically for us, the most important
part when it comes to safety is actually
the development of the curated and vetted
databases, which are prepared well,
for them to be used for factual AI.
And, this is where actually not that
much work is done, nowadays,
because you still, for example, do not have
Good multilingual, and I insist on
the word multilingual, classifiers,
even for sentiment analysis.
You don't have good toxicity classifiers,
which you can actually understand
what data they have been trained on, and
why we say this is toxic and this is not.
I mean, there is so much work to be done
to actually prepare good data foundations
for factual AI, and where you can actually
have more bound models to the data, to
the even proprietary data, which are the
open data that you use in your stack.
And this is where we concentrate our efforts.
We are not building AGI.
We don't have resources for, and I'm not even
sure we do need to, for multiple reasons.
But we do need this working horses, these
small models, which allow to work through
data, which is not good now, but can be
brought to the, to the quality, where
this kind of RAG applications or whatever
technology will be, brought afterwards.
So this kind of live actual
AI, would be deployed.
I was just going to say, I know a guy who's
got 109 billion dollars that he might be
able to help you build AGI, Anastasia.
You might want to tap him up for it.
We will get to that gentleman
a little bit later in the show.
One of the things I do want to talk about
is that there's, the paper of the week,
the flavor of the week, was this,
Simple test-time scaling paper that I feel
like a lot of people have been talking about.
I'll kind of sketch the overview, if
you have been watching, of course, o1 preview
came out and one of OpenAI's kind
of stated sort of advancements in the model,
was really the idea of test time compute.
Kind of the idea that you'll take
a model, you'll get it to basically
think harder, and it's able to achieve
much better results as, as a result.
And this paper came out, saying,
look, we are trying out this technique that
we call s1, where in order to try to replicate
sort of o1 preview's reasoning ability.
We collected about a thousand questions
and they're kind of reasoning traces,
and we use a couple of different hacks I
think one of my favorite hacks is one where
they just insert the token weight to
get the model to keep, you know, thinking
about a problem, rather than stopping.
And they say look with all these kind
of pretty cheap like rough and ready hacks,
we're able to get a model that's right up
there and in competitive with o1 preview.
So kind of a shocking result, I guess
in some sense, but maybe actually Chris,
I see you're already going off mute.
Like, is it that shocking that
people can just do this replication?
Because I don't, this is going
to be the new thing, right?
Like test time computes
going to change everything.
It's going to be the new whiz bang technology.
And then these people, these researchers
have just replicated it in basically no
time and at much lower cost, apparently.
I think it's the "How I Met Your
Mother," Barney Stinson method.
What's 25 + 2?
Wait for it, wait for it, 27.
You know, and that, that's the
basic technique, as you said there.
And, no, it's not a surprise, right?
We kind of already know there from the model
that the longer that it spends thinking about
it and being able to generate more tokens, then
it's going to have the opportunity to reflect.
And we saw that in the DeepSeek paper, right?
I mean, ultimately, the trick underneath
that was to create multiple samples, take
longer, get longer chain of thoughts.
And once you have longer chain of thoughts, then
the model is more likely to get to the answer.
And that's effectively what they're doing there.
And then they're essentially rejection
sampling, anything that has sort
of bad chain of thoughts there.
Get rid of that.
And, and therefore you're going
to end up with a quality day.
So I don't think this is a surprise,
but it's really cool that it
works with just one token, right?
Which is the weight token.
And, and then it generates
that chain of thought.
I didn't, I think if I'm honest, in
the same way as the kind of step by
step thing in about two years time.
Probably a lesson that we're, we're not
going to do these little hacks anymore,
because what we're going to do is we are
going to have a good set of kind of,
cold start, chain of thought data set to
be able to bootstrap the model with anyway,
and therefore going along and saying, wait,
wait, wait, or whatever, isn't going to have
that effect because the model's going to be
producing the correct chain of thoughts
in the first place, but I think from
this kind of starting with a kind of
relatively simple and small base model.
I think it was the coin two five base model they
use but be able to sort of generate, uh, those
chain of thoughts very, very quickly and get to
sort of decent performance on that domain.
I think great job, right?
But it's really just building upon the work
that kind of everybody's seen with DeepSeek.
So yeah, great job from them
Yeah, for sure.
I think that's like, like what I like
about the solution of the wait, wait, wait,
it's like, it's just like classy solution.
It's like very simple, but really nails it.
Marina, one of the questions I have
kind of like reading this paper is just
like, how far can the test time compute go?
Cause it kind of feels like one of
the remarkable things is you take
arguably less sophisticated models and
you just get way better performance.
Um, and you know, I guess there's kind of a
question just like how far that can go or if
like basically, you know, your base level model
just sets a ceiling at some point on how far
you can kind of reason up and be kind of at
parity with much more sophisticated models.
Are you sort of optimistic that like essentially
test time compute will take us very, very far
or is this kind of just like it's sort of a
hack at the margin for some of these things?
So first I'll say that yeah, it
might be a less sophisticated model.
It's more sophisticated data.
So they didn't take a thousand data points.
They took fifty nine thousand data points and
there was a whole bunch of different filtering
and qualifying and stratifying and the rest
of it that got them down to that thousand.
They spent some time talking about
quality, difficulty, diversity.
Listen, couldn't agree more.
Okay, because everything at that point in
time, the work is going to be done somewhere.
If it's not going to be done in the
model, having to deal with the noise of a
whole bunch of data points, instead it's
going to be done with these are really,
really good representative data points.
And, like Chris said, wait sounds
like, oh, let's think step by step
when that little trick was introduced.
There's other ways to do this.
You can have, again, examples that
where you take through the thinking
of, well, as you go through this model,
you could say, oh, we tried this way.
It didn't, that didn't work.
Let's back up.
This consistently reminds me of taking my
nine year old through his math problems,
where it's like, first you try this.
Oh, it seems like this is not working.
Why don't you back up?
Like, these are the kind of tokens
that you go through and go through.
And the important thing again
here is what data is being used.
It's good that people are
trying to get the compute down.
This is a positive thing because I
think right now we're still in a land
of a really huge amount of waste.
You do not need things that big with
data that is that much and that noisy.
Quality goes a really long way.
So I think the more we continue to focus on the
quality and the type, of data that's being used
here, the more progress we're going to make.
This is all very interesting in the context
of like openness versus closeness in data.
Cause I feel like one of the arguments
that I've heard from some people as well, we
just need so much data that it's impossible
for us to figure out what's open and closed
and we just need to be able to like use it.
Totally.
And you don't need that.
You don't need the vast amount.
You need vast amounts of data, but you
first of all, you need good quality data.
Reason in rich data.
However, what is really interesting in
this moment with well after post DeepSeek
moment and test time, compute, et cetera, is
that we are seeing that we can actually boost
smaller models, which will have smaller impact,
actually, in terms of energy, et cetera.
and we can also boost them for specific
domain reasoning, and it has
been has been happening specifically
for math and coding. And at pleias
as of now, we are actually start
we have started to work on
this for legal reasoning.
And these are the domains
where you actually do have truth.
You can create a chain of thought.
You can create the verifiers, et cetera.
But those data sets are a little
bit more complicated to create
than the coding and the math ones.
And we have been experimenting with
this for legal reasoning, reasoning over
administrative documents, as well as even
sociological reasoning, because you actually
have, you can have quite clear guidelines
that you can depart from, but all of these
things are really, let's say, emerging at
this point, and I'm really look, I'm very
much interested in how it will help to boost
smaller, specialized model for, let's say
industry and for specific domains outside of
general reason and capabilities, uh, which are
tested on traditional benchmarks, uh, basically
mass coding on all these kinds of things.
And
I think it's like the future is,
is really in these small models.
Yeah, I was going to say, I've now imagined
this new future where people will be
like, Oh my goodness, they're going to
court and they're only bringing their
two and a half billion parameter model.
That model is up against a lawyer
with a 70 billion parameter model.
And then there'll be like, "Oh no, the
judges got a 405 billion parameter model.""
And then the witnesses, "Oh no, that
is a 3 billion parameter witness."
Is this our legal future where
we're going to be having small
models versus large models in court?
That I find that future interesting.
Actually for me, this is a more desirable
future than being governed by one AGI.
I mean, this is like at least,
you know, you have to—
it's kind of more capitalistic
future that we are discussing here.
Smaller models accompanying some,
bigger models accompanying others.
I'm, I'm not sure that I want to live in the
one AGI world, but, that's probably just me.
Maybe that'll be our next like hot take
question at the top of next episode.
The next topic we will bring up.
is a blog post that came out
from Sam Altman this week,
simply entitled "Three Observations".
It's a provocative blog post, and
I figured it would just be sort of
interesting to raise and talk about.
the effect of the blog post, I think,
is to talk a little bit about the economic
impact, that we expect to see as
AI systems get more and more powerful.
And in effect, I think Sam makes
these kind of two big arguments.
You know, the first one is that
we see model performance scaling.
Right, so the bigger we get with models,
the better they are, which has kind of
been a longstanding kind of article of
faith in the machine learning community.
And then the second bit is that, like, as the
costs of delivering these models drop, we're
just seeing sort of demand keep increasing.
and sort of his ultimate argument is,
look, we should keep scaling, things will
get cheaper, and that will ultimately have a
sort of gigantic impact, on the economy.
And, you know, in effect, it's kind of
a case for why people should continue
believing in open AI, in some ways,
because I think this is really, at the
core, their sort of value proposition.
and I want to get this kind of
group's take as kind of both folks who
really do believe these technologies are
going to get a lot better in the coming
years, but also I think has tended to be.
You know, I would say AGI skeptics
overall about where all this is going.
and I guess maybe, Marina,
I'll pick on you first.
Curious about kind of what you
thought about the blog post.
Do you sort of agree with the argument?
If you got quibbles with it, kind of
curious about where you felt there
was like problems in the logic.
And maybe that sigh tells us
everything we need to know.
Sam's,
main, the main point that I
cottoned onto, there was a couple.
So first of all, his point three.
"The socioeconomic value of linearly increasing
intelligence is super exponential in nature.
We see no reason for exponentially increasing
investment to stop in the near future."
Give me more money.
Money to me.
Give it.
Give it now.
More to me money.
Seems to be most of the message here.
and then also, I don't think that he,
along with very often a lot of other folks
in Silicon Valley, live in the real world.
Very often, because when they're making
statements like, "Hey, in 10 years,
everybody is gonna want to and be able
to and get benefit from accessing the
AI that only some people can access now."
No.
Things don't go in that kind of a scale, and
that is not the kind of a need that people have
and the benefit that people are going to have.
There's a real specific perspective that he
has, and again, I think that that is more
narrow and more limited, and in some ways, a
little bit, off putting, at least to me.
Again, I'd like people to understand what
the benefits and the use of this technology
are without making statements like this,
which I feel like undermine the work that
a lot of us actually do in the field.
Yeah, and so if I have it right, it
seems like part of the critique is
just that it's like an overstatement.
Yeah, I think so.
I mean, Chris, what do you think?
I love it.
I love Sam Altman.
Go for it, Sam.
It's like, it's gonna change the world.
It's, it's great.
You know, you know, I think, I think
everybody's right I think you gotta have a
super positive attitude in this sense, right?
Which is this is going to be a world changing
technology and and we can see that
from how good things have improved over
time. So it's gonna have an impact.
Of course, it's gonna have an impact, right?
But then new value creation is going to happen.
We're going to do new ways of doing things.
I think that's great. So I think it's
healthy to talk about, you know,
what the impact is going to be.
Is it likely to be somewhere in between?
Maybe because you know what, as soon as
we get something that's super cool, we
then just take it for granted as normal.
And then we move on from there.
I mean, I can guarantee I
I've said this for a while.
I think as soon as we get AGI and we can,
we can define what an AGI is later, but
I think, but I, but I think the first
thing that's going to happen is it's
going to be put in a box and then there's
going to be a big museum open somewhere.
And we're all going to walk in
and go, you can chat with the AGI.
And we'll be like,"" Ooh, there's the AGI.""
So that thing's going to
be in a box for a while.
That's, that's his future,
like a kind of circus.
And, and, but, but honestly, I,
I am super positive about things.
I think that this is a
world changing technology.
Is it going to be like a
conscious thing around that?
No.
But if we look at things
like coding, the reality is
that, even if you're using kind of like the
o1 models or the o3 mini models
today, they are better for like a lot of
tasks, you know, to be able to turn out that
code really quickly at a super high quality.
And, and that is a reality.
And if we think it's going to stay in that
domain, I think we're kidding ourselves, right?
As, as the cost comes down, more and
more people are going to use this.
Yeah. And this is kind of, I'm trying to
parse optimisms, I guess, in some sense.
Right. Cause I think like on one hand, Marina,
I agree with you, which is like the
blog post is very frustrating, like
the tone is very frustrating to me.
On the other hand, it's like, okay,
Tim, but do you believe the technology
will have like a really big impact?
I'm like, oh, yeah, for sure.
I think it will have a really big impact.
And it's like, kind of like parsing,
like, how do you articulate a way of being
optimistic about this stuff that doesn't
kind of fall into the usual valley tropes
seems to be like part of the problem.
Not being in the valley.
I, I mean, for me, just, just
to probably continue with,
with the optimism.
Uh, the ChatGPT moment was
like a huge moment of liberation.
I am.
I don't like to write like, I mean, and
we don't sometimes understand the, the,
the emancipation liberating power of this
technologies for people who actually like
who had to really struggle with, let's say,
writing, with coding, who didn't have access to
this, who had less this intelligence, who use
this as tools now to actually do something
they weren't able to use before to do before.
I, however, this being said, I'm not sure
that, this emancipating power is really
well understood and put in the place where
it should be put because we are kind of going
and hunting, the AGI where actually we
do need like this assitants helping us
to be like to be what we actually can be,
in the society with less constraints on like
how you can learn code, to code, how you can
learn to write well, and how not to be judged
by the society and by the economic system on the
on the sheer amount of autographic errors that
you are making, you know, for me, this is
like more important than like hunting the AGI.
However, I think that and Marina,
you actually brought up this
third point of exponential growth.
The planet is limited.
We do have limited resources and I'm very
much surprised to to to read such takes
because AGI won't resolve the issues of
ecological crisis, which we are going through.
We cannot and we won't have time to build enough
nuclear power plants to actually
run everything on nuclear energy.
And even with nuclear energy, it won't help
us to to resolve the ecological crisis.
So I'm sorry to say this.
We it's for me like and probably this is
a bit of like European perspective, an old
fashioned way of saying this, we cannot
just forget everything we were saying just
two years before, two years before,
and like, okay, AGI will solve everything.
It won't solve everything just like
Internet didn't solve everything.
You still have lots of places in multiple
continents without Internet access
and we still haven't resolved multiple
issues which won't be resolved with AGI.
So, I mean, we, I really do feel that we need
to be more pragmatic and this is also important
because Sam does this, and I actually heard him
do the same speech at the Élysée Palace in front of
Emmanuel Macron on Tuesday, where he basically
said the same and saying, we need to invest.
Let's invest in, in data centers.
Let's, and I mean, we need to be careful
about this calls for investment and how
they will actually really impact the society
without saying we don't need this technology.
We do need it.
We need to develop it, but in a reasonable way.
Which is not building a data center
near every school, you know, so
I think I agree but disagree a lot and I think
the reason is that I don't think we need a
nuclear power stations as you say, I think I
know Macron's suffering is nuclear power stations
for everybody as part of his 109 billion but
but but the reality is with test time compute.
We should be spending less time focusing
on the pre train, pre training stage.
Now that's not to say that we're not going
to be pre training, but it, it shouldn't
be like pre train, pre train, pre train.
With test time compute, you can get very,
very far as we were just talking earlier in
the podcast with the, like the s1 models.
By just using high quality data sets.
So, but being able to push that to longer
chain of thoughts, being able to push that onto
consumer grade hardware, I think we are already
proving that scaling can occur at a lower cost.
So I think if we're needing nuclear power
stations to be able to, to, scale AI,
then I think we're on the wrong track there.
But the question is inference,
not pre training only.
So, it's a much bigger,
bigger chain, for energy.
And this is, yeah...
But I can run inference
I can run inference on my laptop,
right? With my Apple silicon chips.
It's fine.
The cost of the cost of inference is much,
much lower than, than the cost of training.
But once we do scale AI applications,
it will also be a question, however.
But let's agree to disagree.
That's not how this podcast works.
Yes,
it's not.
I'll say something that might be funny
coming from my background as a language
primary study person, which is we should not
forget the power of these models in other
non language domains, multimodality, sensors,
time series, all sorts of kinds of things.
There is so much use to be gotten out
of these, not only in helping you write,
which no, I love it too, I hate writing
emails, not only in helping you write
code, but also imagine all of the ways in
which you could improve, factory work.
Imagine all the ways in which you could
improve tracking the sensors that we
put on, migratory animals to see if
we can help, you know, their habitats
out, all sorts of things like that.
If we really actually broaden where this
technology could be thought of being used,
because again, remember, this technology
actually has nothing to do with language.
Let's just remember that.
It doesn't.
And then I think that we could actually go
in places that are much more interesting, much
more pragmatic, much more practical, and
yeah, maybe not only focus on, on the parts that
are language and certainly not on chasing AGI.
Last item I really wanted to touch
on was just a kind of fun sort of
data set, that Anthropic put out.
they're calling it the
Anthropic Economic Index.
And what I like about it so much is, if
you remember from the early days of Google,
they had a project they did called Google
Flu, which is basically using people's,
search results to try to identify where people
were getting sick and creating basically
like a live heat map of like where illness
was kind of spreading around the population.
And this in some ways is kind of like a weird
update of it in some ways, only it's really
looking rather than at people getting sick.
What people are even using AI for.
And so Anthropic basically looked across
a sampling of all of its conversations,
and using an anonymized set of them said,
okay, well, what can we learn about how AI
is kind of spreading across the economy?
And they reached some
pretty interesting results.
I recommend going on the website and looking
Um, the blog posts that they did.
The one that I kind of wanted to touch on,
because we don't have too much time left in the
episode, is really this finding that right now.
about 36%, of usage of AI
assistants is really still in software
development and technical writing tasks.
And I think this is almost a
kind of Marina's point, which is that
we've kind of sold this technology.
I thought about this technology as this
kind of economy spanning thing, but one of
Anthropic's findings is that it still ends up
being quite concentrated what these people,
what people are actually using it, for.
And outside of that, it's
kind of a very, very thing.
And so we just want to first kind of
talk a little bit about that result
and happy to talk about any of the
other data that they kind of mentioned.
But I think that finding was just so
interesting because it, at least for me,
was kind of a violation of my expectations.
I was like, oh, people are using it for
writing emails and composing poems and
essays and, you know, all this other stuff.
But still, it's ultimately very
much in It's like a software tool.
I guess, Anastasia, do you
think we should be surprised or am
I just like, kind of not with it?
It's actually, well, this study
corroborates what we have been also seen,
in other studies as well as, well, the
one that we have conducted for the French
government based on the, well, the
usages, that have been done with the,
with the copilot we developed for them.
I think that.
One of the reasons is when I actually, for
example, see how far from software development
and even from, let's say, application,
you tasks and marketing, et cetera, for
example, financial analysis and things like
this are, I really do think that, once
again, this, the tools that we're, that we
have now, this chatbots, they're not really,
they're not really adapted to the, to the
exact knowledge work, that, most that
people in other industries do have, for
the software development, I wasn't surprised.
And it's true that also Anthropic has been
largely marketed as the state of the art
coding tool.
So it's kind of normal to see this.
I think it's and they say
it in the paper themselves.
So this could be a little
bit biased because of this.
However, I really do think that we this
study shows how actually far we are from a
wider adoption of, large language models
as everyday tools at work, which is
still surprising I mean we could think that
yeah, we all use it now we actually don't
and there are some parts of the of the
population that are much more exposed to this
and It will require also quite an important
part of education on workplaces, actually
to the people who are using it now less, as
well as UX and like other product adaptations
from the model providers, of course, as well.
Chris, I guess this almost builds a little bit
on, I think you were making a joke earlier,
which is like, we're going to invent the AGI and
it will just kind of live in the zoo for a bit.
Like this is literally it, right?
Like not AGI, but like we have like
really powerful AI systems, but it's like
still largely like a technical industry phenomenon.
I'm sure, I mean, Anthropic would look at this
data and say, we have so much more market.
We can grow so much more.
I guess the pessimist view is like,
well, maybe ultimately this thing is most
useful and will continue to be most useful
in stuff like coding.
Is that a concern that you
think for the AI industry?
No, I think I'm gonna agree with Anastasia
on this one which is that I think if you
asked Moët & Chandon, what the primary
use of glass bottles are, I'm pretty sure 30
percent of them is gonna say champagne, right?
And I think that is the reality for
Anthropic as Anastasia said, right?
What Claude has marketed is
this, the best at coding.
If you think of the ecosystem, so if you think
of things like Klein, et cetera, the default
models that they put in there is Claude, right?
So, you know, so anybody in that industry
knows that, you know, you know, up until the
reasoning models point of view, you know, you
would typically go to Claude for code intel.
So I really think that data
is skewed, as you say there.
So I, I think that's probably where it is.
So if we asked OpenAI what the primary uses are,
I think you would get a different result set.
I think it would be a different
variation, just because that's kind of
more aimed at a wider consumer base.
And I would even, I would even
probably argue that the, GPT-4o mini
versus the o1-mini
versus the o3-mini
the o1
pro would have a completely different,
usage set of data there as well.
So I, I, I think it's interesting that
they've came back with it, but I, I, I just
don't know how to read into those answers
just because it's, it's really a very
thin vertical slice, in my opinion.
Yeah. All it's telling you about is like
what Anthropic's being used for.
Yeah, exactly.
I don't, I don't think it's
representative of the world.
I don't think it's representative of America.
I don't think it's representative of, of, of UK.
It, to your point, Tim, it's representative
of how Claude and Anthropic is used.
You know, which is super interesting,
but yeah, that's what it is.
I mean, they explicitly say that, they
freely admit it and explicitly say it.
They say that one of the biggest points of
releasing this dataset is to release this
dataset and to hope that we can get something
from other people that's kind of similar.
It's like releasing search logs.
You're never going to get all of
them, but it's nice to get something.
I don't know.
I liked the economic perspective.
They were very careful with stating
their limitations or their assumptions
that you shouldn't read into it.
And it's more about the process of the analysis
than looking at the results of the analysis.
I agree with that.
Yeah. And I think that's actually, Marina, if I could
kind of follow up question there, I think one
of the ones that I had to kind of talk about
with the panel was, this is cool because I think
to date we really haven't seen these companies
say based on all the aggregate data we have,
there's useful things we can build on top of it.
and I think this is kind of is like
a sort of new start in some ways, right?
Rather than them simply saying, oh,
we provide an AI tool you can use.
This is like our platform now gives
a signal about the world at large.
I guess I had a question for you
is like, do you think companies are
going to do more of that going forwards?
Or if this is kind of like, well, more of
a demo project more than anything else?
I don't know.
I think this technology is still a
hammer in search of a lot more nails.
We've gotten a couple of nails, but there's
a lot more out there and given the investment
that people have put in and are apparently
wanting to continue to put in, it'd be nice
to find some more nails, um, and go out and
try to figure out from people like, hey, do
you even know how to use this technology
or do you not know?
Because I don't think yet we're at the point
where people have an knowledge of how to use it.
When we first started with the internet, people
didn't quite know how to use it correctly.
People got better and you can
make endless examples here.
So we haven't gotten there yet.
I think it's actually to company's benefit
to continue to get people more comfortable
having a broader view of this and all the rest
of it and not just makes this seem like, oh
cool, this is by tech folks, for tech folks.
written about, you know, by tech
folks like, you're going to run
a market as you said correctly.
Yeah, and I think that will be a
really long process of just like
understanding how to, how to use it.
Like I think about those, um, if you've
ever seen those early films of people, just
as film cameras were coming out and, um,
you know, you basically people see people
line up and they kind of like pose it
completely still assuming that it was like
a photograph versus like a film camera.
And, and it's like, it took a while for us
to be like, Oh, you can do movies with this.
It's not just like a camera.
Yeah. So, maybe we'll see.
We're at that early stage with, with AI as well.
well that's all the time we have for today.
Chris, Marina, thanks for joining us again.
Appreciate you doing double
duty for two episodes in a row.
Anastasia, it was great
having you on the show.
We'll have to have you back at some other time.
And, thanks to all you
listeners for joining us as always.
If you enjoyed what you heard, you
can get us on Apple podcasts, Spotify,
and podcast platforms everywhere.
And we will see you next
week on Mixture of Experts.