Disney Signs AI Licensing Deal
Key Points
- Disney is striking a three‑year licensing agreement with OpenAI that lets the company use Disney characters in generative AI models while also taking a roughly $1 billion equity stake in OpenAI to steer fan‑made content back onto Disney‑controlled platforms.
- The deal marks a shift from typical AI licensing (which usually only grants data for training) toward a strategic partnership that gives Disney both creative control and a financial foothold in the AI ecosystem.
- In the broader AI roundup, driverless robo‑taxis from Google, Amazon and Tesla have gone mainstream, Walmart has moved to Nasdaq to rebrand as an AI‑first enterprise, and IBM has teamed with Kaggle to launch a leaderboard for real‑world AI model performance.
- Panelists debate why OpenAI would accept a “sub‑billion‑dollar” deal, noting that the arrangement provides valuable IP access, a new distribution channel for Disney‑generated content, and a potential long‑term revenue stream beyond pure data licensing.
Sections
- Untitled Section
- Testing New AI Models with Copyrighted Prompts - The speaker explains how people routinely challenge fresh image and text models by generating copyrighted characters and quirky test prompts, highlighting legal restrictions, Disney‑OpenAI dynamics, and the prospect of author consent for fan‑fiction‑style outputs.
- AI, Fanfiction, and Disney’s IP Strategy - The speakers discuss how AI‑generated fanfiction reshapes traditional authorial social contracts, prompting corporations like Disney to partner with AI firms and monetize the emerging landscape.
- AI Hype Overshadows Technical Progress - The speaker critiques how 2024’s AI narrative has been driven by business deals and media hype, leaving genuine research and technical advances on the sidelines.
- Full-Stack AI Race & Commoditization - The speakers discuss how AI models are becoming commoditized, sparking a competitive “full‑stack” push—led by Nvidia and others—to integrate, distill, and simplify deployment, turning the focus toward economic viability and user‑friendly access.
- Hype vs Architecture of Mixture‑of‑Experts - The speakers critique the marketing hype around a new AI model, noting its architecture mirrors earlier designs (transformer, Mamba, mixture‑of‑experts) and explain how such models use only a fraction of total parameters during inference by activating subsets of experts across sizes like nano, super, and ultra.
- Embedding Alignment via System Prompts - The speakers discuss how directly integrating alignment principles into a model’s system prompt—rather than relying chiefly on downstream prompting—shapes distinct model personalities and prompts the question of why this approach hasn’t been more widely adopted.
- Anthropic Model Quality and Literary Style - The speakers debate how the phrasing of Anthropic’s documentation contributes to Claude’s perceived friendliness, noting the challenge of quantifying such qualitative traits.
- Prompting: Temporary Trend or Permanent Tool - The speakers debate whether prompting will remain essential or become obsolete as AI models grow more prescriptive and embed explicit moral philosophy.
- Teasing Next Week's Mixture of Experts - The host wraps up the episode and hints that the upcoming show will focus on the Mixture of Experts model.
Full Transcript
# Disney Signs AI Licensing Deal **Source:** [https://www.youtube.com/watch?v=dJTwev1O-mE](https://www.youtube.com/watch?v=dJTwev1O-mE) **Duration:** 00:38:19 ## Summary - Disney is striking a three‑year licensing agreement with OpenAI that lets the company use Disney characters in generative AI models while also taking a roughly $1 billion equity stake in OpenAI to steer fan‑made content back onto Disney‑controlled platforms. - The deal marks a shift from typical AI licensing (which usually only grants data for training) toward a strategic partnership that gives Disney both creative control and a financial foothold in the AI ecosystem. - In the broader AI roundup, driverless robo‑taxis from Google, Amazon and Tesla have gone mainstream, Walmart has moved to Nasdaq to rebrand as an AI‑first enterprise, and IBM has teamed with Kaggle to launch a leaderboard for real‑world AI model performance. - Panelists debate why OpenAI would accept a “sub‑billion‑dollar” deal, noting that the arrangement provides valuable IP access, a new distribution channel for Disney‑generated content, and a potential long‑term revenue stream beyond pure data licensing. ## Sections - [00:00:00](https://www.youtube.com/watch?v=dJTwev1O-mE&t=0s) **Untitled Section** - - [00:03:37](https://www.youtube.com/watch?v=dJTwev1O-mE&t=217s) **Testing New AI Models with Copyrighted Prompts** - The speaker explains how people routinely challenge fresh image and text models by generating copyrighted characters and quirky test prompts, highlighting legal restrictions, Disney‑OpenAI dynamics, and the prospect of author consent for fan‑fiction‑style outputs. - [00:07:08](https://www.youtube.com/watch?v=dJTwev1O-mE&t=428s) **AI, Fanfiction, and Disney’s IP Strategy** - The speakers discuss how AI‑generated fanfiction reshapes traditional authorial social contracts, prompting corporations like Disney to partner with AI firms and monetize the emerging landscape. - [00:12:25](https://www.youtube.com/watch?v=dJTwev1O-mE&t=745s) **AI Hype Overshadows Technical Progress** - The speaker critiques how 2024’s AI narrative has been driven by business deals and media hype, leaving genuine research and technical advances on the sidelines. - [00:18:48](https://www.youtube.com/watch?v=dJTwev1O-mE&t=1128s) **Full-Stack AI Race & Commoditization** - The speakers discuss how AI models are becoming commoditized, sparking a competitive “full‑stack” push—led by Nvidia and others—to integrate, distill, and simplify deployment, turning the focus toward economic viability and user‑friendly access. - [00:22:32](https://www.youtube.com/watch?v=dJTwev1O-mE&t=1352s) **Hype vs Architecture of Mixture‑of‑Experts** - The speakers critique the marketing hype around a new AI model, noting its architecture mirrors earlier designs (transformer, Mamba, mixture‑of‑experts) and explain how such models use only a fraction of total parameters during inference by activating subsets of experts across sizes like nano, super, and ultra. - [00:26:50](https://www.youtube.com/watch?v=dJTwev1O-mE&t=1610s) **Embedding Alignment via System Prompts** - The speakers discuss how directly integrating alignment principles into a model’s system prompt—rather than relying chiefly on downstream prompting—shapes distinct model personalities and prompts the question of why this approach hasn’t been more widely adopted. - [00:30:06](https://www.youtube.com/watch?v=dJTwev1O-mE&t=1806s) **Anthropic Model Quality and Literary Style** - The speakers debate how the phrasing of Anthropic’s documentation contributes to Claude’s perceived friendliness, noting the challenge of quantifying such qualitative traits. - [00:34:50](https://www.youtube.com/watch?v=dJTwev1O-mE&t=2090s) **Prompting: Temporary Trend or Permanent Tool** - The speakers debate whether prompting will remain essential or become obsolete as AI models grow more prescriptive and embed explicit moral philosophy. - [00:38:17](https://www.youtube.com/watch?v=dJTwev1O-mE&t=2297s) **Teasing Next Week's Mixture of Experts** - The host wraps up the episode and hints that the upcoming show will focus on the Mixture of Experts model. ## Full Transcript
There's the flip side of the deal which, yeah, you
can as an individual use Sora to generate things with
Disney images, but Disney is going to stream these kind
of videos. So they're going right back to Disney and
they're trying to basically have control in some way of
that fan generated content and have it come back to
Disney instead of proliferating on X or Blue Sky. So
they're trying to say that before this gets into a
whole. Oh yeah, look at all these wonderful fan generated
shorts. No, no, no, come be on our platform instead.
All that and more on today's Mixture of. I'm Tim
Hoang and welcome to Mixture of Experts. Each week Moe
brings together a panel of charming and brilliant minds in
technology to distill down what's important in the latest news
in artificial intelligence. Joining us today are three incredible panelists.
We've got Martin Keen, master inventor, Marina Danielewski, senior research
scientist, and Kush Varshni, IBM fellow. The year is winding
up, but it's still full of AI news. We're going
to be covering Disney and OpenAI's new licensing deal, the
Time Magazine Person of the year, Nemotron 3, the new
Nvidia launch, and this Claude Soul document. But first we've
got Eilee with the news. Hi, I'm Ili McConnen, a
tech news writer for IBM Think. Here are a few
AI headlines you might have missed this week. 2025 has
been the year that driverless cars have gone mainstream. Google,
Amazon and Tesla each launched their own version of these
robo taxis in various US cities. Walmart has moved from
the New York Stock Exchange to nasdaq, a move illustrating
the retail giant is trying to transform into an AI
powered enterprise. IBM and online platform Kaggle have partnered to
create a new leaderboard that evaluates AI models and agents
as they solve real world enterprise issues. It's official the
generated content. For more, subscribe to the Think newsletter linked
in the show notes. And now let's go back to
our experts. I want to begin today's episode by talking
about the Disney OpenAI deal which was just announced this
past week. So this is sort of an interesting one.
I know at this point we're all kind of very
cynical about deals that are less than, you know, $10
billion. But the core of this deal basically is that
Disney is about to sign a three year licensing deal
with OpenAI to allow its characters and IP to be
used in its generative kind of AI models. Additionally, Disney
is going to be taking what is effectively kind of
like a billion dollar stake in OpenAI itself and become
an equity owner. And so I guess maybe. Martin, I'll
kick it off with you. Why is OpenAI signing this
deal? Exactly. Yeah, it's such an interesting deal, isn't it,
Tim? Because traditionally, the sort of generative AI deals that
we've seen up until now have been for training and
grounding purposes. So you would purchase, for example, a bunch
of news articles. So OpenAI did a deal with Financial
Times for that, and Google did a deal with Reddit
where they pay Reddit, I think, 60 million a year
for the Reddit training data. But this isn't that. But
this is the other end of it. This is taking
the finished model and actually using the output of it
to be able to incorporate the characters in, which is,
you know, a really different way of looking at it.
And I think from one perspective, you can think, well,
as soon as a new image model comes out, people
are trying to like, generate copyrighted material instantly with it
and seeing how much the model lets you get away
with. And often over time they kind of. Yeah, I
think one of my first SORA uses was like, Mickey
Mouse cartoon. Exactly. So, you know, might as well do
deals we might see beyond this if this one works
out for Disney and OpenAI. Whenever a new model comes
out today, there's always the same sort of test that
everybody does on the model. Like, you always see people
try to build a vector image of a pelican riding
a bike to see how good each model is. Right.
Or an otter on a plane. Using Wi fi is
another pretty popular one. I have my own one of
those, which is every time a new model comes out
that's supposed to be good at writing, I have it
generate a Nelson DeMille short story in the style of
Nelson DeMille. And it's not bad, but always it comes
through and says, I can't write in the style of
Nelson DeMille. But here is something similar to it that
includes some of the themes. It's like a disclaimer. Right.
But maybe not in future. Maybe in future authors will
kind of consent as well. And then I could, like,
make fan fiction of my favorite authors. Who knows where
this is going? Yeah. And I guess, Marina, I think
one of the things that comes to mind is, you
know, with Disney, they've got, you know, the most frustrating
thing for Disney for a long time is they've had
like the Vault, right, Where they're like, you can only
access certain movies at certain times and then arbitrarily at
certain times, Disney is like, it's in the vault. You
can't get access to it. It's a long way of
saying, I think like, Disney's very protective of these characters
and this IP so them to say, like, yeah, we're
okay with a world where anyone's going to use generative
AI to, you know, have Mickey Mouse do whatever. Oh,
the Mickey Mouse is a bad case because it's like
now entering kind of more public domain. What's your thinking
on, like, why Disney's finally willing to take this risk?
Because it's kind of a big deal for them in
terms of how they think about and control their intellectual
property. It is. And Bob Iger did say something about
this is kind of coming anyway, so we want to
get ahead of it instead of fall prey to it.
And something that I just want to mention is there's
the flip side of the deal, which, yeah, you can
as an individual use SORA to generate things with Disney
images, but Disney is going to stream these kind of
videos. So they're going right back to Disney and they're
trying to basically have control in some way of that
fan generated content and have it come back to Disney
instead of proliferating on X or Blue sky or wherever
you're going to be. So they're trying to say that
before this gets into a whole. Oh yeah, look at
all these wonderful fan generated shorts. No, no, no, come
be on our platform instead. So this also reads to
me to a large extent like a platform play where
they're like, look, this is going to happen anyway, so
let's make sure that people are doing these things on
our platform and constantly coming back to that much tighter
integration with Disney. So that seemed to be important part
of it to me. Yeah, absolutely. And I think, I
mean, Kush, this sort of opens up. I think what
is kind of like a really strange world, right, is
like you just turn on like the Disney Channel at
some point and it's just like infinite user generated, generated
AI cartoons of these characters. Right. Is that, that kind
of seems like where we're headed. Yeah, I think it
is. And for these diehard listeners of the podcast, I
mean, maybe last year at some point I was mentioning
Foucault and the author function and how, I mean, the
whole social contract of authorship is maybe Changing and stuff.
And I think that's exactly it. Right. So when you
have these thousands or millions or billions of fanfiction out
there, the social contract is just completely different. Now. It's
like before in these oral cultures and stuff, you had
these bards singing. They were kind of, I mean, not
trying to make money off of it. They were shepherds
or farmers or shopkeepers or whatever, and they were just
doing it for their own status kind of, or continuing
a tradition. And then with blogs and stuff, this digital
reality kind of came about again. And now with the
AI tools, I think Disney's just going to capture exactly
that same sort of social contract. So as Marina was
saying, I mean, be ahead of the game and kind
of just be part of where the world is headed,
because I think they're still going to make their money
on the amusement parks and the merchandise, licensing and all
of that sort of stuff. So just like whatever authorship
is turning into, I think they just are a part
of it too. Martin, where does this all go ultimately?
Now that kind of Disney has been willing to jump
in. I assume everybody else who owns significant IP is
also kind of thinking, well, is this my time to
get in and cash in on some of this stuff,
to build a partnership with these AI companies? Yeah. I
don't know if you have any thoughts on basically where
this goes next. Yeah, I mean, like you say, Tim,
this completely flips the script because LLMs come out, everybody's
thinking, hey, how did this get trained on my data?
Do I need to sue to get my data paid
for? And so forth. Now this is the exact opposite.
Disney are paying a billion dollars in an equity investment
to open AI with more to come. So it's completely
the opposite way around. So I think this really signals
Like, are the eyeballs now going to go more to
the Disney IP and not to your own ip? So,
yeah, we may even see kind of the complete reverse
of what we've seen before, where everybody is kind of
pushing to get their content into these models now. Yeah.
And I think Marina, I think that was like, one
thing I want to note, end this sort of segment
on is, you know, obviously Disney's Disney. Right. One of
the biggest, most powerful content companies in the world. Do
you have any thoughts on what this means if you're
just like, I don't know, someone creating art online? Because
I feel like they're in a very different position right.
From someone who happens to own the rights to Harry
Potter or something like that. I was going to say
K pop Demon Hunters. Get back to me when. Yes,
that's right. Playing with my daughter, who's obsessed right now.
But I. I could imagine other content owners following suit
and seeing what this is. I mean, I wonder to
what extent. I know it's an exclusive license for a
Meta or whoever for a year for using their content?
Are we going to be chopping things up again? Are
we going to see more mergers of this kind of
thing? I think there's a lot of economics here that
are really interesting and very much not worked out. Like
what Kush said, the contract with creators and what counts
as official and what counts as unofficial, that line's going
to be way harder to tell. And I think that
there's going to be a lot of people involved in
that in the next few years trying to draw those
lines. Well, we'll keep an eye on it. Super useful
getting all your opinions on it. So I'm going to
move us on to our next segment. Time magazine, as
you all know, has an annual Person of the Year
feature that they do, and this year's Person of the
Year. Not a person, but in fact the architects of
AI. And so if you've seen the magazine cover, it's
a take on a kind of classic image of construction
in New York, but it's sort of all of the
kind of various luminaries of AI sitting on like a
construction girder. And I thought this was so interesting and
worth spending some time on because it lets us talk
a little bit about sort of what's been happening this
year in AI, but also just how the media representation
of AI is evolving. I think the big first thing
that stood out to me was you have CEOs and
infrastructure providers, kind of not a whole lot of researchers
are represented here. Yeah. So, yeah, I think, I mean,
Lisa Su was on there, which I think was surprising
for people. Former IBM researcher, by the way. They got
to get that name checked. Exactly. So, yeah, I mean,
I think having the CEOs on there, really, that's the
signal. I mean, architects of AI, like, what are they
architecting? Right. I think it's the financial aspects, the hype,
the business. I mean, that's what's being architected. And if
they had chosen to feature, I mean, the scientists or
even the data workers, I mean, that would have been
a very impactful sort of COVID But yeah, I think
just capitalism is forefronted. I mean, that's what architecture means,
I think. And is that a good thing or a
bad thing? I mean, I'm not going to comment on
that, but I think that's the message at least. Marina,
does. This is going to be a very pointed question
for someone like you. Does. Does research matter anymore? You
know, I mean, in the sense that basically, like, what
kind of. What sort of time is. Time is saying.
Time magazine is saying, And I think it does kind
of capture something which is fundamental about the moment in
AI, which is almost all the action, all the attention
is on the business side. Right. And so I guess
I'm really curious about kind of how you feel about
sort of like this balance of power, I guess, between
like the folks advancing, like the actual technical research and
then I guess this increasingly large ecosystem, which is important,
no doubt, but kind of sits almost entirely aside from
some of what's happening on the latest papers from Neurips
or what have you. I really, really agree with what
Kush said, is that this is a signal that it
hasn't been as much the year of AI as it's
been the year of AI hype. AI communication. AI is
business. AI is the financial deals. Not necessarily so much
the technical side of things. I mean, look, that's interesting.
It's continu. But also, people were saying that 2025 was
gonna be the year where AI agents put all of
us out of work. Not quite there, guys. Still getting
there, still working on it. But the hype this year,
the stories, the way that people talked about it was
ridiculous. And a lot of it really centered on these
cults of personality, these who could say the most ridiculous
things in the news and move the coverage as ping
pong balls from, oh, now it's this company. No, it's
back here. No, it's back here. No, it's back here.
So it was certainly a year of that. And, you
know, I also like Kush. I'm not completely sure what
to make of it. It's reflective of reality. It's maybe
not reflective of where the real work and the interesting
technical work is happening, but you can't deny the reality
which, yeah, it has been this, for better or worse.
Yeah, it has been the hype, basically. Martin, thoughts on
this? I'm hearing some grumbles, maybe from your other panelists.
No, I'm totally on board with what you're saying there,
Brina. It does seem like the year of AI hype
would be the tag for this rather than the year
of the agent. Perhaps. But I mean, look, the article
points out how much of this focuses on infrastructure, how
much it's been on infrastructure this year, how much spending
there is just raw spending on AI data centers and
so forth. They mentioned in the article Over $400 billion
in 2025 just on kind of AI activities, which is
a huge amount. So the article kind of made the
point that are these the next industrial titans? We had
the railroads and so forth, and now it's the data
center. Is this kind of the next thing? And I
think a lot of the focus is on that now.
When I saw that this year it was kind of
I went back and had a look at some of
the other Time magazine persons of the year. And this
is certainly not the first time it has not been
an actual person. And do you know who the person
of the year in 1982 was? No, I do not.
It was the computer popularized by the IBM PC. So
we've kind of come full circle from hey, everyone has
a computer now to hey, now everyone has access to
AI 2025. I don't know if you can talk 2006
you or something. There was just a mirror that's from
your one. I'm going to move us to kind of
our next topic. This was one. I was joking a
few weeks ago. I was like, I am tired. Because
it feels like every few weeks we do a segment
which is a new model's out. What do you think
about it? And in fact the end result, I forget
who commented on a previous episode. And the end result
is basically there's a lot of models coming out all
the time. They're all really good. And after a while,
kind of a lot of the distinctions tend to blend
into one another. But we're going to do a segment
on this. So Nvidia launched its newest generation of its
Nemotron open source models, Nemotron 3. And there's a lot
of stuff that we've kind of come to expect from
some of the model releases that are coming out in
the last few months. They're focusing a lot more on
agentic behavior. There's a spread of models from the very
largest to the smallest. And there's a bunch of kind
of infrastructure and other kind of component accessories they've released
with this generation of models. I do want to get
into that, but I think I want to start a
little bit with just like a business question for some
of our listeners. Kush, I guess I have a question
basically like why isn't Nvidia always winning when it releases
its models? Like doesn't it have the ability to create
models that are ultra optimized for their own hardware that
everybody else runs on? And so kind of implicitly doesn't
it make sense that the Nemotron models would be some
of the most successful models out there? But that doesn't
really seem to be the case. Right. We tend to
focus on a lot of other players in the space
for their models. And so I guess I'm kind of
curious if you can account for Nvidia being this huge
hardware leader. But I think arguably and I think realistically
is not necessarily like a model leader. Yeah, maybe now
they will be actually. So yeah, I think they've been
kind of, I mean moving up the stack. I mean
starting as a GPU company and then having the CUDA
and then part of this announcement was actually an acquisition
of ShedMD and this is work for workload management, sort
of scheduling sort of software and stuff like that. And
so I mean they keep moving up the stack. They're
kind of like consolidating everything as they go and I
think they're really just, I mean it's like a snowball
a long time ago I did an internship at Sun
Microsystems and the key phrase for them was the network
is the computer. So this was John Gage I think
was the person who came up with this. And I
think they're just rolling up to becoming Nvidia. The AI
stack is the computer and I think that's how it
is. And they're kind of controlling the narrative going forward
as well. So I think we just need to keep
an eye on what comes next. Where else do they
go? Yeah, so you're actually saying this is actually a
pretty big deal. I shouldn't necessarily shrug off like eh,
Nemotron 3, much like Nemotron 2. What's another model? You
actually think this is actually significant in some ways? Maybe.
I mean you can say that maybe the models are
being commoditized in some fashion. So what they're doing is
probably the same recipe that others are doing that we're
doing. I mean on the Granite 4 architecture is pretty
much the same. So maybe it's just. Yeah, I mean
connecting everything together is the thing. Marin, if you agree
with Kush. There's almost kind of like a fun, interesting
race going on. Right. It seems like. Which is like
Nvidia moving up the stack, trying to gobble everybody at
the same time. Everybody's trying to like, get off of
Nvidia onto other hardware platforms. I don't know. Do you
agree with Kush's assessment? Where does this all go in
your mind? I think I definitely do agree with Kush
and the full stack play is something that has been
going on. You're right. People coming from different ends, either
coming from the bottom or from the top, and try
and do it. Because the quality of these models by
themselves has gotten very comparable now that it matters so
much more. The levels of integration, the levels of distillation
that you can do for specific things, how do you
integrate them together? How do you test things for yourself,
see if this works for you or not? When everything
gets commoditized like this, it certainly turns once again much
more into, okay, what's the economic play here? And also
what's the ease of use? I think people's expectations for
ease of use and ease of trying it out is
very, very, very high. So I think that also Nvidia
doesn't want to be dependent on other people to choose
it or to not choose it, so they are necessarily
getting ahead of the game. I think this makes a
lot of sense to me. They're not the only ones
that are doing this, so I think it makes a
lot of sense what they're doing. Martin, you may have
comments on this. I think one question also just throw
into the mix is I think a meta story of
2025 is what are the bounds of kind of like
an open release in the space? And it feels like
that's kind of one of the questions that's playing where
people used to say, oh, well, we just released the
model and now people are like, well, the model and
the data and Nvidia is here because they're also releasing
a couple training data sets and reinforcement learning libraries. It
feels like, if anything, the scope of Open is getting
broader and broader and broader about what's expected when you
do one of these open releases. And curious if you
have thoughts on the trends there or anything that Kush
and Marina just said. Yeah, I mean, it seems like
the openness is going to expand a lot, especially with
the EU AI act coming in next year. That's going
to require things like stating what your training data set
is and so forth. So that's a big open thing.
That is not Discussed even with many open models. So
that's interesting. Now Tim, you made the point of why
do Nvidia not have the best model? Because they have
the GPUs and shouldn't it just be easy to combine
the two? But it's interesting if you think about what
is probably the top rated frontier model right now is
probably Gemini 3 Pro. Right? And that was trained on
exactly zero Nvidia GPUs. So Google have had the advantage
of using their own hardware. They trained the entire model
on TPU's and it was a massive pre training effort
as well. I mean most of the work seemed to
go into the pre training. So that's an example where
owning the hardware and building the model has really worked
out very well because Google has just been able to
make such advances with that. So it will be interesting
to see how these Nvidia models come along as well,
I guess. Kush, anything you'd want to flag in terms
of model architecture otherwise? I know this is again sort
of multi agent, what everybody else is kind of doing,
but anything unique you think worth pointing out? No, I
mean again, on the narrative point, the architecture is this
hybrid. It has a bunch of transformer layers, it has
some Mamba layers, which is a state space sort of
model for long range dependencies and a mixture of experts.
So our name of our podcast in there, but that
combination is the hype again is kind of saying that
oh, this is something unique and new and great and
it can deliver all sorts of performance and efficiency and
makes sense for agents and stuff. And all of that
is true, I would completely agree with it. But it's
not like they're the first to come out with it.
So I think all of this is, I mean, coming
back to the Time magazine thing, right? I mean it's
like what are you centering? It's the hype of it
in addition to the fundamentals. Yeah, that architecture, that sounds
an awful lot like Grande 4, doesn't it? Kirsch, the
Mamba, the transformer and the mixture of experts all together.
Everybody's been very on message today. We've got lots of
IBM references coming in, I guess. Mario, on a final
note, do you want to do any comparison between this.
And Angranin, the actual architecture itself? As I was reading,
I was like, hold on, am I reading the Nvidia
one or am I reading the website, the grand one?
Because it's awfully familiar and they have the same idea
of multiple models as well, which I think a lot
of these open models are Moving to now. So they
have nano, super and Ultra, where they have 30 billion
is nano and 500 billion is ultra. But because it's
a mixture of experts model, you only use about one
tenth of those parameters at inference time. So that, that
30 billion model, the nano model, actually only uses 3
billion active parameters at inference time. So it does mean
that you can take these models and run them on
some pretty small devices, which I think is quite interesting.
I'm going to move us on to our final topic
of the day. So this is a fun story actually
from a number of weeks back. We had scheduled to
talk about it when the news had broke, but just
there's been so much other things happening in AI that
we're only addressing it now, you know, towards the end
of December. But I did think it was a pretty
interesting story and it's a way to talk about what's
happening in model alignment and model safety. So kind of
a fun little sort of disclosure happened. A kind of
independent researcher was digging around with Claude and sort of
uncovered a document that is used in the training process
that Anthropic calls its Claude Soul Document. And what the
Soul Document is, is basically it's a very long sort
of unique in a couple ways. I think the way
the document is drafted is a lot more sort of
narrative and philosophical than a lot of safety documents that
you might have seen. That said, look, here's a long
list of things that we don't want the model to
do. Maybe I'll turn to Marina. What should we make
of Claude having a Soul Document and what exactly is
going on here from a technical standpoint? What is this
actually being used for? So I think it's very in
line with Anthropic's perspective and the way that they want
people to think about how they develop their models and
their company. And I mean, I like Claude, I've always
kind of liked the Claude model. I like the way
they write and what they do. So I think that
something that's interesting here actually is that this document, if
I understood the coverage correctly, is being used at fine
tuning time, not just something that's in fact in the
prompt. And this is interesting because it's different than I
think what a lot of other models do, which is
just do a whole bunch of fine tuning or RL
techniques that don't have any kind of framing around the
examples. The example is just here's a really Specific task,
really specific question. And then this answer is better than
that answer. I'm simplifying. But you know, you keep going
kind of that way and not having anything around that
of. Oh, and you should think about how to answer
this in this way. In reference to this, in reference
to this, in reference to this. So as a result
of doing it during that time early, earlier in the
stack, think of it that way. This means that this
is reinforced over and over and over and over again
in sort of the model's own parameters. Now, yes, calling
it a sole document is cute, but there is a
sense of almost like a value and a structure of
no matter what the task is, you ought to be
referencing a whole bunch of things beyond just the concrete
question and answer. And I think there's something in that.
And what I think is the case is that a
lot of people maybe do that without being so explicit
about it, where because of the default system prompt that
you choose to have or not have as you train
your model, you kind of maybe have your own version
of that document. But it's maybe very small and it's
maybe not very intentional. Whereas it does seem like what
they have in Anthropic has been deployed in that sense
very intentionally. It does result in a somewhat different personality
of a model because you end up building different biases.
And I use that word in a technical sense. And.
Yeah, so I think it's just a slightly different perspective
on when do you put this information sort of into
the model earlier up the stack. And that itself, I
think is worth looking at, looking into how much of
a difference that makes. Yeah, and I kind of offered
a question, I guess, Marina, which you'll probably have the
answer to is like, why haven't we done sort of
model alignment in this way before? Like, why have we
leaned so much on prompting versus just like working it
into the fine tuning? Because I think what you're saying
is really true, Right. Which is. Well, we kind of
want this set of principles kind of embedded into the
behavior of the model. But I think the fine tuning
thing is a little bit different from how a lot
of people do it. Yeah, maybe Kushal have perspective on
this as well. But I think that one thing is
that it becomes a little bit more difficult to do
things from an evalu evaluation perspective of like, well, if
you have all of these things as also part of
what you're going after, can you really tell that this
particular answer that you're offering is actually better than that
particular answer that you're offering? I wonder how much time
they spent not only on figuring out this document, but
figuring out how they need to change their training data
in order to account that they're actually training in line
with what is there. Because most of the time we
don't do that. And then the data sets that we
rely on or that we construct don't do that. They're
a little more straightforward, they're a little bit more focused,
or actually a lot more focused than that. As we've
been doing this for granite, I mean, we've built up
that experience as well for kind of the safety alignment
or morality alignment or whatever have you. And I think
the, I mean the evaluation is clearly, I mean, part
of this, like how do you know which behavior is
preferred or not? And these sort of things. But I
think there's also the modularity question because once you do
the supervised fine tuning or similar sort of fine tuning
sort of things, it's like fully baked in. And not
every use case is exactly the same, especially for the
types of customers and use cases that we often think
about. So it's just, I mean, maybe like too heavy
handed in some fashion because anthropic in their thing, it
states that they want this to be an expert friend
of some kind. And not every LLM should be your
expert friend. So breaking it up, kind of having the
options to turn things on and off, do things in
the way that makes sense for your use case, I
think is another driving factor. Yeah, totally. And I think
these trade offs get very interesting. I think, think Marina,
your comment about, oh, they're actually giving up some evaluation
ability by doing it this way is pretty interesting. Right?
They're kind of like, well, we have a harder time
measuring this. I don't know if this is what you
meant, but it's like we have a harder time measuring
this, but we think it's more aligned if we do
it this way. Martin, I think Marina was being pretty
nice about this, but I think one of my instincts
on reading this was like, wow, this is so anthropic.
This is a very anthropic Y. This is like the
most anthropic. Yeah. Document I've ever read. And I think
it's easy on some level to kind of like eye
roll and be like, oh yeah, sole document. Very, very
anthropic. But as, yeah, I kind of agree with Marina.
Like out of all the models that are currently operating,
Claude's just the most pleasant to interact with right now.
And I don't know, I guess what I want to
Talk to you about is obviously you have a very
literary bent from your guest appearances on this show. And
I guess I'm really curious about whether or not the
way these documents are written have something to do with
this sort of like. Like very hard to quantify quality
that we like in something like Claude. Yeah. So actually
reading the sole document was so interesting because I agree
with you there, Tim, that the nicest model to chat
to, I've always found, has been Claude. For the last
couple of models, I'm thinking of myself. When I prompt
a large language model, I'll read it back, my prompt,
and I'll think, if I gave this to a human,
am I giving them a decent chance that they can
actually perform the task I'm asking them to do? Am
I giving them enough information? I'm editing a document and
I just respond and say, make it better. Well, that
hasn't really given the model a very good idea of
what to do. Right. So I try to be quite
specific. So I was interested to see what anthropic we're
going to include in this sole document that would really
guide the model. And the part that I read where
it said anthropic generally believes this is from the soul
document, it generally believes it might be building one of
the most transformative and potentially dangerous technologies in human history
yet presses forward anyway. And I'm thinking if I tell
that to a human that I'm building those potentially dangerous
history thing in history ever, how does that human proceed?
I have no idea. So how that model is supposed
to take that information and do something with it? Well,
I'm looking forward to finding out. Krish, maybe I'll give
you the last comment here. And this kind of goes
to what I was just asking Martin about is how
much is what's in here actually impacting model behavior and
how much of it is kind of poetic license or
almost like literary flourish, I feel is like, this document's
a real pleasure to read. But I think where I'm
left with at the end of the day is like,
well, how much of this actually impacts how the model
behaves? Yeah, I mean, I think it does have an
effect, certainly. So could it have been more concise? Maybe,
but I think a few different interesting parts. One is
that they do discuss uncertainty, value uncertainty and calibration and
these sort of things in there. And as people, I
mean, most of the things that we encounter in life,
we're also uncertain about. We don't know what we believe
until we kind of encounter it. And so the Fact
that they're going through all of that sort of uncertainty,
or in this case, in this case, how should you
reason about it, is actually a nice thing. And that
can't be done very concisely. So I think that's an
aspect of it. Another thing I've been waiting this whole
podcast history to talk about a little bit is some
of the moral philosophy aspects as well. So, like, I
think there's a little bit of confusion in, like, it's
very anthropic, as you just said. But if you step
back and look at kind of the. The philosophy of
it, maybe it's trying to do too many things at
the same time. And there's this concept of kind of
dualism and non dualism in a lot of moral philosophy
that is the soul of individuals separate, separate for each
individual, or is it kind of all like universal, all
the same thing? And I don't want to get too
philosophical, but it's important here. And the reason it is
is because every instance that a person uses the model,
it's kind of like a new birth, it's a new
session. And so is this soul document really meant to
be, like, universal, or is it meant to be kind
of individualized for the session? And if it's really meant
to be universal, then why is it talking about being
a brilliant friend? Because every context needs to have a
separate sort of soul in that case. So it's just
very confusing of what the exact goal actually should be.
Well, and so what's the end result? Do you feel
like that's a problem for the model? If you just
use it for this very narrow sort of type of
use, then it's fine. But if it's really a general
purpose technology that you're going to use in a lot
of different situations, then I think it's too prescriptive in
certain ways. Yeah, for sure. Yeah. That's a good reminder.
Question. I didn't know you wanted to talk about that.
We should definitely. I'm going to work out an moe
segment next year where we just do moral philosophy. Yeah,
absolutely. I think it'd make for like a super fascinating
episode. Marina, I thought of one last question, so maybe
we'll actually have the last word go to you. A
few years ago, I know I was very excited about
prompting because I was like, okay, for someone who has
over time become more of a writer than a coder,
this is very exciting. And at the time I had
a couple researcher friends who were like, don't invest too
much in prompting. We're going to figure out how to
automate it. Prompting is going away. It's just a temporary
thing. And here we are at the end of 2025,
the architects of AI have done their thing and AI
is bigger than ever. And if anything, the fact that
anthropic is doubling down so hard on this kind of
document, which specifies, as Kush said, the moral philosophy of
these models, are we going to be living with prompting
for much longer, or is this kind of similarly just
like a temporary thing in your point of view, Prompting.
Is a way to get information into the model. It
is certainly a very simple and straightforward way. It doesn't
mean that you know what effect it has when you
get it in. But I will say that a lot
of times when you go in these larger systems, the
prompting does kind of start to dial down where the
prompting is. Yeah, maybe you specify slightly what you want,
but the way that you actually end up executing is
not via prompt. You end up telling the model roughly,
oh, I kind of want you to do this. And
then there's other intermediate steps. Oh, now I'm going to
do this, this, this, this, this. So out of Prompt
Engineer, you end up with like, you know, agentic flow
engineer. And so, yes, prompting in this sense is going
to be part of it, because then you're trying to
get information into the model in a particular way. But,
yeah, I agree with your friends back then who were
like, yeah, we're going to figure this out, because remember
where we started, we started with you prompted and you
put in two spaces instead of one space, and the
model just like, I don't know what to do. I
don't understand. I don't understand. Okay, we got past that.
But the. Yeah, it's good. We're okay now. But the
base idea remains sort of the same of like, look,
you're trying to get information, and the model either can
kind of figure out what you mean or it can't
quite figure out what you mean. Now, it just won't
tell you that it couldn't figure out what you meant.
And it continues to keep going. But I think we
are going to be moving beyond this in terms of
ways to inject information into the models and as we
go beyond the models, ways to inject the information that
we want into the use of the models. The models
themselves are a means to an end most of the
time. Right. So especially as Kush was referring to enterprise
solutions and real use case solutions are going to be
a means to an end. And at that point in
time, yeah, we're going to be moving beyond something as
fragile as prompting. And also, I guess come to think
of it, like these types of fine tuning approaches as
well, right? Where you take a sole document and try
to fine tune it in. Yes. Yeah. Incredible episode. This
ties together like so many threads from the last 12
months. Martin Kush, Marina, thanks for joining us so late
in December. And that's all the time that we have
for today. If you enjoyed what you heard, you can
get us on Apple podcasts, Spotify and podcast platforms everywhere.
And we'll see you next time week on Mixture of
Experts. Experts.