AI's Promise Against Infectious Diseases
Key Points
- The panel debated whether AI can truly eradicate infectious diseases, noting that while AI has accelerated drug discovery, viruses evolve faster than current algorithms, making a complete solution unlikely.
- Dario Amodei’s “Machines of Loving Grace” essay sparked optimism by forecasting AI‑driven scientific breakthroughs, massive GDP growth in developing nations, and even world peace, but many experts cautioned that such visions overlook practical and ethical constraints.
- Speakers highlighted that the impact of AI will depend heavily on how humanity chooses to deploy the technology, with competing interests and regulatory frameworks potentially limiting its benefits.
- Scaling AI capabilities remains a technical hurdle, and achieving the promised health outcomes will require parallel social and policy changes to address misinformation, equity, and responsible use.
Full Transcript
# AI's Promise Against Infectious Diseases **Source:** [https://www.youtube.com/watch?v=BWKFzWUOBOg](https://www.youtube.com/watch?v=BWKFzWUOBOg) **Duration:** 00:41:06 ## Summary - The panel debated whether AI can truly eradicate infectious diseases, noting that while AI has accelerated drug discovery, viruses evolve faster than current algorithms, making a complete solution unlikely. - Dario Amodei’s “Machines of Loving Grace” essay sparked optimism by forecasting AI‑driven scientific breakthroughs, massive GDP growth in developing nations, and even world peace, but many experts cautioned that such visions overlook practical and ethical constraints. - Speakers highlighted that the impact of AI will depend heavily on how humanity chooses to deploy the technology, with competing interests and regulatory frameworks potentially limiting its benefits. - Scaling AI capabilities remains a technical hurdle, and achieving the promised health outcomes will require parallel social and policy changes to address misinformation, equity, and responsible use. ## Sections - [00:00:00](https://www.youtube.com/watch?v=BWKFzWUOBOg&t=0s) **AI's Fight Against Infectious Diseases** - A panel of AI experts debates whether AI has eradicated natural infectious diseases by 2034, highlighting both optimism and the challenges of viral evolution, technological limits, and competing human interests. ## Full Transcript
we're jumping ahead it's October 17th
2034 has AI helped us solve nearly all
natural infectious diseases my mad is a
product manager for AI incubation Maya
welcome to the show um what do you think
thank you for having me so of course The
Optimist in me would love to say yes but
um I don't know if history has always
proven us right and I think it really
depends on how we choose to use this
technology Kar El mcrai is a principal
research scientist AI engineering at the
AI Hardware Center cter welcome back to
the show um tell us what you think thank
you Tim it's great to be back well AI is
making strides in tackling infectuous
diseases but it's not a Magic Bullet
viruses evolve faster than algorithms
and the battle between pathogens and
progress is far from over so there is a
lot more work to be done all right so
some Skeptics on the call and finally
last but not least in joining us for the
first time on the show is Ruben Bonin CN
capability lead for adversary Services
Ruben welcome and let us know what you
think
thanks uh glad to be here uh I think we
can get there uh provided you know
scaling continues uh but I think it's
mostly going to be an issue of competing
human interests if we do all right great
well all that and more on today's
mixture of
[Music]
experts I'm Tim hang and it's Friday
which means that it's time again to take
a whirlwind tour of the biggest stories
moving artificial intelligence we'll
talk about a hot new sampler that's
getting a lot of attention and apple
reing on ai's Parade but first I want to
talk about Machines of Loving Grace an
essay by Dario amade who's the CEO of
anthropic and he makes some very wild
predictions he says that AI might solve
all infectious diseases it could 10x the
rate of scientific discovery he promises
that one you know wild but not
implausible outcome is 20% GDP growth in
the developing world and potentially
even World Peace uh and so I think I
just want to kind of uh bring this topic
up because the essay has been getting a
lot of play and a lot of people have
been talking a little bit about it and I
guess Maya I'll start with you I mean
how believable do you think are these
visions and you know what is more or
less believable in in what Dario is
predicting here so Dario definitely
paints a picture that we would all love
to believe in but of course people are
going to be skeptical because a
technology which is a tool can be used
in different ways and currently the way
that we're seeing AI being used um is
not materializing necessarily in all
this optimism that he said it's a mixed
bag of how it's being used um so
definitely there were advances in uh
drug Discovery um but at the same time
we're seeing articles about the rise of
misinformation so I think the article
overemphasizes the positive and I don't
think it also sets in motion what are
the prerequisites to get to this
positive picture um I think it's going
to have to come hand in hand with a lot
of social change and not just a
technological change yeah so you think
the end result of AI is likely to be
neutral if anything else is that right I
don't think technology is neutral I
think I think how you put it in motion
there's definitely an agenda and a
social context and an e economic context
behind it and then that just unleashing
it in different directions yeah for sure
Kar I want to bring you into this
discussion because I know when you
responded to that first question you
seemed a little bit more skeptical I
don't know do you sort of agree with
Maya that this is sort of achievable
ultimately or you just kind of thinking
this is I don't know marketing or over
optimism about the technology I think
certainly there are lots of things that
we can achieve with AI of course there
is also a hype so in in Dario's essay uh
he explored the potential and also some
limitations of AI and how it might shape
society as it advances so one thing I
particularly found interesting is how he
emphasized the need to rethink AI as a
powerful uh as powerful Ai and and also
tap into the potential but also there
are lots of challenges that I see and
that requires you know continuous work
continuous progress continuous algoritms
so for example if you look in biology
and health which you know he wrote a lot
about that so I mean we've seen what AI
can do a lot of strides and how it can
significantly enhance research in
biology and medicine but progress is
often constrained by the speed of
experiments availability of quality data
regulatory Frameworks like for example
clinical trials and uh and you know
despite you know all of these
revolutionary TOS like for example Alpha
fault uh you know there there is needs
you know also to have things like
virtual scientists driving not just data
analysis but also the entire scientific
process and I think there's a lot of
work to be done there um if you look for
example if we want to look at it from
pragmatic versus long-term impacts in
the short term AI might be limited by EX
infrastructure and societal barriers
however over time I hope that these
things you know can be resolved and that
the intelligence can create new Pathways
for example reforming the way
experiments are conducted and reducing
bureaucratic inefficiencies through
better system design so it has to be
kind of a collaboration between the
intelligence and also society and humans
uh and things need to be regulated
because also like Maya mentioned there's
all these fake news and art and data so
there is also that danger that we have
to be careful about you all those these
threats and I think we're going to talk
about that later also so how do we how
do we balance all of this so we can push
it towards a direction that is
productive that is helping us and not
that not you know in a direction that
can impede our progress or create issues
yeah for sure Kar I think one question I
want to ask you before I move on to
Ruben because I think he's he'll have
some real interesting angles on this
because he works on the many ways these
systems can break or be used in you know
not so great you know um for not so
great purposes uh you work a lot on
Hardware um and I think part of Dario's
dream is the idea that eventually these
systems will be able to control sort of
physical robotics out here in the real
world um and that will be just this huge
kind of like boost to the effect this
this technology has do you buy that are
we close to that kind of world where
it's really easy to kind of instrument
these models to kind of control real
world systems are we still pretty far
away from that you think I think we're
we're making progress towards that
there's a lot of work right now on
making the hardware the infrastructure
more efficient and sustainability is a
big part of that because right now you
know we're hitting the physical limits
the limits of physics so and there's a
lot of work you know needed to create
you know these chips that are capable
of acting in resource constraint devices
especially with you know this huge
compute needs that AI keeps driving
forward words with things like large
language models and so the computational
needs are just growing and now that if
you want to also do things like
reasoning and so it's going to be kind
of an arms race you know more is
required algorithmically but at the
compute side there's a lot of
innovations that needs to be done at the
semiconductor level at you know the
physical the material science all of
that to create you know these chips that
are capable of handling this huge demand
while still doing it in a sustainable
way and the cost effective way K sport
like this subject was not addressed in
this article at all like I think it was
overly optimistic that yeah AI will
solve climate change but in developing
AI we're actually like missing a lot of
sustainability targets that companies
has set and that was like not at all
addressed so if I want to use it to
solve climate change I don't want to
have data centers that are also emitting
tons of carbon and consuming tons of
energy to solve that problem
um Ruben maybe I'll bring you in because
I think as a security guy I mean my
friends who are security people look at
this kind of essay and they're like this
is ridiculous right like this technology
is largely going to be used for like you
know bad purposes or you know these
systems will be so vulnerable that
they'll never actually achieve kind of
the full potential um how do you size up
these claims I guess as a security
expert like you know do you do you sort
of buy into the optimistic Vision here
are you more skeptical I I I am an
optimist uh personally yeah but uh like
I mentioned in our introduction as well
I
think you know the technological
achievements are one thing but then how
do people with competing interest manage
the outcomes of those achievements I
think is something else uh for example
like in the article they talk about or
he talks about um sort of author
authoritarian regimes and um how you
know AI systems clearly have
applications to you know restrict what
people can do how they can think and
manage all of that and I think we can
already see some of those Dynamics at
play like currently in the west and the
East we've sort of diverged on uh AI
development paths and I think you know
those things are going to continue as we
get closer to those you know more
powerful systems uh
also also I think for example with
medical advancements I don't want to
make any proclamations if what he says
is possible or not I don't think I'm a
subject matter expert in that area um
but it will depend then as well if
companies are willing to make those
advancements
available uh at um to people who may not
be able to afford them um currently and
how that distribution is made uh among
the population you know and then finally
what I want to mention also is that um
we talked a little bit about this
information already and we'll talk about
that later I think U but uh one thing he
didn't mention in the article is
education uh which is something uh I'm
personally very hopeful for that uh more
free access to information and high
quality AI assisted education is going
to be a big uplift uh for a lot of
people and I think will also help this
sort
of uh um making our society sort of more
democratic and more accepting of these
Technologies because I think a lot of
times when there is some conflict it's
also because people don't have sort of
the same basis to understand like the
facts for example with vaccine anti
antiv vaccination campaigns and things
like that so I think it's a complex
[Music]
picture so I'm going to move us on to
our next topic um one of the things I've
been watching most carefully in watching
the kind of X Twitter chatter on AI um
is a bunch of hype around this repo
called entropic effectively the story
behind it is that it's an AI researcher
that has introduced kind of a sampler um
that effectively attempts to replicate
some of the cool you know Chain of
Thought Fe features in effect that we
saw um for the open AI 01 release uh
just a few weeks back um and I guess my
I'll turn to you because you're gonna
have to help me out here a little bit is
what is a sampler anyways and why should
we care oh I love this question um so
yeah I spent quite some time focusing on
llm inference so when we talk about AI
we mostly mean large language models
what a large language model does is
given the start of a sentence so few
words it would predict what is the next
word so if I say on the table there is a
automatically in your head there's a few
probable words that pop up a t there is
a book there might be a glass of water
Etc so the model does something similar
there's a statistical representation of
all possible words that could come next
and then there's a probability
attributed to each word to the book to
the glass Etc and all of these
probabilities are based on the data it
has seen in the past so the mo the these
models are injected a lot of data and
then based on what I've seen in in the
past it kind of says most logically this
is the next word that's going to come
next so what a sampler does is it
determines given x amount of words that
the model has seen what should the model
output next and the the sampling
technique that's most widely used today
is called greedy and by greedy we mean
uh just outputting the the token or the
word that has the highest statistical
probability um so I hope I answered your
question on what is sampling um I think
this paper is really interesting and um
takes advantage of additional
information that um we can get uh out of
large language models and out of the uh
yeah acquisitional metadata that we have
so I think it's an interesting paper and
yeah happy to understand more about
other people's thoughts on it yeah for
sure and I guess maybe Kar I'll throw it
to you is you know I think one of the
most interesting bits about it is it
introduces a new sampler um and I think
the promise of it I think one of the
people reason why people are so excited
about it is like oh it really seems to
boost the performance of these models
against all these different types of
tasks and I think the other interesting
thing is that it seems to kind of like
replicate in part like as I mentioned a
little bit earlier like what open AI
kind of touted as its special sauce for
its new great model and I guess you know
I'm sort of sitting here thinking like
well you know open AI seems like you
know the Goliath in the space because
they can do all these crazy cool new
algorithmic changes or improvements on
their model but do you think that the
existence of something like in Tropics
means that like you know open source
will almost be getting as good as fast
as you know these kind of proprietary
models and what these proprietary
companies can do um you know it almost
seems like maybe there actually is no
Special Sauce because some random
researcher can just launch this repo
that that seems to do maybe something
close to what these big companies can do
yeah I totally agree with that and
actually I love what in Tropics is doing
I think they are having an Innovative
approach here that reflects also this
fast moving evolution of open source AI
Community where new methods like these
adaptive sampling are explored without
requiring massive computational
resources which is key here but also
demonstrating also the collaborative and
experimental nature of the field we can
explore more you know in open source and
kind of mimic or even even exceed you
know what the secret sauce of you know
these big companies are doing so of
course entr Tropics aims to replicate
you know some of the unique features
associated with open AIS o1 model models
particularly in the reasoning
capabilities and they have you know this
interesting ways of experimenting with
entropy based and VAR they call also VAR
entropy sampling techniques which kind
of tries to reflect the uncertainty in
the models next step or examines also
the surrounding token landscape and
helping the model decide if it should
Branch or resample based on future token
possibilities really interesting
approach and I think at the end of the
day open source is going to kind of
catch up with what's happening a lot of
innovation happening there and we see
that not just in these algorithmic
things but even with efforts like Triton
for example on the on the uh GPU
Hardware or the accelerator side there's
a lot of work also happening in open
source to kind of go to cou free or you
know and you will see a lot of these
things for example in the v Lambs or
where what's happening in open source is
kind of on par with with the some of the
secret sauce that propriatary companies
are doing in the space of AI across all
the stacks what I think is also
interesting is open source is is giving
kind of the in all the ingredients for
free and with approaches that are more
accessible to everyone in the field so
to explain my point what open AI did
with oan is take a big Frontier Model do
a lots of reinforcement learning in
order to train it on how to do Chain of
Thought reason in at scale what this
open source repo did is take an open
source model llama 3.1 and bypassed all
this reinforcement learning that openai
did and take and take advantage of an
innovation or this additional
information that you get at inference
level so like Kar said um the the model
has ways of of telling us that it's
uncertain of the next token to predict
so for certain situations you could see
with high probability it's going to be
this word but there might be Forks in
the road where lots of different options
are equally probable so taking advantage
of this sort of information you could do
a lot about it um in this repo they
propos to do Chain of Thought or start
from scratch but I'm actually quite
interested in uncertainty quantification
as a means of giving information and
tools for people to use this models in
different ways so if the model could
tell you the answer is uncertain you
could use that to build different
systems to output something
so I think the choice could be different
than what uh this repo does but I do
think it's an interesting research
Direction yeah and I think that's such
an interesting subtlety here is that
it's not just kind of replicating the
end results but this engineer seems to
have basically found a way to do it a
lot cheaper basically it's just like we
just edit the sampler uh rather than
having to do this completely complex
kind of reinforcement learning uh
process this is also encouraging like
deeper reasoning through token control
at inference time so it's kind of Paving
the way also opening different like may
also mentioned this figuring out ways
how do we do these sampling these
selections this at a much deeper and
incorporating other informations about
the uncertainty of the model about also
the future predictions that you can do
about the model to to do the right next
steps so I think um this emerged as an
like a joke uh in the last episode but
I'm thinking about turning it into a bit
for mixture of experts which is we got
to talk about agents on every single
episode it's just like part of what we
need to do uh I guess my in particular
you offered a question when we were
talking about this episode before we
were doing the recording about kind of
the relationship between these types of
uncertainty systems um and kind of like
getting more agentic behaviors out of
these models um do you want to talk a
little bit more about that because I
think that relationship is really
interesting and it's not maybe entirely
clear I think for for some folks who are
not as deep on it as you are first of
all any model can be any model of a
certain size and that respond well to
Chain of Thought kind of stepbystep
thinking with thinking between quotation
marks can be turned agentic now how well
and how good that will perform is up to
the inherent model um and it's it's
performance on various benchmarks and
then we're going to be talking about
benchmarks in an upcoming session um
what is interesting about this new
innovation so taking advantage about
information about
uncertainty um I think this could be
really interesting in the context of
agentic systems because you can
basically stop an agent in its tracks if
it's uncertain of the next step and I
think agents right in the agent World
we're facing a lot of problems with
reliability and actually users are over
trusting the agent's performance because
it looks like it's performing in a way
that is human relatable so it's thought
step by step there's a plan the plan at
the high level seems reasonable actually
catching hallucinations and an agentic
approach roach is harder than just text
in and text out so I think this is a
uncertainty quantification is a tool
that I think would be really important
to bring agentic systems to the next
level and I see it being used in
multiple ways stopping an agent in its
track maybe um based on the repo that
we've seen maybe just starting again or
starting a new Chain of Thought uh
workflow so I we're at the very
beginning of this but this is something
that on my team we've been discussing as
well uh as a really interesting research
direction to inter into our work I think
it kind of goes line in line with what
the agentic approaches is doing because
what in Tropics is doing it's
introducing this entropy based sampling
and with the v v entropy technique you
know they're assessing future token
distributions so and this is what you
know also agentic system the behavior
here requires foresight and planning and
mimicking humanik flexibility and
dynamic and that adaptive decision
making so I think they're kind of go
handin hand here and there's a lot here
that can be learned from you know each
way from the agentic systems you know
they could incorporate those techniques
uh to have this humanik flexibility and
foresights or vice versa I think it's
exciting um uh as the other two um
panelists mentioned that uh there is
this real push in open source which I I
don't know how well we can quantify if
it's cing up to sort of Frontier models
or the efforts that you know those
companies are doing but I think that's
great that this is happening uh in the
public yeah for sure and I think
basically to what Maya said earlier I
think we will see more of the kind of
pattern that we see here which is it's
possible that open source may be very
clever about kind of solving the problem
in a much more resource constrainted way
which actually may keep it ahead of like
the proprietary models and they're kind
of like much more expensive approach to
um some of these problems so definitely
another Dynamic that we'll be returning
to in future
[Music]
episodes so I'm going to turn us to our
next topic um Apple released a paper
that was of some controversy um recently
um and uh I was joking a little bit
earlier in the intro that they kind of
are reigning on the AI parade um
effectively what they did is they took a
benchmark called GSM AK uh which
contains a variety of mathematical
reasoning questions and what they did is
they said okay well we're going to do
this we're going to make some quick
variations to this Benchmark and create
a new Benchmark which we call GSM
symbolic um and these changes are very
very small and subtle and don't really
change the substantive nature of the
mathematical problem so you could
imagine kind of like a grade school
question about you know John having 10
apples and need to subtract three apples
and add four apples and kind of what
they're doing is they're saying okay
well rather than John we'll talk about
Sally and rather than Apples we'll talk
about pears and maybe rather than 10
apples the person will have 12 apples um
and what they find is that these really
kind of small changes can create some
pretty significant drops in performance
uh of the models against these
benchmarks so on one level we kind of
know this right which is that there's a
bunch of overfitting on benchmarks and
people are always kind of like gaming
the benchmarks and models look better
against these benchmarks but this is
also kind of worrisome uh maybe Ruben
I'll toss it back to you right because
it sort of suggests that like maybe
these models reasoning is actually
nowhere near as strong as we we think
they are they appear to be I don't know
if you buy that conclusion yeah I mean I
I think it makes okay first of all like
it makes sense that people want to
Benchmark uh models that get released
and so I think there is an incentive for
companies to also do well on those
benchmarks uh because otherwise people
are going to say oh okay this model
isn't appreciably better than it was
before uh and obviously public data will
end up in training data for these models
so I think I think that makes sense uh
but when I looked at the figures uh in
the paper I thought or I I saw like they
have different sort of tests that they
ran the models through uh one is like
you mentioned they changed the names and
maybe the figures or the objects uh and
there was a drop I think between 0.3 and
9% or something like that uh but looking
at sort of the more Frontier models I
think the drop was not really that large
uh in my opinion like I think for GPT
for R it was only 3% or or something
like that uh and then they
had some other harder benchmarks where
they added and removed conditions to the
statements um or even added multiple
conditions uh where there were much
larger drops like I think up to
40% for 40 mini I think I would have to
look at the paper to get the exact
figure I think it was up to like
65.7% in one of the worst yeah and so I
think even for sort of what we consider
the frontier models you have a lot of
drop there um but then you know when we
have been talking about reasoning and
Chain of Thought I think you saw that
the 01 benchmarks dropped by
substantially less it was still a lot it
was like 177% or something so I'm not
really sure how to feel about the
results of this paper or what they mean
or if this is a problem that will get
resolved over time as reasoning gets
better uh in these types of models or
not yeah for sure and Kar you just
chimed in there I don't know if you've
got views on this paper and whether or
not it's you know I guess you made it
sound I don't want to put words in your
mouth it's kind of like me big whoop
right like we kind of know that these
models have lower performance when you
change the benchmarks and even then the
effect doesn't seem that big and so
maybe not too much to worry about um I
don't know if C you feel the same way I
think some of the results were
surprising to me and this this work from
these Apple researchers it kind of
provided a very critical evaluation of
the reasoning capabilities of flowers
language models from what I saw they're
kind of exposing that llms Reliance
especially on P pattern matching
is is Big right now rather than really
true reasoning so because I I don't
think that the LMS are really engaged in
formal reasoning but instead they use
sophisticated pattern recognition uh and
this approach of course is very brittle
and prone to failure with these minor
changes that they have exposed um so for
example if you look at the the GSM
symbolic test performance so they
created you know the variations like uh
Ruben uh mentioned but with the you know
and what they're seeing you know these
drops uh sometimes can be very big if
they just include irrelevant things to
the problem the reasoning should stay
the same but if you just say oh you know
these uh apples for example some of them
are smaller than others which is not
doing anything you know to the reason in
itself it's just additional irrelevant
information but you know the lam was
taking that and actually was taking the
smaller apples and use that in the
calculation so and another thing that
they expose is the variations the
inconsistent results across the runs so
they showed very high variance between
different runs with the same models
which highlights also the inconsistency
even slight changes in the problem
structure resulted in accuracy drops up
to 65% in certain cases so I think what
the the key highlight here is the LMS
they Tred to mimic reasoning but mostly
relying on data patterns but their
capability to perform consistent logical
lening is still limited and the findings
also suggest that current benchmarks May
overestimate the reasoning capabilities
of llms and and I think we need improved
evaluation methods to really go the
capabilities of LMS especially with
respect to reasoning I'd love this new
Benchmark that Apple put up I know we've
been on uh previous podcast sessions
where we talked about all the issues
with benchmarks so I think this is a
great step in the right Direction in
order to force um more uh more
generalizable insights based on
benchmarks um I also think for me I it
was really predictable that this was
going to happen um whenever I talk about
reasoning I like to say reasoning
between quotation marks because it's us
andromorph anthropomorphizing what we're
seeing uh coming out of llms and like
Kar said they're doing pattern matching
so it's pattern matching at scale um
they showed the model patterns it hasn't
seen before so you could update the
models log the models training with some
new patterns and can infer can maybe
unlock new use cases based on that
that's great so it's it's a technology
it's an imperfect technology but it can
do useful things I don't think we're in
a world where this current technology
can do logical reasoning um it's just
pattern matching at scale and I think we
have to accept it for what it is and
when we're thinking about making these
systems useful I think we're always
going to be in a scenario where there's
going to be a human in the loop or on
the loop we need to have we need to have
ways of uh surfacing whether there's
high confidence or low confidence in the
llms trajectory so I think we have to
use these tools and use this knowledge
that it is an imperfect technology to
make it more robust and there's a lot of
papers that say take taking this sort of
Technology with humans can increase the
overall robustness of the system if we
factor in a human as part of the system
and I think we should accept that as
opposed to thinking I think with the
current technology we have we're on a
Pathway to what is called AI yeah for
sure and I'd love to make that with like
very concrete with maybe a last question
to Ruben you know right now we've talked
about this in previous episodes there's
a lot of excitement about say using AI
to you know Harden computer networks
right as like a complement to cyber
security as a form of cyber security
defense and I guess on the framework
that Maya just laid out it is kind of
interesting question is like is cyber
security a pattern matching question or
is it a is a reasoning question right um
because I guess it would suggest here
that if a lot of what we're doing in
cyber security defense is just pattern
matching well okay maybe the technology
really has some very strong legs here
but if something more is needed there's
actually some really interesting
questions about whether or not it's kind
of fit for purpose just a final thought
I'm curious about whether or not you you
agree with that framing yeah I mean
security is um vast and complex right
doain and then in some cases there are
like reasoning is very important but in
other cases it's all about data
collection correlating those data
sources and summarizing and I think for
many years already there has been use of
sort of traditional machine learning uh
in uh endpoint detection and response
Solutions uh to great effect by the way
just want to say that uh and then now
with generative AI there's a lot of a
lot of push to integrate that also um
sort of into the back end where those
events are correlated and maybe
synthesized in a way that people had to
do manually previously and sort of speed
up those processes uh but humans are
definitely involved there they have to
be uh to evaluate those uh events but
yeah I think it's going to be big yeah
for our
[Music]
industry so we're going to end today's
episode on more of a maybe stress
inducing segment um as you know there's
a big election coming up in the US and a
big election uh coming up around the
world um and uh open AI did a disclosure
recently where they observed that you
know they're seeing State actors um
increasingly try to leverage AI for
election interference and this involves
using models for generating fake
articles and fake social media content
and other sort of persuasive tactics um
which I think is is a really interesting
development that finally you know the
technolog is becoming you know mature
enough um that you're sort of
enterprising you know election interfer
really wants to kind of Leverage this
technology into the field um and I guess
Ruben I'll start with you because you
think a lot about kind of security and
vulnerability in these types of systems
um how what do you think about this I
mean is it kind of an issue that we're
going to just be able to solve at some
point is it going to get worse or better
over time I guess one of the really
interesting things I'm trying to think
about is like what's the trajectory of
these types of Trends is this like do we
just live in this world now or or is
this a temporary thing um no I think we
just live in this world okay this is my
heartache um but um yeah I think um you
know obviously AI has a lot of
implications I think what I would
categorize this as is social engineering
uh and there are many varieties of that
there might be persuasive messaging it
might be persuasive generated images or
videos um you know that's one category
which I think where I think the risks
are more immediately evident um you know
there is another category where
malicious actors are using AI to sort of
speed up their malicious attacks uh
where I think that is much less mature
uh at this point uh but when I was going
through open ai's um uh report on this
uh and I think it's great that they're
sort of being proactive and working with
industry Partners I guess to sort of
combat these threats uh as they appear
it must be very new to them as well um
my sort of conclusion was that they
found that there was limited effect uh
from what they saw uh and I think uh the
most effective post was sort of a hoax
post about uh an account on X where it
replied a message that said oh you
haven't paid your open AI Bill U but
they said in the report that this wasn't
actually generated by the the API so I
think the impacts might still be limited
uh but we may also be biased uh in that
assessment because we're obviously
looking only at um threats between
quotation mark that we detected and
stopped uh so it it wouldn't surprise me
that there are actually much more
successful influence campaigns in social
media where we don't detect that because
uh they are not behaving in a way that's
uh sort of out of the ordinary or
they're using self-hosted open source
models to generate that so we don't have
as much Telemetry on what they're doing
and things like that yeah that's a
little paranoia inducing thank you I
think that's that's where we are yeah um
May any thoughts on this I mean I you
know I guess the obvious question is is
there anything we can do to fix this or
is this pretty much just like you know
we're we're doomed to live in a world of
you know fake AI influence operations
all the time now yeah I think it's it's
just the state of the world
unfortunately so there are Bad actors
when social media came about everyone
was really exciting because it brought
us all closer together it felt that we
were all part of one big Global
Community but for Bad actors this means
bigger ski better skill bigger reach and
I think that's the same thing with AI um
I think the world is moving very fast
and I do wonder about the ability of our
society and the people who are putting
their brain power towards solving these
ISS issues about their ability to catch
up with what's going on I think already
in the school system I think we're
already in a state that the school and
educational system hasn't caught up to a
post AI world and I wonder if in the
field of keeping information factful um
and how our society is organized whether
we'll be able to get there um I do think
it should be a concered effort and I
think more uh global focus and public
spending should be focused on these
issues because we need more resources to
catch up to where the technology is
taking us I want to quickly jump in as
well and say I think again I'm coming
back to competing incentives here I
think a lot of times it's not clear to
me
that social media platforms uh have the
correct incentives to say okay actually
we could deploy our own like AI systems
to do like sentiment analysis and see
which posts are promoting
misinformation or giving harmful
information to people or are clearly
like part of some Network that is
generating similar messages because if
those messages generate a lot of
interaction that might be good for those
platforms so there is a problem with
mislin incentives sometimes I think
which is getting in our way as well yeah
and I think that is actually a really
important question is you know it's not
just what the technology can detect but
is it actually being implemented and and
used and what reasons do people have to
actually do that um Kar do you want to
round us out on this one with a final
comment I'm curious about how you think
about these issues and um yeah if you
think we're doomed yeah this is actually
for me it's it's it's a scary
state of course as the technology gets
uh better more sophisticated especially
gen AI these threats are all also going
to get more sophisticated and more
clever in how they can reach uh massives
massive masses and then uh you know try
to to do harm so of course you know to
mitigate the misuse of AI models like
those reported by open AI there is a lot
that needs to be done uh things like
robus AI detection tools how do we
develop and deploy tools that detect AI
generated content uh and also that we
ensure you know fake materials
regulation and oversight governments and
the companies need to work together need
to collaborate to set clear guidelines
and policies for AI use and transparency
and also I think user education is very
important you know public awareness
about AI generated misinformation to
help people critically evaluate online
content not just everything you see in a
website or the internet or is something
that you have to believe so you have to
critically see the content and maybe
figure out other sources is this really
true or not and also I think partnership
across across industry Corporation to
share insights and prevent Mis I think
increasing awareness about this is
really important I mean open AI did some
things clever ways to at least Identify
some 20 operations that they said for AI
uh for Content creation that they kind
of halted and stopped that are focused
on Election related Mis information so I
think we need more of those but again
like uh you know Maya and Ruben said
this is the world we live in so and it's
going to be an arms race as the
technology gets better the threat's
going to get more sophisticated so and
again I want to say when I read open AI
reports I find that the cases they
highlight I would label as sort of low
sophistication uh in many in sort of
across like the different use cases or
some properties of those campaigns that
they detected so I wonder like with
really good engineer ing efforts right
like if there could be campaigns that
it's not easy or possible to detect that
they're happening so I think yeah I
think this problem is just going to get
yeah especially if they use proprietary
models that you know outside the scope
of open Ai and other Frontier
models yeah what is the like yeah that's
that's a really intriguing outcome that
I haven't really considered as what's
the what's the evil open AI right like
is there an evil Sam Alman that's
running a a criminal Foundation model
like presumably yes right like I think
that's definitely something that exists
so you always know what you're going to
get when you tune into mixture of
experts uh we've gone from solving all
infectious diseases and 20% GDP growth
to uh Sinister invisible influence
operations controlling you as we speak
um so from the very good to the very bad
of AI you'll always get it on mixture of
experts um cter thanks for joining us uh
Maya thanks for coming back on the show
and Ruben we'll hope to have you on
again sometime uh if you enjoyed what
you heard you can get us on Apple
podcast Spotify and podcast platforms
everywhere and uh we will catch you next
week here on mixture of experts