Hybrid Cloud: Key to Generative AI Success
Key Points
- Effective generative‑AI deployments rely on a well‑designed hybrid‑cloud foundation that balances latency, cost, and data‑management requirements, not just on the AI models themselves.
- Many organizations overlook hybrid‑cloud architecture because excitement around “hot” AI technologies distracts them from the underlying infrastructure needed for scalable, reliable AI solutions.
- IBM’s AI‑in‑Action series highlights how integrating hybrid‑cloud strategies with AI can unlock higher innovation ROI and better customer‑centric outcomes.
- Experts Hillary Hunter (CTO, IBM NGM Innovation) and Ashman Hus (IBM Innovation Studio) emphasize that simultaneous focus on AI capabilities and intentional hybrid‑cloud design leads to more successful, enterprise‑grade AI implementations.
Full Transcript
# Hybrid Cloud: Key to Generative AI Success **Source:** [https://www.youtube.com/watch?v=FVXzkUCSu_U](https://www.youtube.com/watch?v=FVXzkUCSu_U) **Duration:** 00:22:23 ## Summary - Effective generative‑AI deployments rely on a well‑designed hybrid‑cloud foundation that balances latency, cost, and data‑management requirements, not just on the AI models themselves. - Many organizations overlook hybrid‑cloud architecture because excitement around “hot” AI technologies distracts them from the underlying infrastructure needed for scalable, reliable AI solutions. - IBM’s AI‑in‑Action series highlights how integrating hybrid‑cloud strategies with AI can unlock higher innovation ROI and better customer‑centric outcomes. - Experts Hillary Hunter (CTO, IBM NGM Innovation) and Ashman Hus (IBM Innovation Studio) emphasize that simultaneous focus on AI capabilities and intentional hybrid‑cloud design leads to more successful, enterprise‑grade AI implementations. ## Sections - [00:00:00](https://www.youtube.com/watch?v=FVXzkUCSu_U&t=0s) **Hybrid Cloud Fuels Generative AI** - The discussion explains how deployment choices—particularly hybrid‑cloud architectures— affect the effective, affordable use of generative AI, with IBM experts highlighting infrastructure considerations, cost, and innovation benefits. ## Full Transcript
in a world where generative AI can
enhance any business function folks want
it everywhere and why shouldn't they
have it well sometimes you can't just
click by using generative AI effectively
also requires managing your existing
data infrastructure in a way that both
makes sense and doesn't break the bank
today we're going to talk about how your
deployment options can enhance or hinder
your ability to use generative AI we're
talking about hybrid clouds y'all on AI
and action in this series we're going to
explore what generative AI can and can't
do how it actually gets built
responsible way to put it into practice
and the real world business problems and
solutions will encounter along the way
so welcome to AI in action brought to
you by IBM I'm Albert Lawrence and today
we're going to get into how hybrid cloud
and AI need each other it's more than
just cost in Roi if your infrastructure
is built right it can allow you to take
your Innovation to the Next Level so
today I'm joined by guests Hillary
Hunter and Ashman hus hey Hillary is CTO
infrastructure NGM Innovation at IBM and
an IBM fellow she's an expert in both
cloud and AI Computing what's up Hillary
hey there thanks for having me glad that
you could be here Ash is a leader in
IBM's Innovation studio and an expert in
machine learning and generative AI he
has extensive experience in building
complex multi Cloud systems welcome Ash
thank you for having me Albert so I know
you both are wondering why did I Choose
You two for today's episode well like I
said today I want to explore what sort
of tech Foundation you need to support
Ai and how to build it because it seems
like people are so focused on generative
AI they stopped talking about hybrid
clouds and my suspicion is that that's
actually the key to success I'm seeing
some nodding heads over here so I think
I'm on the right lane Hillary maybe you
can help me out with this first question
why aren't people talking about how
important an intentionally designed
hybrid cloud is with respect to
implementing generative AI you know
Albert I think a lot of times we get
really swept up in the latest technology
terms and in our desire to try to learn
everything about it and adopt it as well
as possible all of these Hot Topics
really have something to do with AI and
I think it's just sort of our enthusiasm
and excitement around the latest
technology that sometimes we as we
stopped talking about some of those
prior ones but as I'm sure we'll unpack
more in this discussion here hybrid
Cloud absolutely is key to a successful
set of AI deployments where you meet the
latency the cost the consumer experience
that you want out of your AI Solutions
and having both conversations at the
same time will result in much more
successful business outcomes that have a
lot more value to the customer into the
Enterprise okay well ask do you agree
yeah I I do I do agree and I I think
that um just as human beings we we've
been swept up by this magical new
technology called generative Ai and it's
one thing to consume generative Ai and
do what we was known as inferencing
which is you're prompting a model and
getting a response back it's a
completely different thing when you've
got to take data you've got to cleanse
it you've got to format it you've got to
get it into a form that you can then use
and consume with AI and that's all the
non sexy parts and I think people don't
like to think about the non-sexy hard
parts so much and you know that that's
kind of where a lot of the input goes
into creating these magnificent models
so why don't we start off with just
getting a good understanding of exactly
what is a hybrid Cloud hybrid Cloud we
think of as the capabilities that span
from Enterprise Computing through
private Cloud deployments meaning use of
it in the very agile way and use of
kubernetes and other modern Technologies
on premisis and into public cloud and I
would say also out to the edge right so
all the places that I takeen we operated
with high degrees of efficiency with as
of service capabilities with consistency
of operations consistency of visibility
control that's kind of the span of
hybrid cloud and when you think of there
where does it run into AI based on what
Ash said I'll I'll throw another hot
technology topic that we used to all
talk about all the time Big Data in that
big data era we were talking about where
is the data how do we analy ize it how
do we process it how do we get insights
out of it but when we used to talk about
big data we were very conscious of where
that data was and therefore if the data
is spread across that hybrid Cloud
landscape all the way from traditional
Enterprise it into private public clouds
and out to the edge then you're going to
want to have the AI conversation in
those same terms because AI really is
about getting insights from those data
bringing uh new capabilities to your
clients and it's both where the data is
across that ful landcape in hybrid cloud
as well as where your clients and your
customers are and those things then feed
into where do you want to create Ai and
then where do you want to deploy AI so
hybrid cloud and generative AI what
makes these two such a dynamic duo I
think a a Mis Noma that comes across as
people seem to think that AI is just a
single API and there's this magical huge
model and you're just going to be going
to this magical huge model now in
reality for an Enterprise that's not the
case for very ious reasons whether it's
compliance regulatory reasons latency
where your organization is physically
located you're going to end up in a
situation where you need to train
multiple models and there'll be
different models in different places
doing different things and what that
translates to is you're going to have
models running on premise you're going
to have models running in a particular
geography you're going to have models
running in the cloud with the cloud
provider for example it may be something
that's related to e-commerce or
something that's public facing to just
general consumers and so having Ai and
hybrid Cloud as a dynamic to you it was
the only real way that this would work
why is being able to run AI where your
data and where your customers are so
important there's so many reasons since
number one is latency I mean for example
if the latency for you to go and do data
ingestion to something is considerably
long you'll have during the time of
doing training and and and so forth and
even before that labeling and annotating
data lag and that lag can translate into
lots of manh hours and then that doesn't
become very cost effective I work with a
lot of clients in financial services for
example and if you think of banking
insurance we as consumers are always
interested in topics like fraud right we
want to ensure safety and all those kind
of activities um that we're doing in
that sector and therefore the companies
in that sector are constantly concerned
about fraud they have very sophisticated
algorithms and things like that and one
of the best ways to explain latency in
this whole kind of hybrid cloud and AI
context is to talk about you know you
really don't want to lag in detection of
credit card fraud for example you want
that fraud detection to be instantaneous
because we want to know that our bank
has the best possible fraud algorithms
running that are going to be able to
detect um fraud as it's potentially
happening if our you know card is lost
or compromised and the organizations
that are you know moving aggressively
with the AI are really looking at
latency because they want to process and
look at every transaction that's flowing
through a system and that requires
really powerful computers mainframes in
many cases but being able to do AI right
there on the Mainframe can enable an
organization to in an unsampled way
meaning each and every transaction
that's flowing through the system do a
more sophisticated fraud detection and
as consumers you know we have better
protection they have a better product
Etc and so let's let's take a moment now
to zoom on in a bit on the folks that
are building these systems right so now
we've established and I'm getting
understanding more now why these systems
really do matter and how every second
does count but how does a hybrid Cloud
environment support a better experience
for the programmers that are using AI a
typical life cycle that a a programmer
or a you know an engineer who who build
software will they will go through where
they'll have sort of a development
environment normally when that
development is taking place a lot of
that's generally taking place close to
where the developer is and it's in their
own local integrated development
environment whether that's running on a
computer that they have physically with
them or in a data center very close to
them so that you know they get a very
fast feedback and fast experience so
being able to work with data and use
that data to train a model being able to
do that in a fluid and flexible way
makes a night and day difference to to
actually developing software and and and
that doesn't change because now that
developers an AI engineer or or or a
data science plus programmer and in in
this new world those constraints still
apply take a very very simple example A
lot of these kind of things are done
using notebooks okay notebooks running
in in Python if you uh have got your
data very very disconnected and very far
away from where your notebooks are
actually running it's having a real
frustrating impact on being able to to
code and review and get that feedback
cycle going that someone who's on the
ground who needs to build these systems
needs to contend with on a day-to-day
basis okay well that person on the
ground who's need to contend with things
on a day-to-day basis I'm trying to jump
into the mindset there and trying to
understand the benefits of flexibility
but then also the necessity of security
at the same time so I'm curious how does
someone balance those two things when
you're thinking about designing an
architecture for AI one of the most
satisfying things for a developer Albert
is to see their capabilities come to
Market come to fruition reach customers
actually have a difference right I mean
that's we all come to work every day
wanting to to make a difference through
the stuff that we're creating and at
that backend side of having created
something through the process Ash was
describing oftentimes there are a lot of
checks and balances related to your
point Albert related to security related
to compliance and the general topic is
that of AI governance AI model
governance are we confident in this
technology do we know how and where it's
being deployed is it exhibiting any
drift monitoring it all of those kinds
of things and I think the hybrid Cloud
conversation has a lot to do with this
because we like to think about I like to
think about AI is not only the model but
really A a platform conversation that
enables that endtoend developer life
cycle from what we talked about at the
beginning and pulling together data and
curating data to actually testing and
building and modifying a model and
testing its use but then also governing
it when it's put out there into the wild
so to say when it's put into its context
wherever it is across that hybrid Cloud
landscape picking a vendor with whom you
can have an AI governance framework that
makes the deployment of AI be a yes from
all those that are in Risk in security
and compliance because they know the
safety of which that you know
application was constructed they know
how the AI is going to be monitored
moving forward and they know they can do
that monitoring no matter where that AI
is being deployed across the hybrid
Cloud landscape I think that's a really
critical aspect of the overall AI
considerations of an Enterprise as well
because you want to ensure that the
developer has a consistent set of
capabilities all the way from the data
prepping cleansing to the model building
testing the application evaluation and
then the end governance and that really
makes AI a yes not AI a no in many cases
where developers get really excited
they've created something and if there's
you know concerns about Providence if
there's concerns about online monitoring
and their work might not come to
fruition as quickly as if AI is viewed
by the Enterprise as an overall platform
discussion that really follows that
entire AI life cycle and I I want to add
to Hillary's point I I want to talk
about two paradigms here we'll call one
classical software development and the
the second Paradigm AI development in
classical software development you go
through a software development life
cycle where you write a set of
instructions for a computer to follow
and you have got clear business
requirements which translate into sort
of inputs for that application or
algorithm that's being written and you
expect a certain outcome and you write
your application code in such a way that
you handle things that are outside
bounds and so you have a input a set of
instructions and an output and the
output is going to be what you expected
to get or you're going to get some sort
of error message when you're building
something in the other Paradigm which is
using AI you're taking an input and
you're using a function to come out with
a probabilistic output and what that
does is it creates a dynamic situation
where if you're the person who's
creating uh the models and you're doing
the training those inputs that you give
to that model may not necessarily always
be the same as Hillary was saying having
that govern and having that
observability in place is so important
because in a traditional software
development life cycle you can kind of
go yeah we finished the project yeah we
know what this does with AI you need to
observe it and check that it's not
drifting that it's not changing over
time because of the inputs varying
because the business landscape has
changed and so it's kind of like you you
don't necessarily just need to think
about building a model and deploying it
you need to build a model and keep your
eye on it and what and iterate on it all
the time and again the hybrid Cloud
comes into play here because you want to
be able to do all those things really
quickly and very very easily because as
a team you're going to need to work more
cohesively than you ever have so Ash you
and Hillary you both are placing a lot
of weight on the shoulders of data
infrastructure with these examples that
you're giving I just got to know what
happens if your input goes wrong there's
this phrase that one of our IBM leaders
uses which is that there's no AI without
IIA there's no artificial intelligence
without information architecture and the
point there is really that this
conversation about being as effective as
possible with AI is really one also
about making sure that you have a grip
on your data where is it is it in a
platform or an environment where to
Ash's Point your developers can really
get in there and analyze and play with
data and find the most effective Ai and
the most effective solution solution and
then are you using AI again kind of in
this platform approach in a way that you
are prepared to do the governance
necessary to make sure that you know
what's continuing to happen with your
data as you deploy Ai and that you're
compliant with local rules and
regulations make sure to protect
personal privacy and personal
information and things like that and
these are all really good aims but they
become to some extent guardrails that
you have to keep uh in line as you
deploy your AI and so I think there is a
lot of work on the data landscape
fortunately I think because of that
earlier Big Data era some organizations
have gotten a lot of their data
collected well and cataloged and indexed
and have appropriate metadata others are
finding that this is now the impetus and
I think that's actually maybe not a bad
thing you're realizing that your data
architecture doesn't hold up to
everything you wish it was because this
is the greatest opportunity to fix it
and a lot of organizations historically
didn't really get in and rework their
information architecture because they
couldn't find a high enough business
value to doing so now is the time
because there's such high business value
to be had from the current capabilities
of generative Ai and the Next Generation
AI technologies that are out there that
it is a a rallying point and I've seen
some really amazing structures put
together organizationally as well where
Chief data officers Chief AI leaders uh
Chief technology officers cios security
officers risk everyone is coming
together to the table to say now is the
time that we tackle this information
architecture in AI we're going to do it
together this is going to be a shared
Mission a common goal because we can
Define that there's a 2X 3x 4X 10x value
to our business if we do this right one
big word that you just said right there
that's sticking out for me right now
Hillary is opportunity okay but when you
talk about the massive opportunity that
there is here there's another o word
that comes to mind for me which is
overwhelmed
because it seems like it's so massive of
an opportunity that people can feel kind
of overwhelmed so let's help to break
that down some how do you design an
infrastructure for AI in a way that
doesn't break the bank the answer is
hybrid
Cloud right you want to be able to start
small kick around some ideas look for
some use cases figure out what those
benchmarks are to check that the models
are behaving the way they should behave
as I said in this Paradigm of software
development you've got to be able to
measure the benchmarks of what the model
that you're going to create is going to
do and what various inputs you're going
to throw at it and the outcome and so
you want to do that in a really fast
Nimble agile way and the way to do that
is to pick some use cases start small
but think about using a hybrid Cloud
architecture so that when you do start
to get some traction and you start to
make some success things like compliance
governance Observer ability scalability
you're already answered because of the
fact that you're taking advantage of a
of a hybrid Cloud architecture can you
give me like a concrete example or two
of of starting small and then broadening
out yeah yeah absolutely so to do AI
really well you need people from the
business to be involved you need data
scientists to be involved you need
devops Engineers to be involved you need
to bring all of these people together
and so bringing all these people
together and then having some small use
case that you can kind of go you know
what we think that this is going to give
value to the business being able to have
everybody work together and build like a
small model put it into an environment
where you can test it and then that team
can collectively share that information
with each other is this working are we
getting what we thought we were going to
get out of it okay that feedback cycle
between the people involved in creating
these things okay is what's going to
lead to creating sort of like a flywheel
effect and the momentum to be able to
scale so now I'm thinking
about what does the best case scenario
look like when it comes to you know
current generative AI solutions that are
effectively using hybrid Cloud I think
that uh a really good example and one
that is very easy in the world of
generative AI to to translate into sort
of business outcome and for people to
understand especially sort of your CFO
is customer care right dealing with lots
of inbound customer care inquiries okay
being able to have a generative AI model
to be able to interact with customers
and to sort of decipher what it is that
they actually want what they want to get
out of it but customizing that tone and
that that way of like communicating
using a large language model to each
customer individually I think is a
really really powerful and and one where
we're seeing a lot of traction awesome
so customer care right there all right
Hillary what about you a second thought
about customer care we're certainly
seeing a lot of productivity around that
topic I'll share just one of the most
fun parts of my job is I get to work on
a bunch of internal AI products we refer
to that as being customer zero of IBM's
AI so what we're using it uh when it's
fresh off the presses and often you know
straight out of research into our hands
and we have a lot of exciting momentum
there in really doing what what I refer
to is giving people superpowers in their
day-to-day and that means you know
removing TD parts of tasks you know
comparisons of contracts or complex you
know literature about electrical
specifications or all these other kind
of things and throughout many of the
Enterprise clients that we work with you
know a lot of people don't want to be
spending time on these parts of their
job and it gives them superpowers
because they are able to complete these
tasks much more quickly and they're able
to spend more time on the more complex
insights in their role and more time
contributing back into contractual
processes and Technical in engineering
processes rather than waiting through
documents and I think that there's an
enormous amount of productivity that
we're seeing on the teams that I work
with and very similar things coming from
the customers that we're working on very
similar implementations with wonderful
well it seems like both of you are very
committed to making this idea of the
hybrid Cloud a lot less cloudy for
anyone who's very curious about it so
thank you so much for that I'm going to
give you a few of my own personal
takeaways from this and at the end I'm
going to ask you to let me know did I
get this right okay so first off to do
generative AI correctly you need
multiple models running multiple places
so it's not possible without hybrid
clout for intelligent real-time
decision-making every millisecond counts
so latency is a huge factor and start
with the small use case and curate a
multi-disciplinary team in order to
support your build that sound right you
nailed it outut awesome oh my gosh
awesome well Hillary and Ash it means a
great deal to me that you came today for
this conversation this was fantastic
friends thank you so much for tuning on
into this episode we really appreciate
your time and I want you to stay tuned
to this feed for even more great AI
insights see you soon
[Music]