From Cloud AI to Distributed AI
Key Points
- Niru Desai explains that **distributed AI** enables scaling of data and AI workloads across hybrid environments—public cloud, on‑premises, and edge—while providing unified lifecycle management.
- He traces the evolution from **cloud‑centric AI** (centralized training and inference with data streamed from plants to a core cloud) to **edge‑focused AI**, where more processing happens locally to reduce latency, bandwidth use, and sensitivity concerns.
- The move toward **distributed AI** addresses key business challenges such as intermittent connectivity, large‑volume data transfers, and the need for real‑time decision making at remote sites.
- IBM’s new distributed‑AI capabilities are available for free via the **IBM API Hub**, allowing developers to experiment with the tools and platforms that support this paradigm.
Full Transcript
# From Cloud AI to Distributed AI **Source:** [https://www.youtube.com/watch?v=jevuDDjFEsM](https://www.youtube.com/watch?v=jevuDDjFEsM) **Duration:** 00:15:57 ## Summary - Niru Desai explains that **distributed AI** enables scaling of data and AI workloads across hybrid environments—public cloud, on‑premises, and edge—while providing unified lifecycle management. - He traces the evolution from **cloud‑centric AI** (centralized training and inference with data streamed from plants to a core cloud) to **edge‑focused AI**, where more processing happens locally to reduce latency, bandwidth use, and sensitivity concerns. - The move toward **distributed AI** addresses key business challenges such as intermittent connectivity, large‑volume data transfers, and the need for real‑time decision making at remote sites. - IBM’s new distributed‑AI capabilities are available for free via the **IBM API Hub**, allowing developers to experiment with the tools and platforms that support this paradigm. ## Sections - [00:00:00](https://www.youtube.com/watch?v=jevuDDjFEsM&t=0s) **Introducing Distributed AI Paradigm** - Niru Desai outlines IBM’s concept of Distributed AI, tracing its evolution from cloud‑based AI to AGI, the business challenges it solves, and how developers can experiment with related APIs on IBM’s API Hub. ## Full Transcript
hi i'm niru desai from ibm i'm here to
talk to you about distributed ai
distributed ai is a paradigm of
computing that allows you to scale your
data and ai applications across
distributed cloud environments
distributed cloud environments as you
may be familiar
allow you to have a single pane of glass
application life cycle management across
public cloud on premise and edge
environments
now
as we
look at the emergence of distributed ai
i want to take you through the journey
of how we arrived there we started with
the cloud-based ai we go to agi and then
we talk about distributed ai
also i'm going to introduce to you the
challenges that distributed ai helps you
address in your business
finally
all the capabilities that we are
creating
for enabling distributed ai
are available for you to try freely at
ibm api hub see the link in the
description
without further ado let me take you
through the journey of where we've been
first we're gonna talk about
cloud-based
ai
what happens here
is you have a
let's take a concrete example
so that concrete example is going to
involve a plant it actually could be any
location where you have your business
operations
and you're making some local decisions
on the other side of the picture
you have some kind of core location
could be your enterprise data center
could be public cloud
i'm just going to
take an example here so let's say it's a
public cloud and what you have is in
this public cloud you have
some kind of kubernetes service with
your data and ai
middleware
and then on top you're deploying one or
more applications
these applications when they are data ai
based you may be actually doing some
kind of training for your ai pipelines
and you may be doing inferencing as well
all right
what happens on the business process
side on your plant is that as the
process takes place it generates a
tremendous amount of data and all that
data is getting pushed to the core
location where the decisions are being
made through the ai pipeline inference
those decisions are then communicated
back to your plant where it drives your
downstream automation
so
clearly because you're sending all the
data over to core and it could be a
large amount of data it could be
sensitive data
uh it could run into the challenges of
connectivity intermittent connectivity
issues with
your core location
this has run into challenges this is why
we are seeing emergence of
what i'm going to call
edge ai
so what happens in aji
so in the case of agi you still have
your plant i'm going to draw a slightly
bigger box here because more is going to
happen at the plant
and you still have your
core location
unlike before
where most of the decision making
actually was happening in the core
you're going to have the decision making
uh happening here
you're going to take advantage of
distributed cloud environments
distributed cloud platform capabilities
to make the application lifecycle from
core to all your plans so what happens
then
is you actually have
a container platform
with data and ai middleware deployed
on it
and the application deployed right in
your plant your core
still does what it did before
except it is now taking care of
deploying
the application
and taking care of its life cycle
so let's complete the picture here
you have data in ai
and then you also have the application
deployed on it
unlike before you're going to train your
applications here
deploy them
through the distributed cloud platform
in single pane of glass that's important
uh and these applications are going to
make inferences here so if your business
process
is taking place
or your business operations are taking
place here
what happens is that your application is
generating
decisions
and they are driving your business
process and this process is then feeding
data back
to your computing stack that is implant
so what we have done here is we localize
decision making
we no longer have to continuously send
data up to a core location and wait for
it to make a decision that then can
automate our business process of course
we still need to send some data over
and we have to use that data to train
this or retrain these ai pipelines and
redeploy them
so
we made some progress when we switched
from cloud base ai to distributed ai
or actually aji pardon me
but
when we try to deploy
this pattern
across a large number of locations and
across a large number of a large variety
of applications we run into certain
challenges and so we have then a need
to
address those challenges
with the capabilities we are describing
as distributed ai the pattern of
distributed ai is very similar to aji
but i'm going to replicate this and i'm
going to move away
from the terminology of edge and cloud
and core to actually talk about what
matters the most what matters the most
is where is the data and where does it
need to be analyzed so it is possible
that you have a vast amount of data
sitting in a public cloud but you want
to consume the ai capabilities from
another cloud in this case the first
cloud is what we call
as
a spoke
and
this is where your data is
on the other
hand the cloud where you have the ai
capabilities and the application and
analytics is what we call the hub
and this is where your control plane is
see how it allows us to talk about hub
and spokes
where hub and spoke do not really have a
connotation of cloud or edge or whether
this is a mobile
vehicle
or this is a stationary data center it
doesn't really matter what matters is
your data here your control plane here
you want to manage the deployment of
applications from hub to spokes you also
want to take control of the data and ai
lifecycle from the hub so i'm going to
for the sake of completeness
complete this picture
which looks not very different from what
we have seen before
so that i can take you through
the challenges you're going to run into
when we try to scale such a stack
to a large number of
spokes and large number of applications
okay so let's say
we have this application and you have
you know you have your
business process here
the decisions
are going down
the data is coming back
uh you are of course
deploying
through the hybrid distributed cloud
environments and you're
pushing some data over okay just to
complete that picture
so
what happens
when you have a potentially large number
of
these perks
on which
you're trying to
enable this ai application
the first thing that comes to mind
is that because you're still
collecting the data for training
and you're pushing
a large amount of training sets you know
these models consume a large amount
large amounts of data
and you have large number of
applications and large number locations
doing that you're going to run into a
challenge we call as
data gravity
it's just
causing two main problems for you
you're putting tremendous pressure on
the resources in the hub to manage all
that data
and you're actually incurring
costs in then having having to analyze
the data having to train that data
not to also mention some of the network
bandwidth limitations that may come in
your way especially as you try to do
this for a large number of applications
so data gravity is a key challenge
the next challenge i want to introduce
to you
is the fact that each of these spokes
may be slightly different you're
probably manufacturing a slightly
different product mix at each of your
plants or each of your retail stores are
serving a slightly different
demographics because of that
one model that you've trained or one
pipeline that you've trained in your hub
is not going to be fit for all your
sports so there is always going to be
a challenge in dealing with that
heterogeneity and not having to do
manual work so we'll get to
how we address that in a second
the third challenge is just the sheer
scale we've talked about scale but it
actually has two aspects
one is just the number of spokes you
have to deal with and the computational
complexity of
doing that
training so many models deploying so
many applications in so many locations
the second part is the variety
in
applications and data
remember
in all these cases we have look where
we've looked at data the data could come
in many different types so you have
data types of
let's say images
it could be sounds
it could be sensor information
it could also be lidar
network
information
and time series information there is
just a
very wide variety of data modalities
that
different applications that you are
trying to deploy and manage would need
to consume so that variety in
applications and data
make it even harder for you to scale and
accelerate deployments
the last challenge i want to introduce
to you is the challenge of resource
constraints
so
although
the spokes and the hubs may have some
resource it is also quite common for
some of the spokes such as plants and
retail stores to have a finite amount
and a small amount of resource so you
have a resource budget that must be
respected as you deploy your data and ai
pipelines to them and that causes new
challenges right so resource constraint
is a key challenge as well now
we are very excited that in ibm we have
addressed these challenges head on
and enable this distributed ai
that scales across distributed cloud
environments across locations and
applications so how do we address
data gravity well the key thing to do in
addressing data gravity is to not
collect all the data but only the
important data so intelligent
data collection is a key capability
that
we are going to bring to you
and you can actually try it out through
api hub as i mentioned earlier
a lot of the data at the spokes is
repetitive some of that is noisy so you
don't want to necessarily collect all of
it you only want to collect what's
important and identifying what's
important especially when you have a
large number of locations and vast
variety in the data modalities and
applications is a challenging problem to
solve
the second part about heterogeneity
uh basically means that
when you are clear deploying your ai
pipelines or applications across
different spokes you want to target them
you want to adapt to those spokes so
adapting
and also then after deployment you want
to monitor
so to make sure that they are performing
well
uh adaptation and monitoring at each of
the spoke locations
is critical in addressing the
heterogeneity challenge
and then in terms of the scale it simply
means you need greater amount of
automation
in
controlling
your data in ai life cycles so
automation of
data lifecycle basically is
about
policy based decision making to see what
data should stay where
when should it be purged when should it
be replicated where what
policies in terms of data localization
apply so that you can respect those
constraints as you take care of data
lifecycle automating that
then
lets you address the large number of
locations that we've been talking about
similarly ai life cycle
can also be automated
so starting from
training the models deploying them
monitoring them if the data or the
environment drifts then retraining them
collecting the rash the
right kind of samples through
intelligent data collection and then
using them for retaining the model
automating all that life cycle is
critical as well because
you may end up with hundreds if not
thousands of different ai models and
pipelines that are automating various
aspects of your business as you start as
you start scaling this
lastly on resource constraints what is
essential is that we have some
ability to optimize
the data in ai pipelines
what this does is it does things like
feature extraction a model compression
pruning and some of those techniques it
brings them to bed to make sure your
resource budget is respected at all
times during your
pipeline execution so in summary
we have introduced to you a new paradigm
called distributed ai and we've
introduced to you some capabilities that
actually bring it alive distributed ai
will allow you to scale applications to
a large number of
locations large number of spokes and it
allows you to scale across wide variety
of applications thank you
thank you for watching this video if you
are interested in more content like this
please hit like and subscribe to this
channel also please check out the links
in the description which will get you
started on distributed ai apis on ibm
api hub
you