Open AI Transforming Enterprise Operations
Key Points
- The episode explores “openness” in AI, examining how transparent, open‑source approaches are reshaping business models and expanding what enterprises can achieve with artificial intelligence.
- Maram Ashuri, IBM Watson x’s Director of Product Management, explains how IBM’s foundational models—particularly the Granite family—enable faster, more accurate customer‑care responses by leveraging internal company data while maintaining higher levels of model transparency.
- She outlines the shift from mere generative‑AI experimentation to production‑grade deployments, noting challenges such as latency, cost, and energy use, and how smaller, proprietary‑data‑tuned models can deliver comparable performance at a fraction of the expense.
- Ashuri highlights generative AI as a paradigm shift on par with the rise of the internet and personal computers, arguing that it will dramatically boost workforce productivity by automating routine tasks and freeing employees for higher‑value work.
- With over 15 years of data‑driven technology experience, Ashuri’s role at IBM places her at the forefront of bringing enterprise‑grade AI capabilities to market, emphasizing the strategic importance of open, transparent models for future business innovation.
Sections
Full Transcript
# Open AI Transforming Enterprise Operations **Source:** [https://www.youtube.com/watch?v=YIIbugJbgpE](https://www.youtube.com/watch?v=YIIbugJbgpE) **Duration:** 00:31:09 ## Summary - The episode explores “openness” in AI, examining how transparent, open‑source approaches are reshaping business models and expanding what enterprises can achieve with artificial intelligence. - Maram Ashuri, IBM Watson x’s Director of Product Management, explains how IBM’s foundational models—particularly the Granite family—enable faster, more accurate customer‑care responses by leveraging internal company data while maintaining higher levels of model transparency. - She outlines the shift from mere generative‑AI experimentation to production‑grade deployments, noting challenges such as latency, cost, and energy use, and how smaller, proprietary‑data‑tuned models can deliver comparable performance at a fraction of the expense. - Ashuri highlights generative AI as a paradigm shift on par with the rise of the internet and personal computers, arguing that it will dramatically boost workforce productivity by automating routine tasks and freeing employees for higher‑value work. - With over 15 years of data‑driven technology experience, Ashuri’s role at IBM places her at the forefront of bringing enterprise‑grade AI capabilities to market, emphasizing the strategic importance of open, transparent models for future business innovation. ## Sections - [00:00:00](https://www.youtube.com/watch?v=YIIbugJbgpE&t=0s) **Untitled Section** - ## Full Transcript
bushkin hello hello welcome to Smart
talks with IBM a podcast from Pushkin
Industries iHeart radio and IBM I'm
Malcolm glao this season we're diving
back into the world of artificial
intelligence but with a focus on the
powerful concept of open it's Poss AB
ilities implications and misconceptions
we'll look at openness from a variety of
angles and explore how the concept is
already reshaping Industries ways of
doing business and our very notion of
what's possible on today's episode Jacob
Goldstein sat down with maram ashuri the
director of product management and head
of product for IBM's Watson
x. where she spearheads the product
strategy and delivery of IBM's Watson
Foundation models she is a technologist
with more than 15 years of experience
developing datadriven Technologies the
conversation focused on how Enterprises
can use technology to build and deliver
greater transparency in AI with granite
maram explained how granite can be
utilized to improve efficiency across
various domains she discussed how these
models are being used in real world
business applications
particularly in areas like customer care
where AI can help enable quick accurate
responses based on internal company data
Mariam provided a fascinating look into
how Enterprises have moved from Mere
experimentation with generative AI to
actual production navigating challenges
such as increased latency cost and
energy consumption she highlighted how
the emerging trend of smaller models
customized with proprietary data can
potentially deliver high performance at
a fraction of the cost marking a
significant shift in how Enterprises
leverage AI whether you're an AI
Enthusiast or a business leader looking
to harness the power of artificial
intelligence this episode is packed with
valuable insights and forward-thinking
[Music]
strategies let's just start with your
background how did you come to work at
IBM I joined IBM right after I graduated
I have an AI background and throughout
the years I've
held many roles in design engineering
development research mostly focus on AI
application development and design in my
current job I'm the product owner for
what's the next that AI which is the IBM
platform for Enterprise AI what excites
me about this job I would say is the
technology advancements over the last 18
months in the market we've been
witnessing how generative AI has been
changing the market the way that I see
that is Gen has been perhaps one of the
largest Paradigm shifts when we think
about productivity the same way that
internet and personal computers impacted
the productivity of Workforce now we are
witnessing another wave of all those
opportunities that it can unlock for
especially Enterprise AI when it comes
to enhancing the productivity of the
workforce and releasing some time that
can potentially be put into creating
more value work for Enterprise so that's
the major part that I picked this team
to have an impact on the market and the
community but also of course uh using
the skills that I gain through all these
years through IBM to help to establish
IBM as the market lead there for
Enterprise AI so you talked about geni
as this sort of generational
transformational technological force and
I'm curious just in terms of how it's
going to come into the world like how do
you see market adoption of geni sort of
evolving from here well last year was
the year of excitement about generative
AI most of the companies were
experimenting and exploring with Gen we
see that energy shifted towards how to
best monetized that technology almost
half of the market has moved from
investigation to Pilots 10% has moved to
production when you are exploring with
this technology you're looking for a
valve Factor you're looking for an aha
moment that's why very large general
purpose models shine but as companies
move toward production and scale they
soon realize the path to Success is Not
That
straightforward for example the larger
the model the larger compute res as it
requires that translates to increased
latency that's your response time that
translates to increased cost that
translates to increase carbon food print
and energy consumption so think about
that at the scale of Enterprise in
production some of them can be a
showstopper because of this reason what
actually see is emerging in the market
is instead of focusing on very large
general purpose models coming back to
very small trustworthy models that they
can customize on their own proprietary
data that's the data about their
customers that the data about their
specific domains to create something
differentiated that is much smaller and
delivers the performance that they want
on a Target use case for a fraction of
the cost uhuh so let's talk a little bit
more specifically about what you're
working on let's talk about Granite
first first of all tell me what is
granite Granite is our industry leading
family of models Flagship IBM models
okay these are the models that we train
from scratch when offered through our
platform we offer indemnification and we
stand behind them today it comes in four
flavors language code time series and
GEOS special models Granite language
series is covering English Spanish
German Portuguese and Japanese we have a
combination of commercial and
open-source language models on granite
for example we recently released the
granite 7B language model small powerful
English model okay on the code front our
models are state-ofthe-art models
ranging from 3 billion to 34 billion
parameters these are very powerful
models that performs or outperforms in
some cases the popular open- Source
models in their weight class so very
powerful models so I get the idea big
picture about these models but it would
be helpful to just get a sense
specifically of what they're doing like
can you give me any specific examples of
how these models are being used in
businesses in the real world right now
well the top use cases for generative AI
are really content generation
summarization information extraction
perhaps the most popular use case that
we are seeing in Enterprise is content
grounded question and answering so using
these models as a base to connect them
to a body of information let's say their
policies their documents that is
internal to the Enterprise and get the
model to provide answers based on that
questions one example of that is for
customer agents customer care when a
customer is asking a question previously
the agent that responds to the customer
had to answer the question and if they
don't know the answer escalated to the
product specialist keeping people on
hold on the line to go figure out the
answer for that and then come back you
can think of the time it takes to
resolve an issue but now with llms we
have an opportunity to automatically
retrieve the information based on the
internal documents of the company
formulate an answer show it to the human
agent and then if they verify with the
sources of where is coming from they can
just translate it directly to the
customer right this is a very simple
example of how it's impacting the
customer care so one big theme of this
season is this idea of open and one of
the things that's interesting to me
about the work you're doing is you are
using not only Granite this uh model IBM
developed but you're also using uh
third-party models right from other
places so tell me about that work and
how that is sort of fitting into your
kind of real world typically Enterprise
gen work when it comes to model strategy
our strategy is really focused on two
pillars multimodel and multi- deployment
it means that we don't believe one
single model rules all the use cases and
I think at this point the market has
also realized the Enterprise markets in
average today are using 5 to 10
different models for different use cases
oh interesting so in our portfolio if
you look into Boston x. a today we are
offering a large sets of high performing
state-of-the-art models coming from
open- Source commercial models that we
are bringing through our partners and
also IBM develop models in addition to
all of these we also have an option for
bring your own model from outside the
platform let's say you have a custom
model that you made it yourself you can
bring it to the platform and really
helping the customers to navigate
through a wide range of models and the
right model for their target use case
throughout that we've been heavily
working with our partners and you know
this is the market that is evolving
rapidly we've been at the Forefront of
speedit to delivery one example that I
like to highlight is recently meta
released a llama 405 billion such a
powerful model on the same day that it
was released to the market we made it
available in our platform to our
customers the same day and not only we
delivered it on the the same day we are
offering competitive pricing but also
flexibility in where to deploy so we are
giving an option to Enterprise to deploy
these models on the platform of dat
choice either multicloud it can be gcp
AWS Azure IBM cloud or on premises the
same for U misol ai misol ai recently
released the model misol large to on the
same day we deliver that through the
platform that's an example of a
commercial model llama was open source
but mrr large 2 is a commercial model
that we made available through the
platform great so I want to talk about
Enterprise grade Foundation models just
to get into it briefly what's a
foundation model people associate
Foundation models with a large language
model but large language models are
really a subset of foundation models
large language models are focused on
language but Foundation models can be
code generator
can be focused on time series model we
talked about they can be images it can
be GEOS special models so Foundation
model as the term suggests your
foundations to create a series of
subsequent models that can be customized
for a downstream use case and that's why
they are calling them Foundation models
llm is a good example of that as a
subset for language that you can further
customize on your specific data to get
them model to do other work so the core
of these Foundation models they are
basically trained on an absurd amount of
data LGE data sets that uh most of the
institutions today are sourcing them
from the internet so you can imagine
what can potentially go to those models
and then it comes to the Enterprise and
they start using it so for us also when
we started looking into in particular it
was triggered by customers asking us to
Prov client protections on these models
and we started thinking about let's look
into how the models are trained and if
we are comfortable offering client
protections on the models that are
available in the market and guess what
for a majority of these models there is
absolutely no visibility into what data
went into those models not much
transparency into how the model trained
and the responsibility lies on you as a
customers to start using those models so
just to be clear that is presenting like
potential risk real potential risk to a
company that is using these models it is
it is a potential risk in particular for
the customers in highly regulated
Industries so what we did for granite
was when we started training this models
from scratch basically we went to the
Corpus of data that was available to us
so for example the very first version of
granite was exposed to 20% of its data
from finance and legal because we have a
lot of um financial institutions as our
clients we work directly with our IBM
research to identify detectors for
harmful information like hate abuse and
profanity detectors uhuh okay so we're
talking about Granite we're talking
about this set of models IBM has
developed let's talk about using Granite
on Watson X compared to downloading
open- Source models like how do those
differ by using granite and bson X you
get two things the first one is the
client protection and identification
that we talked about you get that if the
model is consumed through our platform
and the second one is really the
ecosystem of platform capabilities that
we are offering to help you create value
on top of those data so for example
bringing your data to customize Granite
for your own specific use case but also
one thing that I like to highlight in
particular is the AI governance so when
you get one of these pre-trained models
you put it in front of your own users
through the input and instructions that
the user provides for that model they
can nudge the model to potentially
create undesired behavior and change the
behavior of the model and because of
this is extremely important to
automatically document the lineage of
who touched the model at what point so
if something happens you can trace it
back and see where it's coming from and
that's what what's the next that
governance is offering automatically
documenting the lineage when you use the
granite within the platform you get all
of those you can have the endtoend
governance you can have access to all
this scalable deployment opportunities
that is available for you like to allow
you deploy them on the platform of your
choice that we talked about either multi
uh cloud or um Prem and it also helps
you to have access to a wide range of
model customizations approaches promp
tuning fine-tuning retrieval augmented
Generations agents there is a series of
them available to use and apply to your
model this distinction between large
language models and Foundation models is
eye openening Miriam emphasized that
Foundation models can be tailored to
specific tasks but with that versatility
comes a significant challenge the lack
of transparency and how these models are
trained this composee a real risk
especially in high regulated Industries
like Finance essentially by using
granite and Watson X together
Enterprises get powerful and
customizable
tools so let's talk about the future a
little bit what do you think are some of
the big developments we're likely to see
in the realm of AI models very good
question I feel like the generative AI
of the past was powered by large
language models the generative AI of the
future is going to reason plan act and
reflect huh and so I mean in the context
of granite in particular like what are
we likely to see both you know in the
near term and in the sort of medium to
long term there are multiple elements to
implement an agentic workflow that I
just mentioned one element of that is
the llm itself to be able to do the
planning and reasoning and acting and
doing something that we call tool
calling so basically a series of tools
are available to the model you ask the
model to call those and make a call for
example we can say hey Granite what is
the weather like where uh Jacob lives
it's going to connect to web search API
look up your location then it's going to
connect to Weather API calculate the
weather and come back and formulate an
answer and respond to that so during
this process
it first has to plan the task of how to
answer that question look into what are
the tools that are available to it and
call them and that's an ability of the
model to do that what we did with
granite was we expanded the granite
capabilities to be able to do function
calling so for example today we have an
open- Source granny 20b function calling
that is available on hogging face to try
on and you can grab the model and the
model has capability to do the tool
calling I'm anticipating that in the
near future the planning and reasoning
and acting and reflecting capabilities
of the large language models are going
to continue to
evolve so thinking now from the point of
view of buyers and users of AIS really
people who are listening kind of from
that perspective as people are
evaluating AI tools and solutions what
is the most important thing they should
be thinking about how do you think about
kind of that
process I think they should always start
with the area at which they think it
would benefits from Ai and then within
that area looking to what data they have
available to potentially fit into those
AI service Architects do they have
access to Quality data and the second
question that they have to ask
themselves is do I have a trusted
partner that can supply what I need to
be able to implement AI that can can be
a collection of the foundation models
that you're going to need that can be a
collection of the platform capabilities
that the trusted partner can offer you
to implement such a thing the third
thing is go and evaluate the
regulations does regulation allow you to
apply AI to that specific area that you
are investigating and you're targeting
for AI and the last part but not least
is back to the principles of design
thinking what is the problem in that
area I'm solving with AI and if AI is
even appropriate because we want to make
sure that you use AI not just because
it's a cool hot toy in the market but
you are convinced that it can
significantly enhance the user
experience of your customers in that
area and once you have an answer to
those all these four questions then
maybe you have a good candidates to
start applying AI to and what about from
the side of project managers who are
trying to just keep up with how fast
things are changing how fast Innovation
is happening like what advice would you
give those people my advice would be
focus on agility this is a market that
is evolving rapidly and the winners of
the market would be those that are able
to take advantage of the best the market
can offer at any point of time so in
order to do that they need to be open to
experimentation continuous learning and
open to rapidly adopting the new
ideas and when you think about the
future and geni is there a particular
say problem that you are most excited to
solve I think that would be productivity
if you look into the stats that are out
there there are surveys that confirm
that 60 to 70% of the time of our
employees can be
potentially enhanced through the
productivity gains of generative AI for
example I personally myself use my
product for Content generation a lot so
the time that it frees up can be
potentially put into generating a higher
value work and because of that I'm super
excited with all the opportunities that
it represents for Enterprises to go and
dedicate the time of their employees to
higher value items great okay a couple
Granite specific questions so what are
like the key things you want the world
to know about Granite Granite is open
trusted and
targeted two ways to think about
openness One open as open weights it's
available for public to download and the
second one is open as in there is less
restrictions on how the customers can
legally use these models for a range of
use cases we have released Granite open
source models under Apache license that
is enabling a large range of use cases
the second one was trusted we talked
about that like it's a rooted in the
trustworthy governance process that we
established around how we are training
these models and the responsibility that
we take for these models and the third
one is targeted targeted for enterprise
we talked about like exposing Granite to
Enterprise data or the domain specific
Granite some of them like Cobalt to Java
translation that is targeting to solve a
specific Enterprise needs and that's
Granite so open trusted and targeted so
there are a lot of models out in the
world all of a sudden right it's a it's
a crowded Market where does granite fit
in that Universe what is the market for
granted we talked about the Enterprise
Market shifting away from very large en
purpose models to targeted smaller
models and granite is a small model that
Enterprise can pick up and customize on
their proprietary data to create
something that is differentiated for a
Target use case so Granite is well
suited as a small domain specific
business ready tailored for business and
trained on Enterprise data to solve
Enterprise questions you mentioned small
as one of the things that granted is why
is that useful in some contexts for
Enterprise for businesses the larger the
model the larger compute resources it
requires it translates to increased
latency that's your response time it
translates to increased cost add in
translates to increase carbon footprint
and energy consumption so at the scale
of enterprise transactions when you move
to production and you want to scale some
of these challenges can be multiple
times stronger like cost can add up the
energy consumption can be a serious
thing and the latency is depending on
the application can be a showstopper and
um blocker because for longer larger
models more powerful models it just
takes a way longer time to process and
calculate the output for you
we are going to finish up with a speed
round and I want you to just answer with
the first thing that comes to mind don't
overthink these okay complete this
sentence in five years AI will be
invisible ah I like that what do you
mean by that today AI is everywhere but
if you ask my kids at home they know AI
but if you say where is AI like how do
you use AI they don't know the answer
because it's so Blended in their life
that they don't feel like it's something
that they are using they are getting
used to that so when I think of Next
Generation and the years to come that
generation is so used to AI being part
of their life that they feel like it's
just there that's one and the second one
is the Simplicity of interaction with AI
that you don't feel like you're
interacting with the system it's just
there like you talk to AI everything is
automated so I would say the Simplicity
and being Blended to solve the right
problems is the part that I'm referring
to as invisible like internet is
everywhere and it's invisible but we
used to dial in like you remember the
dialing some to connect to Internet it's
gone internet is completely invisible
today right like we used to talk about
logging on right and you don't log on
anymore cuz you're always logged on yep
you're always connected
what's the number one thing that people
misunderstand about ai ai is inevitable
but should not be feared what advice
would you give yourself 10 years ago to
better prepare you for today I would say
develop a broad range of skills even if
you think they will not help you today
they may be valuable in the future so on
the consumer side right now we hear a
lot of about chatbots and image
generators but on the business side what
do you think is the next big business
application AI influencers generating
content huh how do you use AI in your
day-to-day life today one simple example
is LinkedIn posts I love it to just go
to my product I'll give you an example
which is my favorite one llama 3.1
405b the post that I announced on
LinkedIn on hey IBM is releasing the
model on the same day it was generated
by llama 3.1 405 billion so using the
same model to post generate the
announcement uh note very elegant is
there anything else I should ask you oh
we didn't talk about instruct lab so
when you grab a model you start from the
model but you need to then customize it
on your proprietary data to create value
on top of that so instru lab is giving
you a method based on open-source
contributions to collectively contribute
to improve the base model so if you're
an Enterprise you can leverage your
internal employees to collectively all
contribute to improve the models and
I'll give you an example of why it
matters like if you go to hogging face
today and look for llama there are about
50,000 different llamas coming up and
the reason is because there is no way to
contribute to the base model if you're a
developer you have to make a clone of
the copy of the model and F tune it for
your own purpose we figure the method
that is we call instruct lab to be able
to collectively collect all that
information and contribute to the base
model and enhance that so that's
instru I just wanted to highlight the
value of being open uhuh because that's
another topic that has been emerging in
the market over the past 18 months in
particular I believe the future of AI is
open and we've been seeing how the open
source markets has been changing how the
models are accessible to a wider
audience and good things typically
happen when you make technology pieces
accessible to a broader range of
community to stress test them and that's
the direction that we've been adopting
with granite and I feel like that's
really the adoption that the market is
going to immersed to moving forward yeah
there's this interesting I think maybe
naively unintuitive but it makes sense
once you think about it thing that open-
Source things are safer you might
naively think oh no put it in a box so
nobody can see it and that'll be safer
but like it turns out in the world if
you let everybody poke at it the world
will find the vulnerabilities for you
and you can fix them right that's
exactly what's going to happen yeah
great it was lovely to talk with you
thank you so much for your time this
same here thanks
Jacob and that wraps up this episode a
huge thanks to maram and Jacob today's
conversation opened my eyes as to how
open technology and AI are intersecting
to create more transparent and efficient
systems for Enterprises from the power
of smaller more targeted models like
Granite to the importance of trust and
governance in AI these developments are
reshaping how businesses operate at
their core as we continue to unpack the
complexities of artificial intelligence
it's clear that openness whether in data
technology or collaboration is not just
a concept but a driving force that can
unlock new
possibilities smart talks with IBM is
produced by Matt Romano Joey fishr Amy
Gaines McQuade and Jacob Goldstein were
edited by Lydia Jean cot our Engineers
are Sarah buger and Ben toall the song
by gramoscope special thanks to the
eight bar and IBM teams as well as the
Pushkin marketing team smart talks with
IBM is a production of Pushkin
Industries and Ruby Studio at iHeart
media to find more Pushkin podcasts
listen on the iHeart Radio app Apple
podcast or wherever you listen to
podcasts I'm Malcolm
Gladwell this is a paid advertisement
from IBM the conversations on this
podcast don't necessarily represent
IBM's positions strategies or opinions
[Music]