Ensembling Traditional AI with LLMs
Key Points
- The speaker introduces an “AI toolbox” concept, emphasizing the need to dynamically select and combine different AI models to maximize value as new techniques emerge.
- A new ensemble approach is proposed that leverages both traditional AI (machine‑learning/deep‑learning models) and large language models (LLMs) to capitalize on each type’s strengths.
- Traditional AI excels with structured data, offers fast, low‑latency, energy‑efficient predictions, and is commonly used for tasks such as fraud detection, AML, insurance claim analysis, and medical imaging.
- LLMs—particularly encoder‑based models—provide higher accuracy at the cost of greater computational power, higher latency, and lower energy efficiency, making them suitable for the same domains when precision outweighs speed.
Full Transcript
# Ensembling Traditional AI with LLMs **Source:** [https://www.youtube.com/watch?v=UvZAaeBhOBs](https://www.youtube.com/watch?v=UvZAaeBhOBs) **Duration:** 00:08:06 ## Summary - The speaker introduces an “AI toolbox” concept, emphasizing the need to dynamically select and combine different AI models to maximize value as new techniques emerge. - A new ensemble approach is proposed that leverages both traditional AI (machine‑learning/deep‑learning models) and large language models (LLMs) to capitalize on each type’s strengths. - Traditional AI excels with structured data, offers fast, low‑latency, energy‑efficient predictions, and is commonly used for tasks such as fraud detection, AML, insurance claim analysis, and medical imaging. - LLMs—particularly encoder‑based models—provide higher accuracy at the cost of greater computational power, higher latency, and lower energy efficiency, making them suitable for the same domains when precision outweighs speed. ## Sections - [00:00:00](https://www.youtube.com/watch?v=UvZAaeBhOBs&t=0s) **Ensemble AI Toolbox Framework** - The speaker outlines a dynamic approach that categorizes traditional AI and large language models, then uses an ensemble strategy to combine their complementary strengths for greater business value. ## Full Transcript
AI is everywhere in business these days
and with rapid Innovations continuously
adding new models and new techniques to
what I'm going to conceptualize here is
our AI toolbox it's important that we
also continually adapt how and when we
use what out of that toolbox and in a
very Dynamic way today I want to talk
through a new approach using an ensemble
of AI models that will help get you more
value out of that growing
toolbox so let's go through three
things let's dive in to that
toolbox and understand our tools a
little bit better we'll set ourselves a
framework second let's talk about their
different attributes their
characteristics so that we can
understand their strengths and therefore
when we might use
what and then lastly we'll overlay all
of that with some examples with some use
cases
so let's take a look at our first
tool up until now a lot of the
development and the use cases has been
around traditional
AI traditional AI is built on machine
learning and deep learning
models another tool that we
have are large language models or llms
large language models are largely built
on encoder and decoder models and and
but we'll table that for now
historically a lot of the discussion has
been on when to use traditional AI or
when to use large language models what
this technique opens up is the and it
allows you to leverage multiple AI
models to take advantage of their
different strengths based on the
situation to get you the most out of
your data
so now that we have a framework of our
tools let's dive in a little further and
let's talk about their different
attributes so traditional AI that again
very simplistic view how does
traditional AI work it looks at
structured
data and then following a set of
rules it makes a
prediction and along with that
prediction it gives you a confidence
rating the types of models that you
would see that on there's a couple in
the financial
industry you'll see fraud analysis done
anti-money
laundering Insurance claim
analysis and medical image
analysis those strengths that I
mentioned earlier for traditional
AI that tend to be smaller in size
they tend to have lower latency so
they're
faster and they tend to use less power
so they're more energy
efficient let's jump over to our large
language
models starting with the encoder models
let's talk through two spaces here the
first space they work similar to
traditional AI they start with
structured data they follow a set of
rules they make a prediction and they
give a confidence rating but because
those coder models use a different set
of
techniques and they also use larger more
complex
models they also tend to have a little
bit
higher power they're less energy
efficient a little higher latency so
they're a little slower if you look at
that list why would you ever use those
models they're more accurate their
accuracy is increased
where might you see those I said similar
case fraud anti-money laundering
Insurance analysis and image
analysis I mentioned a second type of
encoder models so a separate space there
instead of starting with the structured
data they actually convert unstructured
data into structured data and I'll give
an example later to to bring that home a
little bit more decoder
models they also start with unstructured
data but they actually then
generate new data these are chat Bots
for
example and again and and that's a big
space there so you can start to
see as you look at the different
characteristics the different strengths
of these um models the dependent on the
situation you may want to use a
different model type so this technique
with that allows you to do is live in
this hybrid
world where you can have multiple models
and then based on the situation based on
what strengths you want to leverage so
maybe accuracy or sustainability speed
size you can very in a very Dynamic way
switch between those models giving you
the most accurate prediction in the
least amount of
time so examples I said now that we have
this framework let's overlay some use
cases on
there so let's start in the um financial
industry let's zoom into fraud analysis
for a minute um because credit card
fraud is unfortunately very relatable to
a lot of us and you can also understand
that in that situation you want the
highest accuracy but in the least amount
of time how do you get those two
different strengths on two different
models again the power of this approach
so you go to the store you swipe your
credit card and because that financial
transaction is probably already running
through a main frame you can use the
powerful AI capabilities that mainframes
have to extract some data while that
transaction is already taking place so
you can run it through a traditional
model and you can get a prediction with
a confidence rating whether that
transaction was fraudulent or not most
of the time you'll have a high
confidence and you move on periodically
if you have a lower confidence you can
switch over to that large language model
where you get the accuracy that you need
so you can see that approach on a
Mainframe maintains the speed and the
sustainability of the smaller models
while leveraging the larger models for
accuracy when you need
it I'll close by doing one short
additional example where we showcase the
reverse of those two so let's look at
Insurance claim analysis insurance
claims are a mix of structured and
unstructured data structured your name
your geographic location a dollar amount
and then unstructured a bunch of text
about that particular incident so you'll
start over with a large language mod
model you'll run it through the
unstructured to structured so you get um
more data to then run through and do
that analysis you may jump straight to a
large language model to get the accuracy
that you need or similar to the last
example you first run it through a
traditional AI model get your prediction
your confidence and then only then if
needed switch
over so hopefully you can start to see
with these two examples the power of
these multimodal Ai and envir
environments and the value that you can
get out of this technique