Tool Calling: Traditional vs Embedded Approaches
Key Points
- Tool calling lets an LLM access real‑time data (e.g., APIs, databases) by having the client send messages plus tool definitions, after which the model suggests which tool to invoke.
- A tool definition includes the tool’s name, description, and required input parameters, and can represent anything from external APIs to code executed by a code interpreter.
- In a typical flow, the LLM recommends a tool call, the client executes the call (e.g., fetching Miami’s temperature), returns the result to the LLM, and the model produces a final answer.
- Traditional tool calling can suffer from hallucinations or incorrect tool calls because the LLM directly dictates which tool to use.
- Embedded tool calling mitigates those issues by inserting a library/framework between the application and the LLM to manage tool definitions and execution, offering tighter control and fewer errors.
Sections
- Tool Calling Workflow Overview - The passage explains how a client app sends messages and tool definitions to an LLM, which then recommends a tool to invoke, after which the client calls the tool, returns the result to the LLM, and the cycle continues until a final answer is produced.
- Embedded Tool Calling Architecture - The passage explains how a middleware library defines and executes tools, appends those definitions to LLM prompts, and uses embedded tool calling to reduce hallucinations compared to traditional tool calling.
Full Transcript
# Tool Calling: Traditional vs Embedded Approaches **Source:** [https://www.youtube.com/watch?v=h8gMhXYAv1k](https://www.youtube.com/watch?v=h8gMhXYAv1k) **Duration:** 00:04:44 ## Summary - Tool calling lets an LLM access real‑time data (e.g., APIs, databases) by having the client send messages plus tool definitions, after which the model suggests which tool to invoke. - A tool definition includes the tool’s name, description, and required input parameters, and can represent anything from external APIs to code executed by a code interpreter. - In a typical flow, the LLM recommends a tool call, the client executes the call (e.g., fetching Miami’s temperature), returns the result to the LLM, and the model produces a final answer. - Traditional tool calling can suffer from hallucinations or incorrect tool calls because the LLM directly dictates which tool to use. - Embedded tool calling mitigates those issues by inserting a library/framework between the application and the LLM to manage tool definitions and execution, offering tighter control and fewer errors. ## Sections - [00:00:00](https://www.youtube.com/watch?v=h8gMhXYAv1k&t=0s) **Tool Calling Workflow Overview** - The passage explains how a client app sends messages and tool definitions to an LLM, which then recommends a tool to invoke, after which the client calls the tool, returns the result to the LLM, and the cycle continues until a final answer is produced. - [00:03:04](https://www.youtube.com/watch?v=h8gMhXYAv1k&t=184s) **Embedded Tool Calling Architecture** - The passage explains how a middleware library defines and executes tools, appends those definitions to LLM prompts, and uses embedded tool calling to reduce hallucinations compared to traditional tool calling. ## Full Transcript
So what is tool calling?
Tool calling is a powerful technique where you make the and context aware of real time data
such as databases or APIs. Typically used to call in via a chat interface.
So you would have your client application in one hand,
And an LLM team on the other side.
For your client application, you would send a set of messages
together with a tool definition to the LLM so you would have your messages here.
Together with your list of tools.
The LLM then look at both your message and the list of tools as going through recommends a tool you should call.
From your client application, you should call this tool and then supply the answer back to the LLM
So it is still response will be interpreted by the LLM.
And this will either tell you the next tool to call or it will give you any final answer.
In your application, you're responsible for creating the tool definition.
So this tool definition includes a couple of things, such as the name of every tool.
It also includes a description for the tool.
So this is where you can give additional information about how to use the tool or when to use it
And it also includes the input parameters needed to make a tool call.
And the tools can be anything.
So the tools could be APIs or databases,
but it could also be code that you interpret via code interpreter.
So let's look at an example.
Assume you want to find the weather in Miami.
You might ask the LLM about the temperature in Miami.
You also provide a list of tools.
And one of these tools is the weather API.
The LLM will look at both your question, which is what is the temperature in Miami?
And we'll also look at the weather API.
And then based on the tool definition for the weather API, it's going to tell you how to call the weather tool.
So in here it's going to create a tool that you can use
right here on this side where you call the API to collect the weather information.
You would then supply the weather information back to the LLM.
So let's say it would be 71 degrees.
The LLM then we'll look at the tool response and then give the final answer, which might be
something in the trends of the weather in Miami is pretty nice. It's 71 degrees.
This has some downsides.
So when you do traditional tool calling, where you have an LLM and a client application, you could see the LLM hallucinate.
Sometimes the LLM can also make up incorrect tool calls.
That's why I also want to look at embedded tool calling.
We just looked at traditional tool calling.
A traditional tool calling has its flaws.
As I mentioned, The LLM could hallucinate or create incorrect tool calls.
That's why I also want to take a embedded tool calling into account.
With a medical calling, you use a library or framework to interact with the LLM and your tool definitions.
The library would be somewhere between your application and the large language model.
In the library, you would do the dual definition, which you would also execute at all goals.
Let's draw a line between these sections here.
So the library will contain your tool definition.
It would also contain the tool execution.
So when you send the message from your application to the large language model, it will go through the library.
So your message could still be what is the temperature in Miami?
The library will then append the tool definition and send your message together with the tools to the LLM
So this will be your message.
Plus your list of tools.
Instead of sending the tool to call to the application or the user, it will be sent to the library, which will then delete all execution.
And this way the library will provide you with the final answer, which could be at 71 degrees in Miami.
When you use embedded tool calling,
the LLM will no longer hallucinate as the library
to help you with it's tool calling or the embedded tool calling is going to take care of the total execution.
And we'll retry the two calls in case it's needed.
So in this video, we looked at both traditional tool calling and also embedded tool calling,
where a specially embedded tool calling will help you to prevent hallucination
or help you with the execution of tools, which could be APIs, databases or code.