Modern LangChain Architecture & Tool-Calling Agents

This lesson on Modern LangChain Architecture & Tool-Calling Agents is hands-on and example-driven. You will be able to construct schema-validated custom tools using Python type hints, configure tool-calling agents with memory and intermediate scratchpads, and execute autonomous multi-step reasoning loops using AgentExecutor.

What You'll Be Able To Do

• Define custom tools from Python functions using the @tool decorator and strict type annotations.
• Inspect tool JSON schemas using Pydantic validation utilities to verify parameter descriptions.
• Construct an agent prompt template incorporating system instructions, chat history, and the agent_scratchpad placeholder.
• Instantiate a tool-calling agent using create_tool_calling_agent and wire it with ChatOpenAI.
• Orchestrate multi-step execution loops by wrapping agents and tool lists inside AgentExecutor.

Detailed Concept Walkthrough

1. Custom Tool Creation with @tool

Tools are Python functions wrapped into StructuredTool instances that expose JSON schemas so LLMs understand when and how to call them. The model inspects the docstring and type hints to determine parameter names, types, and operational requirements.

• Mechanism: Applying the @tool decorator automatically inspects the Python function's docstring and type hints to generate an underlying Pydantic schema. This schema tells the LLM what the tool accomplishes and what argument types it accepts.
• Under the Hood: LangChain converts the function into a StructuredTool object containing .args_schema, .name, and .description. You can inspect the generated parameter structure directly by calling .args_schema.schema() or .args_schema.model_json_schema() on the decorated object.
• Best Practice: Always provide clear, unambiguous docstrings and explicit type hints (such as int or str). The LLM relies directly on the docstring text to decide tool selection during reasoning.

from langchain_core.tools import tool

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers together."""
    return a * b

# Inspect the generated Pydantic JSON schema
print(multiply.name)
print(multiply.args_schema.model_json_schema())

Key Takeaway: An LLM sees only the tool name, docstring, and argument schema—accurate type hints and docstrings are required for correct tool selection.

2. Prompt Design & Agent Scratchpad

Agent prompts require dedicated placeholders to separate persistent conversational history from internal reasoning steps. The agent scratchpad dynamically stores intermediate tool calls and resulting observations within a single execution cycle.

• Mechanism: An agent prompt requires a MessagesPlaceholder(variable_name='chat_history') for conversation continuity and a MessagesPlaceholder(variable_name='agent_scratchpad') for multi-step reasoning. The scratchpad tracks intermediate tool invocations and execution outputs before the final answer is produced.
• Execution Flow: When an agent decides to invoke a tool, the tool request and subsequent tool observation are appended to agent_scratchpad as messages. The agent re-evaluates the prompt with this updated context to decide if further tool calls are necessary.
• Syntax Rule: The placeholder variable name agent_scratchpad is mandatory when using create_tool_calling_agent because the internal formatter specifically writes intermediate tool messages into that exact key.

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant with calculation tools."),
    MessagesPlaceholder(variable_name="chat_history", optional=True),
    ("human", "{input}"),
    MessagesPlaceholder(variable_name="agent_scratchpad"),
])

Key Takeaway: The agent scratchpad acts as working memory for multi-step tool calls and observations, distinct from user chat history.

3. Agent Assembly & create_tool_calling_agent

The tool-calling agent binds a language model, a defined toolset, and a structured prompt into a unified decision engine. The agent's sole responsibility is planning: evaluating context and emitting either an action decision or a final response.

• Mechanism: create_tool_calling_agent(llm, tools, prompt) binds the provided tool schemas directly to the chat model's tool-calling API. It constructs a Runnable sequence that accepts inputs and outputs an AgentAction or AgentFinish object.
• Under the Hood: The agent does not execute the underlying Python functions itself; it only returns a structured decision containing the tool name and parsed keyword arguments. If the model determines no tools are required, it yields an AgentFinish containing the final text output.
• Nuance: Memory components like ConversationBufferMemory can store previous conversation turns, but modern LangChain architectures increasingly favour explicit message histories via RunnableWithMessageHistory.

from langchain_openai import ChatOpenAI
from langchain.agents import create_tool_calling_agent

llm = ChatOpenAI(model="gpt-4o", temperature=0)
tools = [multiply]

# Assemble the decision engine
agent = create_tool_calling_agent(llm=llm, tools=tools, prompt=prompt)

Key Takeaway: An agent is purely a decision engine that selects tools and constructs arguments; it never executes the tools itself.

4. Agent Execution Loop with AgentExecutor

AgentExecutor is the runtime harness that manages the iterative loop between LLM decisions and actual tool execution. It executes the tools selected by the agent and feeds their outputs back into the scratchpad until a final response is generated.

• Execution Flow: AgentExecutor queries the agent, intercepts any returned AgentAction, finds the matching tool in its registered tool list, executes the function with the provided arguments, and appends the result to agent_scratchpad.
• Under the Hood: This cycle repeats until the agent outputs an AgentFinish object, indicating no further tool calls are required, at which point AgentExecutor returns the final response payload.
• Syntax Rule: You must pass the list of tools explicitly to both create_tool_calling_agent (for schema binding) and AgentExecutor (for runtime execution).

from langchain.agents import AgentExecutor

# Initialize the executor loop with agent and tools
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# Execute a multi-step query
result = executor.invoke({"input": "What is 12 multiplied by 14?"})
print(result["output"])

Key Takeaway: AgentExecutor provides the runtime environment that runs the tool code and orchestrates the feedback loop until reaching an AgentFinish.

Topics Covered in Modern LangChain Architecture & Tool-Calling Agents

1. Introduction & Environment Setup (0:00 - 1:27) — Covers agent fundamentals, SerpAPI search setup, and LangSmith tracing configuration.
2. Tool Anatomy & Requirements (1:27 - 3:18) — Explains requirements for tool docstrings, explicit naming, and type annotations for LLM compatibility.
3. Defining Custom Tools with @tool (3:18 - 6:05) — Demonstrates converting Python arithmetic functions into StructuredTools and inspecting their Pydantic schemas.
4. Tool Invocation Mechanics (6:05 - 7:05) — Details how JSON strings returned by the LLM are parsed and executed as keyword arguments against functions.
5. Prompt Design & Agent Scratchpad (7:05 - 8:50) — Walks through building ChatPromptTemplate with placeholders for chat history and the intermediate agent scratchpad.
6. Agent Assembly & Memory (8:50 - 10:42) — Combines ChatOpenAI, tool definitions, prompts, and memory via create_tool_calling_agent.
7. Agent Actions & AgentExecutor Loop (10:42 - 13:31) — Contrasts AgentAction decision outputs with the full execution loop managed by AgentExecutor.

LangChain & LangGraph for AI Agents Cheat Sheet

• @tool — Converts Python function to schema-validated LangChain tool

  @tool
  def add(a: int, b: int) -> int:
      """Add two numbers."""
      return a + b
  

• tool.args_schema.model_json_schema() — Inspects generated Pydantic JSON schema for parameters

  schema = add.args_schema.model_json_schema()
  

• MessagesPlaceholder(variable_name=...) — Allocates dynamic message slots in prompt templates

  MessagesPlaceholder(variable_name="agent_scratchpad")
  

• create_tool_calling_agent(llm, tools, prompt) — Builds decision-making agent Runnable with tool bindings

  agent = create_tool_calling_agent(llm, [add], prompt)
  

• AgentExecutor(agent=..., tools=..., verbose=...) — Orchestrates multi-step agent reasoning and tool execution loop

  executor = AgentExecutor(agent=agent, tools=[add], verbose=True)
  

• executor.invoke({"input": ...}) — Executes full agent loop and returns result dictionary

  response = executor.invoke({"input": "Add 4 and 5"})
  

Comparison Table

Common Pitfalls

• Mistake: Omitting docstrings or type annotations on functions decorated with @tool. Avoid: Always include clear docstrings and parameter type hints so the LLM receives a complete JSON schema.
• Mistake: Expecting create_tool_calling_agent to execute tool functions automatically. Avoid: Wrap the agent inside an AgentExecutor and pass the tool list to execute the code.
• Mistake: Omitting the agent_scratchpad MessagesPlaceholder from the agent prompt. Avoid: Include MessagesPlaceholder(variable_name='agent_scratchpad') as the final prompt element before invocation.
• Mistake: Supplying tools only to create_tool_calling_agent but omitting them from AgentExecutor. Avoid: Pass the tool list explicitly to both the agent and the executor constructor.

FAQs

• Why do tools require both type hints and docstrings? LangChain uses type hints to construct the JSON schema parameter types, while the LLM reads the docstring to understand what the tool does and when to call it.
• What is the purpose of the agent_scratchpad? It holds the sequence of intermediate tool calls and their returned outputs during multi-step reasoning before the model delivers a final answer.
• Why must the tools array be passed to both create_tool_calling_agent and AgentExecutor? The agent requires tools to format schemas for LLM tool binding, while AgentExecutor requires the actual functions to run them during execution.
• How does AgentExecutor determine when to stop running? It continues looping until the agent emits an AgentFinish object instead of an AgentAction request, or until configured iteration limits are met.