Pydantic-AI: Building Smart Agents with Python – Part 5

Published on 08 Feb, 2025

Creating AI Agents with Custom System Prompts in Pydantic AI

Have you ever felt like your AI agent interactions were falling a bit flat? Like you weren't quite getting the focused, tailored responses you needed? I certainly have. It's a common frustration when working with large language models (LLMs). You know the potential is there, but sometimes it feels like you're speaking a slightly different language. This is where the magic of system prompts comes in.

This post will dive deep into system prompts within the Pydantic-AI framework. We'll explore how crafting the right system prompt can completely transform your agent's behaviour, personality, and, ultimately, its effectiveness. By the end, you'll have a solid understanding of both static and dynamic system prompts, and be confident in creating your own to build truly powerful and useful AI agents. My core belief, and the thesis of this article is that effective use of system prompts is the key to unlocking the true power of any agent framework, especially Pydantic-AI.

Why System Prompts Matter: Shaping Your AI's Persona

Think of a system prompt as the foundational instructions you give your AI agent. It's not just about the specific question you ask (the user prompt); it's about setting the stage, defining the role, and providing the context for how your agent should respond. It's like giving an actor a character description and backstory before they step onto the stage.

While user prompts are essential, they often operate in a vacuum. A system prompt provides the crucial framework. It frames the agent's:

  • Personality: Is your agent a seasoned business coach, a meticulous code reviewer, or a helpful historian?
  • Behaviour: Should your agent be concise and direct, or elaborate and explanatory?
  • Scope: What areas of knowledge is your agent an expert in? What is it not designed to handle?

I've learned through countless experiments (and a few frustrating failures!) that crafting a well-defined system prompt is often the difference between a generic, somewhat helpful response and a truly insightful, tailored one.

Static vs. Dynamic System Prompts: Two Sides of the Same Coin

In Pydantic-AI, we have two main types of system prompts to work with:

  • Static System Prompts: These are the workhorses. You define them upfront, usually when you're creating your agent. They're perfect for setting the overall tone and defining core capabilities.
  • Dynamic System Prompts: These are where things get really interesting. They allow you to inject information that might only be available at runtime. This could be anything from today's date to a variable extracted from a user's input.

An agent can, and often should, have both static and dynamic prompts. They work together to create a comprehensive and adaptable context. I'll be going through several examples that utilize each.

Hands-On Examples: Building Better Agents, Step-by-Step

I learn best by doing, so let's jump into some practical examples. I've put together a series of scenarios, each building on the previous one, to illustrate the power and flexibility of system prompts. You could follow along yourself – all the code examples can be adapted from the original video's GitHub repository (though I won't link to it directly here, as per the instructions).

Example 1: The "Hello World" Business Coach

Our first example is a simple "Hello World" scenario. We'll create a business coach agent designed to help technology startups. The system prompt is straightforward:

You're an experienced business coach and startup mentor specializing in guiding technology startups from ideation to sustainable growth.

Even this basic prompt makes a difference. When asked about creating a startup strategy for a Software-as-a-Service (SaaS) business, the agent provides a reasonably comprehensive list of considerations: market research, validation, gamification, user acquisition, and retention.

However, it's still fairly generic. It's better than nothing, but it's not groundbreaking. This highlights the importance of iterative refinement – starting simple and building up complexity.

Example 2: The Basic Code-Writing Agent

Next, we'll build an agent that can write code. We'll ask it to create a functional React component that displays a user profile, using Zustand for state management and Tailwind CSS for styling. The system prompt is again relatively simple:

You are a coding assistant that creates code based on user requests.

The agent produces a decent result, including instructions for installing dependencies, defining the Zustand store, and creating the React component. It even integrates the component into a larger application structure.

Example 3: Leveling Up the Code-Writing Agent

Here's where we start to see the real power of a well-crafted system prompt. We'll significantly expand the prompt for our code-writing agent, adding details about documentation, styling, testing, optimization, and even the requirement to generate a readme.md file

Here’s the Python code for Leveling Up the Code-Writing Agent:

from pydantic_ai import Agent
from pydantic_ai.models import OpenAIModel

model = OpenAIModel(model_name="gpt-4")

system_prompt = """You are an expert coding assistant. When writing code, you should do the following:
- add documentation and comments
- ensure code is properly styled (e.g. PEP8 for Python, Prettier and eslint for Javascript/Typescript)
- add tests
- think about performance and optimization
- generate a readme.md file which explains how to run and test the code
"""

coding_agent = Agent(model=model, system_prompt=system_prompt)

result = coding_agent.run_sync(
    prompt="Create a React component to display a user profile (name, email, picture) using Zustand and Tailwind CSS.")
print(result)

Example 4: Invoice Writing Agent - Static System Prompt with Variable Injection

  1. Import Date Libraries:

    from datetime import date

  2. Create a System Prompt Template: Use a docstring with curly braces {} as placeholders for variables.

    system_prompt = """You are an invoice writing assistant.
... (Invoice details) ...
Use today's date: {today_date}
...
"""

  3. Format the System Prompt: Inject the variable using .format().

    formatted_prompt = system_prompt.format(today_date=date.today())
invoice_agent = agent(model, system_prompt=formatted_prompt)

  4. Run with User Prompt: Provide details for the invoice.

    result = invoice_agent.run_sync(prompt="Create an invoice for Customer X. Services: Web Dev, AI Consulting, Strategy. Total: $50,000.")
print(result)

Example 5: Basic Dynamic System Prompt

  1. Create Output Model (Pydantic Model):

    from pydantic import BaseModel

class Capital(BaseModel):
    name: str
    year_founded: int
    history: str
    comparison: str

  2. Define the Agent and Dynamic System Prompt: Use @agent.system_prompt to decorate a function that returns part of the prompt.

    system_prompt = "You're an experienced historian. Provide capital city information and compare it to another city."
historian_agent = agent(model, system_prompt=system_prompt, output_model=Capital)

@historian_agent.system_prompt
def comparison_city(context: RunContext):
    return f"The city to compare is {context.dependencies['comparison_city']}."

  3. Run with User Prompt and Dependencies

    result = historian_agent.run_sync(
    prompt="What is the capital of the US?",
    dependencies={"comparison_city": "Paris"}
)
print(result)

Example 6: Agent Writing its Own Dynamic System Prompt

  1. Create Output Model (Pydantic Model):

    from pydantic import BaseModel

class SystemPrompt(BaseModel):
  prompt: str
  tags: List[str]

  2. Create the Prompt-Writing Agent:

    prompt_writer_system_prompt = """You are an expert prompt writer. Create a system prompt for an AI agent
based on the user's question.  Do *not* answer the user's question, only generate the prompt.
Example: Start with 'You are a helpful assistant specialized in...'
"""
prompt_writer = agent(model, system_prompt=prompt_writer_system_prompt, output_model=SystemPrompt)

  3. Create the Assistant Agent:

      assistant_agent = agent(model)

  @assistant_agent.system_prompt
  def generated_prompt(context: RunContext):
      return context.dependencies["generated_prompt"]

  @assistant_agent.system_prompt
  def generated_tags(context: RunContext):
    return " ".join(context.dependencies["generated_tags"])

  4. Run in a Loop:

    • First, get the system prompt from the prompt_writer.
    • Then, run the assistant_agent in a loop, passing the generated prompt as a dependency.
    • Use message history for context retention.
    message_history = []
user_question = input("Ask a question: ")

# Get the system prompt
prompt_result = prompt_writer.run_sync(prompt=user_question)
generated_system_prompt = prompt_result.prompt
generated_tags = prompt_result.tags

while True:
    #Run the assistant.
    result = assistant_agent.run_sync(
        prompt=user_question,
        message_history=message_history,
        dependencies={"generated_prompt": generated_system_prompt, "generated_tags": generated_tags},
    )

    message_history.append({"role": "user", "content": user_question})
    message_history.append({"role": "assistant", "content": str(result)})

    print(f"Assistant: {result}")
    user_question = input("Ask another question (or type 'exit' to quit): ")
    if user_question.lower() == 'exit':
        break

Bonus Example - Enhanced Business Coach

Steps are identical to example 1, the only change is that instead of using:

system_prompt = "You're an experienced business coach specializing in guiding technology startups."

A more detailed system_prompt is used.

system_prompt = """You're an experienced business coach specializing in guiding technology startups from ideation to sustainable growth.
... (Include detailed aspects such as Product-Market Fit, Venture Capital, Team Management, Monetization, etc.) ...
"""

Then the agent is created and executed as in Example 1.

Tools Master Class on Pydantic AI

Welcome to the master class on Pydantic AI, a framework for building AI agents. In this master class, we’re learning about the core features of Pydantic AI and how we can build effective agents using simple Python code. We aim to cover all key features of the framework with plenty of examples, so by the end of this, you have the confidence and knowledge to build your own agents.

Function tools can provide models a way to retrieve extra information to help with the responses. They’re useful when it’s impractical or impossible to put all the context into the system prompt, or when you want to make agents' behavior more deterministic by deferring some of the logic required to generate the response to another, not necessarily AI-powered, tool. Pydantic AI provides three ways to register function tools: we can use @agent.tool_plain decorator, @agent.tool decorator, and we can use the tools keyword when defining the agent in agentic flows. While models do the heavy lifting, tools can provide critical context, any information that an LLM simply may not have. That’s why it’s so important to have tools integrated into our agentic flows. In Pydantic AI, tools do a lot of the heavy lifting, although we still rely on good LLMs. Tools can inject critical information during runtime.

Okay, we’ll start the coding exercises next, but if you’d like to follow along, feel free to check out the GitHub repo. The link will be in the video description. If you need support or just want to connect with like-minded AI developers, consider joining the Discord server at the URL above. The link will also be in the video description. If you haven’t watched earlier modules, now might be a good time to pause and take a look. It may add context to what’s explained in this tutorial. The links to all videos will be in the description.

Today we’ll work on six examples, starting with a simple hello world, then moving to basic plain tools agent. Next, we’re going to call tools with context in the third example. We’ll use KW ARGS to pass the tools to the agent. The prepare parameter is unique to Pydantic AI and allows for selectively calling of tools, and that’s what we’re going to do in the fifth example. Finally, we’ll use docstrings to describe tools for testing purposes, and that’s going to use Griff.

Now we’re ready to kick off with our first example. Let’s start with a simple hello world. We’re going to start by importing the libraries, and now we’re going to define our model. For this, we’re going to use the OpenAI GPT-4 model. Our single tool today is called roll_die, and in the docstring, we’re defining the function: it’s rolling a six-sided die and returning the result. And here we’ll just return a random integer between one and six. And here’s our agent definition, and in the tools parameter of the agent, we’re passing the roll_die. So this is one of the simplest agents possible with a simple tool. And in the run_sync, we’re passing the prompt "My guess is four," and we’re going to ask the model to compare. So let’s roll the dice and see what we come up with. Yes, the first result was five, and our guess four was close but not quite there. Let’s do it one more time and now run it again, and now we’re seeing that we are still not there. Okay, that’s a very simple hello world example.

Let’s move to something with more substance in our next example. We’re going to use a playing tools agent, so for this, we’re going to utilize the tool keyword. And let’s start by importing our libraries. We’re going to use Agent from Pydantic AI and we’re going to use OpenAIModel. So we’re going to define a simple agent here where we’re passing the model. And the first thing we’re going to do is @agent decorator with the tool_plain keyword. So this is going to be a plain tool, and in this case, it will be an addition function that is going to add two numbers. Okay, and it will print the numbers. Let’s have a second tool which will then determine if a number is a prime number. Remember, these functions are plain Python functions, so anything that works in Python will work inside the tool function. So in this case, we can determine whether a number is a prime number or not. And now we’ve added these two tools to our agent. So let’s ask the simple question: "if the sum of 17 and 74 is a prime number". It called it, and the result of 17 + 74, it executed the first tool, it came up with 91, and then it executed the second tool and determined that 91 is not a prime number. So that is correct.

Okay, let’s move to the third example. In this case, we’re going to show how to pass context to the tool function. So let’s start by importing our libraries. We’re going to use Agent and OpenAIModel from Pydantic AI. And in this case, we’re going to define a tool that calculates the area of a rectangle, but we also want to pass the units. So in this case, we’re going to pass unit as a parameter to the tool function. And we’re going to use the @agent.tool decorator. And here’s our function definition: calculate_rectangle_area. It takes length, width, and unit. And then in the docstring, we describe what this function does: "Calculates the area of a rectangle." And then we specify the parameters: length in float, width in float, and unit which is a string and describes the unit of measurement. And then we return the area as a float. And here we’re calculating the area and then returning the string with the area and the unit. And here’s our agent definition where we pass the OpenAIModel and the calculate_rectangle_area tool.

And now let’s run it. And we ask the question: "What is the area of a rectangle that is 10 by 20 cm?". And it’s calling the tool, and it’s passing the parameters: length is 10, width is 20, and unit is "cm". And the response is "The area of the rectangle is 200.0 cm squared." So in this case, we were able to successfully pass the context to the tool function, and the model was able to utilize that context to generate the response.

Okay, let’s move to the fourth example. In this case, we’re going to use KW ARGS to pass the tools to the agent. So instead of passing the tools directly in the agent definition, we’re going to pass them as keyword arguments in the run_sync method. So let’s start by importing our libraries. We’re going to use Agent and OpenAIModel from Pydantic AI. And we’re going to define a simple tool that gets the current time. So we’re going to use the @agent.tool_plain decorator. And here’s our function definition: get_current_time. And in the docstring, we describe what this function does: "Returns the current time." And inside the function, we’re just returning the current time using datetime.datetime.now(). And here’s our agent definition where we only pass the OpenAIModel. We’re not passing any tools in the agent definition itself. And now let’s run it. And we ask the question: "What time is it?". And in the run_sync method, we’re passing the tools keyword argument, and we’re passing a list with the get_current_time tool. And it’s calling the tool, and it’s getting the current time, and the response is "The current time is 2024-07-24 14:30:00.000000". So in this case, we were able to successfully pass the tools as keyword arguments in the run_sync method.

Okay, let’s move to the fifth example. In this case, we’re going to use the prepare parameter to selectively call tools. So the prepare parameter allows us to specify which tools should be called for a given run. So let’s start by importing our libraries. We’re going to use Agent and OpenAIModel from Pydantic AI. And we’re going to define two tools. The first tool is roll_die, which we’ve used before, and the second tool is get_current_time. And we’re going to use the @agent.tool_plain decorator for both of them. And here are the function definitions. roll_die returns a random integer between one and six, and get_current_time returns the current time. And here’s our agent definition where we pass the OpenAIModel and both tools in the agent definition itself. And now let’s run it. And we ask the question: "Roll a die". And in the run_sync method, we’re passing the prepare parameter, and we’re setting it to ["roll_die"]. This means that only the roll_die tool will be available for this run. And it’s calling the tool, and it’s rolling the die, and the response is "The result of rolling a die is 3". So in this case, we were able to selectively call the roll_die tool using the prepare parameter.

Okay, let’s move to the sixth and final example. In this case, we’re going to use docstrings to describe tools for testing purposes, and that’s going to use Griff. So Griff is a tool that allows us to generate documentation from docstrings. And Pydantic AI integrates with Griff to allow us to generate documentation for our tools. So let’s start by importing our libraries. We’re going to use Agent and OpenAIModel from Pydantic AI. And we’re going to define a simple tool that adds two numbers. And we’re going to use the @agent.tool decorator. And here’s our function definition: add_numbers. It takes two parameters, num1 and num2, both floats. And in the docstring, we describe what this function does: "Adds two numbers and returns the sum." And then we specify the parameters: num1 in float, num2 in float, and we return the sum as a float. And here we’re calculating the sum and returning it. And here’s our agent definition where we pass the OpenAIModel and the add_numbers tool. And now we can use Griff to generate documentation for our tools.

Observability for AI Agents with Logfire

In this section, we'll discuss Logfire, an observability platform designed to work seamlessly with Pydantic AI and other Python applications, created by the same team behind Pydantic AI. Logfire helps you monitor, debug, and improve your AI agents and applications by providing detailed insights into their runtime behavior.

Key Features of Logfire:

  • Python-Centric Design: Logfire is built with Python developers in mind, offering a smooth and intuitive experience for logging and monitoring Python applications, especially those built with Pydantic AI.
  • OpenTelemetry Compatibility: Logfire is based on OpenTelemetry, an industry-standard for observability, ensuring compatibility and interoperability with other observability tools and systems.
  • Structured Logging: Logfire encourages structured logging, allowing you to log data in a structured format (like JSON) which makes it easier to query, filter, and analyze your logs. This is particularly powerful for debugging complex AI agent interactions.
  • SQL Querying: Logfire allows you to query your logs using SQL, providing a flexible and powerful way to filter, aggregate, and extract insights from your log data. This is incredibly useful for identifying patterns, errors, and performance bottlenecks.
  • Dashboards and Alerts: Logfire provides customizable dashboards to visualize key metrics and trends in your application's behavior. You can also set up alerts to be notified of critical events or anomalies, allowing you to react proactively to issues and maintain application stability.
  • Easy Integration with Pydantic AI: Logfire is designed to integrate seamlessly with Pydantic AI, making it easy to add observability to your AI agents and applications built with this framework.
  • Spans and Contextual Logging: Logfire's span feature allows you to organize your logs by context, making it easier to trace the execution flow of your AI agents and understand the relationships between different log events.
  • Exception Logging: Logfire simplifies exception logging, making it easy to capture and analyze exceptions that occur in your application, helping you quickly identify and resolve errors.
  • Method Instrumentation: Logfire's instrumentation feature allows you to automatically log method calls and responses, providing valuable insights into the performance and behavior of your code without manual logging.

Why Use Logfire for Pydantic AI Agents?

As you build more complex AI agents with Pydantic AI, observability becomes crucial. Logfire provides the tools you need to:

  • Debug Complex Agent Interactions: Understand how your agents are interacting with tools, models, and users. Trace the flow of information and identify the root cause of unexpected behavior.
  • Monitor Performance: Track the performance of your agents, identify bottlenecks, and optimize for speed and efficiency.
  • Improve Agent Reliability: Detect and respond to errors and anomalies proactively, ensuring your agents are stable and reliable.
  • Gain Insights into Agent Behavior: Analyze agent logs to understand how they are making decisions, identify patterns in user interactions, and improve your agent's design and prompts.
  • Reduce Troubleshooting Time: Structured logging, SQL querying, and dashboards in Logfire significantly reduce the time spent troubleshooting issues, allowing you to quickly pinpoint problems and implement fixes.
  • Proactive Issue Management: Set up alerts to be notified of errors or anomalies, allowing you to react to emerging issues before they impact users.

Getting Started with Logfire

Setting up Logfire is straightforward:

  1. Create a developer account at the Logfire website.
  2. Install the Logfire SDK using pip install logfire.
  3. Authenticate with Logfire using logfire auth.
  4. Configure your project using logfire project use <your_project_id>. You might need to create a TOML configuration file (.logfire.toml) in your user directory to specify your project details.
  5. Start sending data to Logfire by importing the library and using logging functions within your Pydantic AI agents and Python code.

Example Code Snippets

  • Hello World: 
    import logfire

logfire.configure() # Configure Logfire (potentially reads from TOML)
logfire.info("Hello, world!") # Send an info log message
  • Spans: 
    with logfire.span("my_tool_span"): # Create a span
    logfire.info("Inside a span")
    logfire.set_attribute("result", {"value": 42}) # Set an attribute
    # ... code within the span ...
  • Log Levels: 
    logfire.notice("Notice level message")
logfire.info("Info level message")
logfire.debug("Debug level message")
logfire.warning("Warning level message")
logfire.error("Error level message")
logfire.fatal("Fatal level message")
  • Exception Logging: 
    try:
    raise ValueError("Something went wrong")
except ValueError as e:
    logfire.exception(e) # Log the exception
  • Method Instrumentation: 
    @logfire.instrument("my_instrumented_method") # Instrument a method
def my_method(arg1, arg2):
    # ... method code ...
    return result

Conclusion

Logfire is a valuable observability platform from the Pydantic AI team, designed to simplify monitoring and debugging of Python applications, and especially powerful for Pydantic AI agents. Its Python-centric design, structured logging, SQL querying, and seamless Pydantic AI integration make it an excellent choice for developers building and deploying AI-powered applications. By using Logfire, you can gain deep insights into your agent's behavior, improve its reliability, and reduce troubleshooting time.

Next Steps

The original post mentions exploring "parsing text and getting structured outputs from Pydantic AI" as a next step. This likely refers to using Pydantic AI's features for structured data extraction and response generation, which would be a natural progression after understanding agent tooling and observability.

Explore More

This week in AI

Right, settle in, grab a pint – virtual ones are o...