```

PydanticAI: Bringing the FastAPI Feel to Generative AI Application Development

The landscape of web development was irrevocably changed with the arrival of FastAPI. Built upon the robust foundation of Pydantic, FastAPI offered a breath of fresh air – an innovative, ergonomic design that prioritised developer experience without sacrificing performance. It was a framework that simply felt right for modern Python web applications.

Interestingly, Pydantic's influence extends far beyond web servers. Look under the hood of virtually any agent framework or Large Language Model (LLM) library in Python, and you'll likely find Pydantic diligently validating data. Yet, when the team behind Pydantic embarked on their own LLM journey with Pydantic Logfire, they recognised a gap. The intuitive, streamlined feeling of FastAPI was conspicuously absent in the existing GenAI toolkits. This realisation sparked the creation of PydanticAI, a Python Agent Framework born from a simple yet powerful aim: to transplant that beloved FastAPI feeling directly into the realm of Generative AI application development.

Personally I love Pydantic AI and use it in all my production apps.

Here is a link to my github repository for you to check out -- a Python Agent Framework designed to make it easier to build production-grade applications with Generative AI.

Why Choose PydanticAI?

PydanticAI isn't just another agent framework; it arrives with a pedigree and a philosophy that sets it apart. Here's a breakdown of the core reasons to consider PydanticAI for your next GenAI project:

Built by the Pydantic Team: This is a significant advantage from the outset. The creators of Pydantic are deeply embedded in the Python GenAI ecosystem. Pydantic is the validation backbone for giants like the OpenAI SDK, Anthropic SDK, LangChain, LlamaIndex, AutoGPT, Transformers, CrewAI, Instructor, and countless others. This lineage guarantees a framework built on solid foundations and a profound understanding of the challenges and nuances of working with AI models.
Model-Agnostic Design: In a rapidly evolving landscape, flexibility is paramount. PydanticAI embraces model agnosticism, offering native support for a wide array of providers including OpenAI, Anthropic, Gemini, Deepseek, Ollama, Groq, Cohere, and Mistral. Furthermore, its architecture includes a straightforward interface, empowering developers to seamlessly integrate support for additional models as they emerge. This future-proofs your applications and prevents vendor lock-in.
Seamless Pydantic Logfire Integration: Debugging and monitoring LLM applications can be notoriously opaque. PydanticAI addresses this head-on with its tight integration with Pydantic Logfire. This powerful combination provides real-time debugging capabilities, performance monitoring, and insightful behaviour tracking for your LLM-powered creations. Understanding the inner workings of your agents becomes significantly easier, accelerating development and ensuring production stability.
Type-Safe from the Ground Up: Leveraging Pydantic's core strength, PydanticAI prioritises type safety. It is designed to make type checking not just a formality, but a powerful tool for building robust and predictable AI applications. This emphasis on types improves code clarity, reduces errors, and enhances the overall developer experience, especially in complex agent orchestrations.
Python-Centric Design Philosophy: PydanticAI champions Python's inherent strengths. It encourages the use of familiar Pythonic control flow and agent composition techniques. This deliberate design choice makes it remarkably easy to apply standard Python best practices – the same methodologies you would employ in any conventional (non-AI) software project – to your AI-driven initiatives. This reduces the learning curve and promotes maintainable, idiomatic code.
Structured and Validated Responses: Harnessing the data validation prowess of Pydantic, PydanticAI ensures that model outputs are not just text blobs, but structured, validated data. By defining Pydantic models for your expected responses, you guarantee consistency across different runs and simplify downstream processing. This structured approach is crucial for building reliable and predictable AI systems.
Optional Dependency Injection System: PydanticAI incorporates an optional dependency injection system, a feature often associated with more mature frameworks. This system allows you to inject data and services into your agent's system prompts, tools, and result validators. This seemingly advanced feature is incredibly practical, particularly for testing and evaluation-driven iterative development, promoting modularity and testability.
Streamed Responses for Real-Time Interactions: In user-facing AI applications, responsiveness is key. PydanticAI's support for streamed LLM outputs delivers continuous, immediate results. Crucially, this streaming is coupled with real-time validation, ensuring rapid and accurate responses, enhancing the user experience and perceived performance of your applications.
Graph Support with Pydantic Graph: For complex AI applications where intricate agent interactions are needed, managing control flow can quickly become unwieldy. PydanticAI, in conjunction with Pydantic Graph, offers a solution. Pydantic Graph allows you to define agent interactions and workflows as graphs using Python type hints. This declarative approach provides a powerful way to manage complexity and avoid the dreaded "spaghetti code" often associated with intricate application logic.

Diving into PydanticAI in Practice

To truly appreciate the capabilities of PydanticAI, let's explore some practical examples, mirroring an exploration conducted by Andy, a seasoned AI practitioner. These examples showcase the core concepts and demonstrate how PydanticAI brings structure and clarity to agent development.

The Basic Agent: Hello World

Every framework introduction deserves a "Hello World" moment. In PydanticAI, this starts with defining a model and an agent. First, selecting the LLM:

from pydantic_ai import OpenAIModel, Agent

model = OpenAIModel(model_name="gpt-4")

Here, we instantiate an OpenAI model, specifying "gpt-4". PydanticAI offers flexibility in model selection, allowing direct instantiation as shown, or via a more generic approach using OpenAI(model_name="gpt-4"). The more explicit method is favoured for clarity and reusability.

Next, we define a basic agent:

basic_agent = Agent(
    model=model,
    system_prompt="Helpful customer support agent"
)

The Agent class acts as a container for essential elements: a system prompt, potential tools, desired result types, dependencies, and the chosen LLM model. At its core, an agent requires at least a model and a system prompt. In this basic example, the system prompt is straightforward: "Helpful customer support agent".

Running this agent is simple:

response = basic_agent.run_sync("How do I track my order?")
print(response.print_data())

The run_sync method executes the agent synchronously. PydanticAI also offers run for asynchronous execution and run_stream for streaming responses. The result is encapsulated in a response object, packed with information including the LLM's reply, message history, and cost metrics. Accessing response.print_data() reveals the model's answer, for instance: "To track your order, please check the confirmation email we sent to you."

The response object provides a wealth of detail. response.messages displays the full conversation history, including the system prompt, user prompt, and model reply. Even cost breakdowns are available via response.cost. For conversational applications, response.messages can be passed back into subsequent agent.run_sync calls, enabling seamless multi-turn dialogues.

Structured Output with Pydantic

Moving beyond simple text responses, PydanticAI truly shines when structuring LLM outputs using Pydantic models. Let's define a ResponseModel:

from pydantic import BaseModel

class ResponseModel(BaseModel):
    response: str
    needs_escalation: bool
    followup_required: bool
    sentiment: str

This model dictates the desired structure of the LLM's reply. Now, we create an agent that leverages this model:

structured_agent = Agent[ResponseModel]( # Note the type hint here
    model=model,
    result_type=ResponseModel,
    system_prompt="Helpful customer support agent that provides structured responses."
)

structured_response = structured_agent.run_sync("How do I track my order?")
print(structured_response.data.model_dump_json(indent=2))

Crucially, note the type hint Agent[ResponseModel] and the result_type=ResponseModel parameter. This instructs PydanticAI to validate the LLM's output against the ResponseModel schema. The structured_response.data now contains a fully validated ResponseModel instance, not just raw text. model_dump_json(indent=2) neatly displays the structured JSON output, revealing fields like response, needs_escalation, followup_required, and sentiment populated by the LLM in accordance with the defined schema.

Dependency Injection for Contextual Awareness

PydanticAI's dependency injection system elevates agent capabilities significantly. Imagine a customer service scenario where agent responses need to be tailored to specific customer details. Let's define Pydantic models for Customer and Order information:

class Order(BaseModel):
    order_id: str
    items: list[str]

class CustomerDetails(BaseModel):
    customer_id: str
    name: str
    email: str
    orders: list[Order]

Now, we define an agent that depends on CustomerDetails:

dependency_agent = Agent[ResponseModel, CustomerDetails]( # Type hint with ResponseModel and CustomerDetails
    model=model,
    result_type=ResponseModel,
    dependency_type=CustomerDetails,
    system_prompt="Helpful customer support agent, contextual aware."
)

Again, type hints are key: Agent[ResponseModel, CustomerDetails] and dependency_type=CustomerDetails. To inject dynamic customer information into the system prompt, we use a decorator:

from pydantic_ai import RunContext

@dependency_agent.add_system_prompt
async def add_customer_name(context: RunContext[CustomerDetails]):
    customer_md = context.dependencies.to_markdown() # Assuming a to_markdown utility function exists
    return f"Customer Details:\n{customer_md}"

The @dependency_agent.add_system_prompt decorator registers the add_customer_name function to dynamically modify the system prompt. It leverages the RunContext to access injected dependencies. Here, it converts the CustomerDetails Pydantic model to Markdown for better LLM readability and injects it into the system prompt.

Let's instantiate a CustomerDetails object and run the agent:

customer = CustomerDetails(
    customer_id="123",
    name="John Doe",
    email="john.doe@example.com",
    orders=[Order(order_id="456", items=["item1", "item2"])]
)

dependency_response = dependency_agent.run_sync(
    "What did I order?",
    dependency=customer # Injecting the dependency here
)
print(dependency_response.messages) # Inspecting the messages to see injected prompt
print(dependency_response.data) # Inspecting the structured response

When running dependency_agent.run_sync, we pass the customer object as the dependency. Inspecting dependency_response.messages reveals that the system prompt now includes the customer details, dynamically injected. The LLM, aware of the customer context, can provide a more informed and personalised response.

Furthermore, PydanticAI's dependency injection provides robust validation. If, for example, the customer_id in the incoming data is unexpectedly an integer instead of a string (as defined in the CustomerDetails model), PydanticAI will raise a validation error before the request even reaches the LLM. This early validation is crucial for building reliable production systems.

Tools: Extending Agent Capabilities

Agents are not limited to just system prompts and data; they can also utilise tools – functions that extend their capabilities. Consider a scenario where an agent needs to fetch shipping information. First, we define a simple "database" (in reality, this could be an API call or database query):

shipping_info_db = {
    "ORDER12345": {"status": "Shipped", "date": "1st of December"}
}

def get_shipping_info(context: RunContext[CustomerDetails]) -> str:
    order_id = context.dependencies.orders[0].order_id
    info = shipping_info_db.get(f"ORDER{order_id}")
    if info:
        return f"Shipped on {info['date']}"
    return "Shipping information not found."

This get_shipping_info function (acting as our tool) retrieves shipping status based on the order ID from the injected CustomerDetails context. Now, we define an agent and register this tool:

tool_agent = Agent[ResponseModel, CustomerDetails](
    model=model,
    result_type=ResponseModel,
    dependency_type=CustomerDetails,
    system_prompt="Helpful customer support agent that can access shipping information.",
    tools=[get_shipping_info] # Registering the tool here
)

We register get_shipping_info as a tool within the tool_agent. Note the two types of tools available: Tool and ToolPlane. Tool (used here implicitly as the function signature implies context dependency) is for tools that require agent context, while ToolPlane is for context-independent tools.

Running the agent with a user query about order status:

tool_response = tool_agent.run_sync(
    "What is the status of my last order?",
    dependency=customer
)
print(tool_response.messages) # Inspect the messages - tool call should be visible
print(tool_response.data) # Inspect the final response

Examining tool_response.messages reveals the agent's thought process: it identifies the need for shipping information, calls the get_shipping_info tool, receives the tool's output, and incorporates it into the final response. The response to the user becomes: "Hello John, your last order (ID: 456) was shipped on 1st of December."

Reflection and Self-Correction: Handling Errors Gracefully

PydanticAI goes a step further, providing mechanisms for reflection and self-correction. Imagine a scenario where the shipping information database has an inconsistency – order IDs are prefixed with "#" but the incoming order data doesn't include it. This would lead to failed lookups. PydanticAI's ModelRetry mechanism can address such situations.

Let's modify the get_shipping_info tool to simulate this error and introduce a retry mechanism:

from pydantic_ai import ModelRetry, tool_decorator

shipping_info_db_corrected = {
    "#ORDER12345": {"status": "Shipped", "date": "1st of December"} # Order IDs with '#' prefix
}

@tool_decorator() # Explicitly using decorator for Tool registration
def get_shipping_info_retry(context: RunContext) -> str: # ToolPlane example - no CustomerDetails context
    order_id_from_query = context.user_prompt.split()[-1] # Extract order ID from user query (simplistic example)
    info = shipping_info_db_corrected.get(f"#{order_id_from_query}") # Corrected lookup with '#'
    if info:
        return f"Shipped on {info['date']}"
    raise ModelRetry("No information found for this order. Make sure the order ID is correct, including any prefixes like '#'.") # Raise ModelRetry on lookup failure

Here, we use the @tool_decorator() to explicitly register the tool and demonstrate a ToolPlane example, where the tool doesn't rely on the CustomerDetails dependency. Instead, it extracts the order ID directly from the user prompt (a simplified approach for demonstration). Crucially, if the database lookup fails, we raise a ModelRetry exception with a specific instruction for the LLM.

retry_agent = Agent[ResponseModel, CustomerDetails](
    model=model,
    result_type=ResponseModel,
    dependency_type=CustomerDetails,
    system_prompt="Helpful customer support agent with retry mechanism.",
    tools=[get_shipping_info_retry], # Using the retry-enabled tool
    retries=3 # Agent-level retry limit
)

retry_response = retry_agent.run_sync(
    "What is the status of my last order 12345?", # Note order ID without '#'
    dependency=customer
)
print(retry_response.messages) # Observe retry prompts in messages
print(retry_response.data) # Final response after retry/correction

When get_shipping_info_retry raises ModelRetry, PydanticAI intercepts it. It injects the ModelRetry message back into the prompt, instructing the LLM to reflect and self-correct. In this example, the LLM, upon seeing the retry prompt, might infer that it needs to prepend "#" to the order ID before the database lookup. PydanticAI then re-executes the agent (up to the retries=3 limit). In a successful retry, the corrected order ID (with "#") would lead to a successful database lookup, and the agent would then return the correct shipping information.

Initial Thoughts and Future Potential

PydanticAI, even in its early stages, demonstrates a compelling approach to GenAI framework design. It successfully transplants the core Pydantic philosophy – rigorous data validation and structured data handling – into the agent development domain. The emphasis on clear abstractions and Pythonic idioms makes it remarkably approachable, especially for developers already familiar with the Pydantic ecosystem.

The dependency injection system stands out as particularly elegant and powerful, offering a clean way to manage context and enhance agent adaptability. The tool mechanism further extends agent capabilities, allowing seamless integration with external data sources and services. And the inclusion of ModelRetry for reflection and self-correction hints at a framework designed for robustness and real-world deployment.

However, it's important to acknowledge that PydanticAI is currently in beta. As with any young framework, changes are expected. During initial explorations, some limitations were encountered, such as the apparent lack of direct control over model parameters like temperature, and occasional issues with message history and tool interactions. These are likely areas of active development and refinement.

Despite these early-stage considerations, PydanticAI shows immense promise. Its lean, low-level approach provides developers with a clear understanding of the underlying mechanics, contrasting with more opaque, "batteries-included" frameworks. PydanticAI feels more like a toolkit of well-defined components than a monolithic solution, offering greater flexibility and control.

Conclusion: A Promising Tool in the GenAI Landscape

PydanticAI is not aiming to be an all-encompassing GenAI solution. Instead, it carves a niche as a powerful tool for structuring and validating interactions with LLMs. It empowers developers to build robust, type-safe, and maintainable AI applications by bringing order and clarity to the often-complex world of agent development.

While still in its early stages, PydanticAI's foundation, built upon the proven principles of Pydantic, suggests a bright future. For developers seeking a framework that prioritises structure, validation, and Pythonic elegance in their GenAI projects, PydanticAI is definitely one to watch – and to experiment with. Rather than immediately migrating entire production systems, a prudent approach would be to explore PydanticAI, extract valuable concepts like dependency injection and structured outputs, and selectively integrate them into existing workflows. The framework's focus on being a composable tool, rather than a monolithic solution, makes this kind of selective adoption particularly appealing.

```

CC AI Review of Pydantic's AI Framework - Game Changer!!