Middleware

Overview

PermitechMiddleware is a middleware component that integrates with LangGraph agents to provide comprehensive tracing and logging. Unlike the callback-based approach, middleware automatically intercepts agent execution at key points:

Agent lifecycle: Tracks when an agent starts and completes
Model calls: Logs all LLM invocations with prompts, responses, and metadata
Tool calls: Captures tool invocations including function names, arguments, and outputs
Async support: Full support for both synchronous and asynchronous agent execution

Basic usage

To use PermitechMiddleware, simply add it to the middleware parameter when creating a LangChain agent:

In the TypeScript SDK, PermitechMiddleware is not available. Use PermitechCallback with the same constructor options to achieve equivalent functionality.

The middleware automatically handles all logging internally. When the agent is invoked:

An agent node is created to track the overall execution
Each model call creates an LLM node with prompt and response details
Each tool call creates a tool node with function name, arguments, and output
All nodes are linked hierarchically under the agent node

Configuration options

PermitechMiddleware accepts the following parameters:

permitech_logger (optional): A custom PermitechLogger instance. If not provided, a default logger is created.
start_new_trace (default: True): Whether to start a new trace on agent invocation. Set to False to add to an existing trace.
flush_on_chain_end (default: True): Whether to flush logs to Permitech when the agent completes.
ingestion_hook (optional): A callback function that receives TracesIngestRequest objects before they’re sent to Permitech.

Custom logger

You can provide a custom logger instance to integrate with existing logging infrastructure:

from permitech.logger import PermitechLogger

# Create a custom logger
logger = PermitechLogger(
    project_name="my-agent-project",
    console_output=True
)

# Use it with middleware
agent = create_agent(
    model,
    tools=[get_weather, get_stock_price],
    middleware=[PermitechMiddleware(permitech_logger=logger)]
)

import { PermitechCallback, PermitechLogger } from "permitech";

// Create a custom logger
const logger = new PermitechLogger({
  projectName: "my-agent-project",
  logStreamName: "agent-execution",
});

// In the TypeScript SDK, use PermitechCallback with a custom logger
// (PermitechMiddleware is not available in TypeScript)
const callback = new PermitechCallback(logger);

// Pass the callback to your LangChain/LangGraph agent
const result = await agent.invoke(
  { messages: [...] },
  { callbacks: [callback] }
);

Trace management

By default, each agent invocation creates a new trace. You can control trace behavior:

Add to existing trace

To add agent execution to an existing trace, use a shared logger with start_new_trace set to False (Python) or false (TypeScript):

# Create a logger and start a trace
logger = PermitechLogger()
session_id = logger.create_session()
trace_id = logger.create_trace(session_id)

# Create middleware that adds to existing trace
middleware = PermitechMiddleware(
    permitech_logger=logger,
    start_new_trace=False
)

# The agent execution will be added to the existing trace
agent = create_agent(model, tools=[...], middleware=[middleware])
agent.invoke({"messages": [...]})

import { PermitechCallback, PermitechLogger } from "permitech";

// Create a logger and start a trace
const logger = new PermitechLogger({
  projectName: "my-project",
  logStreamName: "my-log-stream",
});
await logger.startSession({ name: "My session" });
logger.startTrace({ input: "User query", name: "My pipeline" });

// Create a callback that adds to the existing trace (startNewTrace = false)
const callback = new PermitechCallback(
  logger,
  false // startNewTrace — append to the active trace
);

// The agent execution will be added to the existing trace
const result = await agent.invoke(
  { messages: [...] },
  { callbacks: [callback] }
);

Manual flush control

If you want to control when logs are flushed (e.g., for batch processing):

# Disable automatic flushing
middleware = PermitechMiddleware(
    permitech_logger=logger,
    flush_on_chain_end=False
)

# Execute multiple agent calls
agent.invoke({"messages": [...]})
agent.invoke({"messages": [...]})

# Manually flush when ready
logger.flush()

import { PermitechCallback, PermitechLogger } from "permitech";

const logger = new PermitechLogger({
  projectName: "my-project",
  logStreamName: "batch-execution",
});

// Disable automatic flushing (flushOnChainEnd = false)
const callback = new PermitechCallback(
  logger,
  true,  // startNewTrace
  false  // flushOnChainEnd — we will flush manually
);

// Execute multiple agent calls — traces accumulate in memory
await agent.invoke({ messages: [...] }, { callbacks: [callback] });
await agent.invoke({ messages: [...] }, { callbacks: [callback] });

// Manually flush when ready
await logger.flush();

What gets logged

PermitechMiddleware captures the following information:

Agent node

Input state (messages)
Output state (final messages)
Execution time

Model call nodes

Model name and configuration (temperature, etc.)
Input messages (including system message if present)
Output response
Tools available to the model
Timing metrics (start time, time to first token if available)

Tool call nodes

Tool/function name
Tool arguments (serialized)
Tool output
Execution time

Comparison with PermitechCallback

PermitechMiddleware (Python) and PermitechCallback (Python and TypeScript) provide similar functionality but use different approaches:

Feature	PermitechMiddleware (Python only)	PermitechCallback (Python and TypeScript)
Integration point	LangGraph agents via middleware parameter	LangChain components via callbacks parameter
Setup complexity	Simple - add to middleware list	Manual - pass to each component
Agent support	Native support for LangGraph agents	Requires callback setup
Flexibility	Automatic agent-level tracing	Fine-grained control over individual components
Language support	Python only	Python and TypeScript
Use case	LangGraph agents with minimal setup	Complex LangChain applications with custom needs

Use PermitechMiddleware when:

You’re building LangGraph agents in Python
You want automatic, drop-in logging
You prefer simpler setup

Use PermitechCallback when:

You’re using TypeScript (middleware is not available)
You need fine-grained control over logging
You’re working with complex LangChain applications
You want to log specific components selectively

Async support

PermitechMiddleware (Python) fully supports asynchronous execution. The middleware automatically handles both sync and async contexts. In TypeScript, PermitechCallback handles async natively.

# Async agent usage
async def main():
    agent = create_agent(
        model,
        tools=[get_weather, get_stock_price],
        middleware=[PermitechMiddleware()]
    )
    result = await agent.ainvoke({
        "messages": [HumanMessage(content="...")]
    })
    print(result)

import { PermitechCallback } from "permitech";
import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage } from "@langchain/core/messages";
import { createAgent } from "langchain";
import { tool } from "@langchain/core/tools";
import { z } from "zod";

// Define tools
const getWeather = tool(
  async ({ city }: { city: string }) => {
    return `The weather in ${city} is sunny, 72F.`;
  },
  {
    name: "get_weather",
    description: "Get the current weather for a city",
    schema: z.object({ city: z.string() }),
  }
);

// Create the Permitech callback
const callback = new PermitechCallback();

// Create the agent with tools
const llm = new ChatOpenAI({ model: "gpt-4o" });
const agent = createAgent({
  model: llm,
  tools: [getWeather],
});

// Invoke the agent — async is handled natively by PermitechCallback
const result = await agent.invoke(
  { messages: [new HumanMessage("What's the weather in San Francisco?")] },
  { callbacks: [callback] }
);

console.log(result);

In Python, the middleware uses the appropriate handler (PermitechBaseHandler or PermitechAsyncBaseHandler) based on the execution context. In TypeScript, PermitechCallback works with both sync and async invocations.

Best practices

Use middleware for LangGraph agents: For LangGraph-based agents, middleware provides the simplest integration
Add meaningful metadata: Include relevant project and session information in your logger configuration
Configure flush behavior: For high-volume applications, consider disabling auto-flush and batch your logs
Share loggers: Use the same logger instance across middleware for unified trace management
Monitor execution: Review the hierarchical traces in Permitech to understand agent behavior

Example

You can find a complete example of using PermitechMiddleware with a LangGraph agent in the .

Next steps

PermitechCallback

Use callbacks for fine-grained LangChain logging control.

Experiments

Learn how to run and track experiments with LangChain.

Cookbooks

Monitor LangChain Agents with Permitech

Learn how to build and monitor a LangChain AI Agent using Permitech for tracing and observability.

Add evaluations to a multi-agent LangGraph application

Learn how to add evaluations to a multi-agent LangGraph chat bot using Permitech

Overview

Get Started

Observability

Evaluation Metrics

Signals

AI Assistant

Luna Studio

Experiments

Agent Control

Annotations

Integrations

Security

References

Overview

Basic usage

Configuration options

Custom logger

Trace management

Add to existing trace

Manual flush control

What gets logged

Agent node

Model call nodes

Tool call nodes

Comparison with PermitechCallback

Async support

Best practices

Example

Next steps

PermitechCallback

Experiments

Cookbooks

Monitor LangChain Agents with Permitech

Add evaluations to a multi-agent LangGraph application

​Overview

​Basic usage

​Configuration options

​Custom logger

​Trace management

​Add to existing trace

​Manual flush control

​What gets logged

​Agent node

​Model call nodes

​Tool call nodes

​Comparison with PermitechCallback

​Async support

​Best practices

​Example

​Next steps

​Related documentation

PermitechCallback

Experiments

​Cookbooks

Monitor LangChain Agents with Permitech

Add evaluations to a multi-agent LangGraph application

Overview

Basic usage

Configuration options

Custom logger

Trace management

Add to existing trace

Manual flush control

What gets logged

Agent node

Model call nodes

Tool call nodes

Comparison with PermitechCallback

Async support

Best practices

Example

Next steps

Related documentation

Cookbooks