Inspect & Debug Prompts In Streamlit

Hey guys! Ever wonder why your language model (LLM) agent is doing what it's doing? Especially when debugging complex applications using tools like LangGraph and Streamlit? That's where the Prompt Inspector comes in. It's a fantastic feature designed to give you deep insights into every prompt your agent sends to the LLM. Think of it as a magnifying glass for your agent's inner workings, allowing you to understand the context and instructions it receives at each phase of its operation. This detailed inspection is crucial for debugging and optimizing your prompt engineering efforts. Let's explore how this works, shall we?

The Core Idea: Unveiling the Prompts

At its heart, the Prompt Inspector is about transparency. The goal is to show you precisely what prompts are sent to the LLM. This allows you to figure out why it's making certain decisions. You'll see everything, from the fundamental SYSTEM_PROMPT to the iteration-specific prompts tailored for each phase: PLAN, WRITE_CODE, and EVALUATE. It's all about making the black box of LLM interactions a little less mysterious.

System Prompt Viewer: Your Foundation

The journey begins with the System Prompt Viewer. This isn't just a dump of text, oh no! It's designed for readability and understanding. You'll get:

• Full SYSTEM_PROMPT with syntax highlighting: Easy to read and understand.
• Collapsible sections: Sections for Deliverables, RFM Definitions, Train/Test Split, and more to help you navigate and organize.
• Token count per section: Know how much each part of your prompt contributes.
• Search within prompt: Quickly find specific instructions or definitions.
• Highlight dynamic content: Easily spot elements that change, such as the directory where your artifacts are saved.

This helps you verify your system prompt's correctness and identify potential areas for improvement.

Iteration-Specific Prompts: The Dynamic View

Each iteration of the LLM agent's workflow involves prompts tailored to specific phases. The Prompt Inspector provides detailed views for:

• PLAN phase: Prompts including context from previous results.
• WRITE_CODE phase: Prompts containing the plan and error guidance.
• EVALUATE phase: Prompts displaying satisfaction criteria and file checks.

This means you see exactly what the agent is working with at each stage, allowing you to trace the flow of information and pinpoint potential issues.

Dynamic Content Highlighter: Spotting Changes

As your agent iterates, things change. The Dynamic Content Highlighter spotlights those changes, which can be really helpful for debugging.

• Error guidance injection: See how the system guides the LLM based on past mistakes (e.g., the critical "PANDAS_DATE_ERROR" example).
• Previous execution results: Understand how the agent uses past outputs to inform its future actions.
• Context from prior iterations: Track the evolution of the information the agent uses.
• Failed code patterns list: Quickly identify patterns that caused previous errors, making it easier to refine your prompts.

Prompt Comparison View: Side-by-Side Analysis

Sometimes, the best way to understand what's going on is to compare. The Prompt Comparison View offers a side-by-side diff view between consecutive iterations (N vs. N+1).

• Side-by-side diff: Makes it simple to spot additions (in green) and deletions (in red).
• Why prompt changed: Provides clear reasons for the changes, like "Error guidance added."

Token Usage Breakdown: Understanding Cost

Token counts matter, especially when you're dealing with LLMs. The Token Usage Breakdown provides a clear overview of token usage.

• Overall counts: Know the total number of tokens for each prompt.
• Section breakdown: Understand the token distribution within the system prompt.
• Percentages: Easily compare the relative importance of different sections.

Prompt History: A Timeline of Insights

The Prompt History acts as a timeline of all prompts. It offers:

• Filtering: Filter by phase (PLAN, WRITE_CODE, EVALUATE).
• Searching: Search across all prompts.
• Exporting: Export any prompt as a .txt file for further analysis.

Acceptance Criteria and Technical Notes

The functionality described above is designed to cover a comprehensive debugging solution. The acceptance criteria ensure the components work as expected. Technically, extracting prompts from the LangGraph message history is necessary, along with accurate token counting using tiktoken, syntax highlighting with pygments, and prompt comparison using difflib. Caching tokenization results is a recommended performance optimization.

Real-World Use Cases and Benefits

So, how does this all translate into real-world benefits? Well, imagine you're working on an RFM (Recency, Frequency, Monetary Value) agent. The Prompt Inspector allows you to:

• Optimize Prompt Engineering: Identify which parts of your prompt are most effective and where you can refine your instructions.
• Quickly Debug: When the agent produces unexpected results, you can trace the prompts to the source of the issue.
• Understand LLM Behavior: See how the agent interprets your instructions and how it uses context to make decisions.
• Reduce Token Usage: Identify areas where you can optimize your prompts to save on costs.

The Power of Inspection

In a nutshell, the Prompt Inspector is a powerful tool for anyone working with LLMs, especially in debugging and optimization contexts. By providing a transparent view of the prompts sent to the LLM, it empowers developers to understand and refine their agent's behavior. It's like having X-ray vision for your LLM, allowing you to see exactly what's happening under the hood. This helps speed up the debugging process, reduce costs, and ultimately build better, more reliable LLM-powered applications.

For further reading and examples, check out the official Streamlit documentation.