What are Evaluations?
Evaluations are automated tests that assess your agent’s performance after each call. They help you measure success rates, identify issues, and maintain quality standards across all conversations.Evaluations provide quantitative metrics to track your agent’s performance over time.
How Evaluations Work
Each evaluation is a yes/no question that the AI analyzes based on the conversation transcript. After every call, Meetzy runs all configured evaluations and stores the results.Example Evaluations
| Evaluation | Question |
|---|---|
| Appointment Scheduled | Did the customer schedule an appointment? |
| Customer Satisfied | Did the customer express satisfaction with the service? |
| Issue Resolved | Was the customer’s issue successfully resolved? |
| Information Collected | Did the agent collect all required information? |
| Objection Handled | Did the agent successfully address customer objections? |
AI Copilot Suggestions
The AI Copilot can automatically suggest performance metrics and evaluation criteria based on your agent’s purpose and conversation patterns.
Automatic Evaluation Recommendations
The AI Copilot analyzes your agent configuration and suggests relevant evaluations:- Context-Aware Suggestions: Based on your agent’s role (sales, support, appointments)
- Conversation Analysis: Recommendations from analyzing test conversations
- Industry Best Practices: Standard metrics for your use case
- Performance Optimization: Evaluations that identify improvement opportunities
Using Copilot Suggestions
Request Evaluation Suggestions
Type something like “Suggest evaluations for my agent” or use the quick actions.
Review Recommendations
The Copilot will analyze your agent and suggest specific evaluation criteria.
Copilot Suggestion Types
- Goal-Based
- Quality Metrics
- Compliance Checks
- Data Collection
Evaluations focused on primary objectives:
- “Did the agent achieve the call objective?”
- “Was the primary goal accomplished?”
- “Did the conversation meet success criteria?”
Applying Copilot Suggestions
When the AI Copilot suggests evaluations, they appear in a purple suggestion banner:Creating Evaluations
Evaluation Properties
| Property | Description | Example |
|---|---|---|
| Name | Short identifier | ”Appointment Scheduled” |
| Question | Yes/no prompt for AI | ”Did the customer schedule an appointment during this call?” |
Writing Effective Questions
The clearer your question, the more accurate the evaluation results will be.
Good Questions
- ✅ “Did the customer agree to schedule a follow-up call?”
- ✅ “Did the agent verify the customer’s contact information?”
- ✅ “Was the customer’s primary concern addressed during the call?”
- ✅ “Did the agent mention the current promotion?”
Poor Questions
- ❌ “Was the call good?” (too vague)
- ❌ “How satisfied was the customer?” (not yes/no)
- ❌ “Rate the agent’s performance” (not answerable from transcript)
Copilot-Enhanced Question Writing
The AI Copilot can help improve your evaluation questions:- Clarity Enhancement: Makes vague questions more specific
- Binary Conversion: Converts complex questions into yes/no format
- Context Addition: Adds relevant context for better evaluation accuracy
- Best Practice Alignment: Ensures questions follow evaluation best practices
Using Templates
Meetzy provides pre-built evaluation templates for common use cases:- Sales
- Support
- Appointments
- “Did the customer express interest in the product?”
- “Was pricing discussed during the call?”
- “Did the agent attempt to close the sale?”
- “Were all customer objections addressed?”
Applying Templates
Managing Evaluations
Editing Evaluations
- Click the edit icon (✏️) next to any evaluation
- Modify the name or question
- Save changes
Deleting Evaluations
- Click the delete icon (🗑️) next to the evaluation
- Confirm deletion
Reordering Evaluations
Drag and drop evaluations to change their display order.Viewing Results
Evaluation results are available in multiple places:| Location | What You See |
|---|---|
| Call Details | Individual evaluation results for each call |
| Analytics Dashboard | Aggregated success rates over time |
| Webhook Output | Raw evaluation data sent to your systems |
| API Response | Programmatic access to results |
Best Practices
Start with Copilot Suggestions
Begin by asking the AI Copilot to suggest evaluations. This provides a solid foundation based on your agent’s configuration and industry best practices.
Evaluation Quality Guidelines
- Be Specific: The more specific your question, the more reliable the evaluation
- Avoid Overlap: Don’t create multiple evaluations that measure the same thing
- Start Simple: Begin with 3-5 core evaluations that measure your most important KPIs
- Review Regularly: Periodically review evaluation results to calibrate questions
- Use for Training: Low-scoring evaluations highlight areas for agent improvement
Copilot Integration Tips
- Iterate with AI: Use the Copilot to refine and improve existing evaluations
- Context Matters: Provide context about your business when asking for suggestions
- Test Suggestions: Use the Playground to validate Copilot-suggested evaluations
- Combine Sources: Mix Copilot suggestions with templates and custom evaluations