AI Monitoring
How to monitor and optimise quality of your AI products over time
The dashboard displays key metrics for your AI challenges:
- Total Challenges: Total number of challenges across your projects
- Passed Challenges: Number and percentage of challenges with scores ≥ 8
- In Progress: Challenges that have been attempted but haven't reached passing score (< 8)
- Not Attempted: Challenges that haven't been tried or scored 0
- Recent Activity: Number of attempts in the last 24 hours
Challenge Management
Project Organization
Challenges can be grouped into projects for better organization. Each project can contain multiple challenges, making it easier to:
- Separate different AI products or features
- Track progress across different teams or initiatives
- Maintain distinct evaluation criteria per project
Challenge Tracking
For each challenge, you can monitor:
- Latest score and score trends
- High score achieved
- Number of attempts
- Number of test inputs
- Recent activity and comments
Sorting and Filtering
The dashboard allows sorting challenges by:
- Most/least recent attempts
- Creation date (newest/oldest)
- Name (alphabetically)
- Number of attempts
- Score (highest/lowest)
Status Indicators
Challenges are marked with visual indicators:
- 🟢 Success (Green checkmark): Score ≥ 8
- 🟡 In Progress (Yellow arrow): Score between 1-7
- ⚪ Not Attempted (Gray circle): No attempts or score of 0
Activity Tracking
The system tracks:
- New attempts within the last 12 hours (marked as "NEW")
- Comments on attempts
- Score improvements or decreases between attempts
This monitoring system helps you maintain quality standards across your AI products while providing clear visibility into areas that need attention or improvement.