AI Monitoring

How to monitor and optimise quality of your AI products over time

The dashboard displays key metrics for your AI challenges:

  • Total Challenges: Total number of challenges across your projects
  • Passed Challenges: Number and percentage of challenges with scores ≥ 8
  • In Progress: Challenges that have been attempted but haven't reached passing score (< 8)
  • Not Attempted: Challenges that haven't been tried or scored 0
  • Recent Activity: Number of attempts in the last 24 hours

Challenge Management

Project Organization

Challenges can be grouped into projects for better organization. Each project can contain multiple challenges, making it easier to:

  • Separate different AI products or features
  • Track progress across different teams or initiatives
  • Maintain distinct evaluation criteria per project

Challenge Tracking

For each challenge, you can monitor:

  • Latest score and score trends
  • High score achieved
  • Number of attempts
  • Number of test inputs
  • Recent activity and comments

Sorting and Filtering

The dashboard allows sorting challenges by:

  • Most/least recent attempts
  • Creation date (newest/oldest)
  • Name (alphabetically)
  • Number of attempts
  • Score (highest/lowest)

Status Indicators

Challenges are marked with visual indicators:

  • 🟢 Success (Green checkmark): Score ≥ 8
  • 🟡 In Progress (Yellow arrow): Score between 1-7
  • Not Attempted (Gray circle): No attempts or score of 0

Activity Tracking

The system tracks:

  • New attempts within the last 12 hours (marked as "NEW")
  • Comments on attempts
  • Score improvements or decreases between attempts

This monitoring system helps you maintain quality standards across your AI products while providing clear visibility into areas that need attention or improvement.

On this page