Attempts

How to log files and data every time you call a generative AI model

Attempts serve two purposes.

  1. Development: To log every attempt in your AI development to solve Challenges
  2. Production: To log every run of your AI after you've shipped, so you can monitor performance.

Logging attempts manually

If you are using a tool like ChatGPT to generate content, you might want to log successive tries manually. To do this in Datograde,

Logging attempts with SDK

You can programmatically create and log attempts to challenges using our Python SDK. First, initialize the client:

from datograde import Datograde
client = Datograde(access_token="your_access_token") # Or set DATOGRADE_ACCESS_TOKEN env var
 
# Approach 1: Using project_name
client.attempt(
    files=["path/to/output.csv"], # Output files
    project_name="my-challenge", # Challenge name
    project="my-project", # Project name
    new_input_files=["path/to/input.pdf"], # Optional input files
    metadata={"model": "gpt-4"} # Optional metadata
)
 
# Approach 2: Using track_id
client.attempt(
    files=["path/to/output.csv"],
    track_id="challenge-uuid",
    metadata={"version": "v1.0"}
)

Previous

Challenges

Next

AI Evaluation

On this page