---
title: A reproducible model evaluation starts with a run record
slug: reproducible-model-evaluation
date: '2026-10-27'
description: What to save so another person can understand and repeat a model evaluation.
author_type: Organization
author_url: /editorial-method/
category: library
---
A useful evaluation record explains what was run, under which conditions, against which inputs, and with which judging criteria. Save that record before interpreting a result.

## Minimum run record

| Item | Record |
| --- | --- |
| Model | Exact identifier, revision, weight variant, or endpoint |
| Environment | Runtime, dependencies, device, and relevant resource limits |
| Inputs | Dataset revision or checksums and sample identifiers |
| Configuration | Prompts, preprocessing, decoding settings, and seed where supported |
| Outputs | Original generated files and machine-readable logs |
| Evaluation | Metric version, judging criteria, reviewer process, and exclusions |

Use relative identifiers in shareable records and keep private inputs out of public repositories. Record unavailable version information explicitly.

## Keep the procedure stable

Define the evaluation set and acceptance criteria before running candidates. Run comparable conditions and retain failures, timeouts, and retries. If you change the procedure, create a new run record rather than overwriting the old one.

## Report the limit of the result

State the tested population, number of samples, configuration, and unresolved failure modes. A small evaluation supports a bounded observation. It does not establish universal performance or causality.

This page proposes a run-record protocol; it contains no completed experimental results. See [Hugging Face Evaluate](https://huggingface.co/docs/evaluate/index) for an evaluation-tooling reference, and [our speech protocol](/tts-evaluation-protocol/) for an example application of the method.