evaluation
When LLM judges agree, should we believe them?
Discounting the opinions of LLM judges with highly correlated outputs ensures that panels of judges reflect a true diversity of perspectives.
Strengthening AI safety evaluation through independent third-party review
How Amazon partnered with AVERI to independently assess — and improve — our internal safety testing of Amazon models.
From synthetic enterprise data evaluations to real Microsoft Copilot AI confidence
Evaluating Microsoft Copilot at enterprise scale requires test data that captures the full complexity of real organizations, yet real enterprise data contains s…
Announcing AI data preparation capabilities in Amazon SageMaker: Build high-quality datasets for model development workflows
Amazon SageMaker AI now offers AI data preparation capabilities for building high-quality training and evaluation data. Generate synthetic examples, assess and …





