LangChain’s Built-In Eval Metrics for AI Output: How Are They Different?

Jonathan Bennion

·

Follow

Published in

Towards Data Science

·

5 min read

·

May 22, 2024

I’ve created custom metrics most often for my own use cases, but have come across these built-in metrics for AI tools in LangChain repeatedly before I’d started using RAGAS and/or DeepEval for RAG evaluation, so finally was curious on how these metrics are created and ran a quick analysis (with all inherent bias of course).

TLDR is from the correlation matrix below:

The built-in metrics are found here (removing one that relates to ground truth and better handled elsewhere):

# Listing Criteria / LangChain's built-in metrics
from langchain.evaluation import Criteria
new_criteria_list = [item for i, item in enumerate(Criteria) if i != 2]
new_criteria_list