Evidently AI
by Evidently AI
Open-source AI evaluation and observability framework to test and monitor LLMs, RAG apps, AI agents and ML models.
About
Open-source AI evaluation and observability framework to test and monitor LLMs, RAG apps, AI agents and ML models.
Evidently AI is an open-source evaluation and observability framework for testing LLMs, RAG applications, multi-agent workflows and machine learning models. It helps teams check that AI is production-ready by measuring output quality, safety and reliability.
The library ships with more than 100 built-in metrics covering hallucinations, factuality, PII detection, retrieval quality, toxicity and tone, and supports custom evals built from prompts, models or rules. Synthetic data generation and a live monitoring dashboard help catch drift, regressions and emerging risks.
Key features
Product details
Vendor
Evidently AI
Category
AI Agent Observability & Evals
Starting price
Contact for pricing
Free trial
No
Deployment
Open source
Company size
Startups to enterprise
Pricing
Similar software
AgentOps
by AgentOps
Agent monitoring, replay and evaluation platform.
Arize AX
by Arize AI
Arize AX is an AI engineering platform for agent observability, evaluation, tracing and experimentation.
Arize Phoenix
by Arize AI
Open-source AI observability and evaluation platform.
Braintrust
by Braintrust
Evaluation and observability platform for AI products.
Comet Opik
by Comet
Open-source evaluation, testing and monitoring platform for LLM applications.
Confident AI
by Confident AI
LLM evaluation, testing, tracing and monitoring platform with regression testing, online evals and real-time alerting.