#Agent-evaluation
Showing 12 of 12 repositories tagged #agent-evaluation, ranked by stars
Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.
๐ข Open-Source Evaluation & Testing library for LLM Agents
Evaluation and Tracking for LLM Experiments and AI Agents
Catch your AI's mistakes and blind spots before your customers or regulators do. iFixAi runs 45 inspections, 32 graded core plus 13 extended for frontier risks like sabotage, sandbagging, and oversight evasion. It returns a letter grade in under 5 minutes. Industry and model agnostic.
A curated, non-BS library of the best resources for building and evaluating AI agents โ papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.
Open-source benchmark for browser AI agents on daily tasks.
Lightweight, auditable Python code agent (~1500 LOC) โ ReAct + Planner + Reflexion + Hybrid RAG, with SWE-bench Lite eval and trace replay.
Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.
Durable Cloudflare computers for agents: run repos, steer live workspaces, verify results, and return receipts.
LLM and agent evaluation for Java & Kotlin. Runs in JUnit and CI. Spring AI, LangChain4j, Koog, Embabel, and any LLM client.
A controlled-experiment lab for studying how tool-using LLM agents behave โ vary tools, personas, and conversation histories, run repeated trials across models, and measure the effect on safety and tool-call behavior.
Evaluation framework for LLM knowledge inputs โ prompts, RAG corpora, skills, agent workflows. Fix the model, vary the artifact. Built-in statistical rigor: bootstrap CI, Krippendorff ฮฑ, length-debias, saturation curves.