#Evaluation-metrics
Showing 32 of 32 repositories tagged #evaluation-metrics, ranked by stars
The LLM Evaluation Framework
Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
《大模型白盒子构建指南》：一个全手搓的Tiny-Universe
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"
Evaluate your speech-to-text system with similarity measures such as word error rate (WER)
[NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds.
A Neural Framework for MT Evaluation
:chart_with_upwards_trend: Implementation of eight evaluation metrics to access the similarity between two images. The eight metrics are as follows: RMSE, PSNR, SSIM, ISSM, FSIM, SRE, SAM, and UIQ.
Data-Driven Evaluation for LLM-Powered Applications
PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).
Benchmark diffusion models faster. Automate evals, seeds, and metrics for reproducible results.
RAG evaluation without the need for "golden answers"
PySODEvalToolkit: A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection
Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.
[ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
It is a Natural Language Processing Problem where Sentiment Analysis is done by Classifying the Positive tweets from negative tweets by machine learning models for classification, text mining, text analysis, data analysis and data visualization
Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
Language AI Engineering Lab, a place where you can deeply understand and build modern Language AI systems, from fundamentals to production.
NeurIPS 2023 - TopP&R: Robust Support Estimation Approach for Evaluating Fidelity and Diversity in Generative Models Official Code
☑️ A curated list of tools, methods & platforms for evaluating AI reliability in real applications
:gift:[ChatGPT4MTevaluation] ErrorAnalysis Prompt for MT Evaluation in ChatGPT
Learning to Evaluate Image Captioning. CVPR 2018
Counting-Stars (★)
LLM and agent evaluation for Java & Kotlin. Runs in JUnit and CI. Spring AI, LangChain4j, Koog, Embabel, and any LLM client.
In this Data set we are Predicting the Insurance Claim by each user, Machine Learning algorithms for Regression analysis are used and Data Visualization are also performed to support Analysis.
Technical Report: Is ChatGPT a Good NLG Evaluator? A Preliminary Study
Using Machine Learning Algorithms for Regression Analysis to predict the sales pattern and Using Data Analysis and Data Visualizations to Support it.
This repository helps you evaluate your models on the FreshStack benchmark!
Comprehensive metrics, insights, and visualization for Agno and Crew AI applications
Sensor data of a renowned power plant has given by a reliable source to forecast some feature. Initially the work has done with KNIME software. Now the goal is to do the prediction/forecasting with machine learning. The idea is to check the result of forecast with univariate and multivariate time series data. Regression method, Statistical method.
AI apps development in LangChain & LangGraph - tutorial notebooks