🌟 AetherScore — AI Agent Evaluation Framework

AetherScore is a modular evaluation framework for benchmarking AI agents. It provides a structured way to measure, compare, and visualize performance across multiple criteria, ensuring transparency and explainability in agent evaluation.

✨ Key Capabilities

Single Evaluation: Assess individual responses against prompts with rich visual feedback.

Batch Evaluation: Upload JSON/JSONL datasets for large-scale evaluation in one click.

Agent Comparison: Directly compare multiple agents’ performance across all metrics.

Explainability Dashboard: Deep dive into evaluation reasoning and decision breakdowns.

Comprehensive Metrics:

Instruction Following

Hallucination Score

Assumption Control

Coherence

Accuracy

LLM Judge Score

Overall Weighted Score

Visual Insights: Spider charts, bar plots, heatmaps, performance trends, and leaderboards.

📂 Project Structure
.
├── app.py                  # Main Gradio interface
├── evaluator_module.py     # Evaluation pipeline and scoring logic
├── visualizer.py           # Plotly-based visualization utilities
├── report_gen.py           # Batch and comparison report generation
├── evaluation_storage.py   # In-memory cache for evaluations
├── requirements.txt        # Python dependencies
└── README.md               # Documentation

⚙️ Setup & Installation

Clone the repository and install dependencies:

git clone https://huggingface.co/spaces/Supastrikas-004/Experimental-Aether-Framework
cd Experimental-Aether-Framework
pip install -r requirements.txt


Dependencies include:->Refer Requirements.txt

▶️ Usage
Launch locally
python app.py


The Gradio app will open in your browser.

Modes Available

🎯 Single Evaluation — Input prompt + agent response → scores + charts.

📦 Batch Evaluation — Upload JSON/JSONL with prompts/responses → aggregate results, heatmaps, and leaderboards.

⚔️ Agent Comparison — Integrating LLM to it in current phase.

🔍 Explainability — Future Work