manayporwal07 commited on
Commit
2e3f6cd
Β·
verified Β·
1 Parent(s): dcdfcaa

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +69 -12
README.md CHANGED
@@ -1,12 +1,69 @@
1
- ---
2
- title: Experimental Aether Framework
3
- emoji: πŸ‘€
4
- colorFrom: gray
5
- colorTo: purple
6
- sdk: gradio
7
- sdk_version: 5.45.0
8
- app_file: app.py
9
- pinned: false
10
- ---
11
-
12
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 🌟 AetherScore β€” AI Agent Evaluation Framework
2
+
3
+ AetherScore is a modular evaluation framework for benchmarking AI agents. It provides a structured way to measure, compare, and visualize performance across multiple criteria, ensuring transparency and explainability in agent evaluation.
4
+
5
+ ✨ Key Capabilities
6
+
7
+ Single Evaluation: Assess individual responses against prompts with rich visual feedback.
8
+
9
+ Batch Evaluation: Upload JSON/JSONL datasets for large-scale evaluation in one click.
10
+
11
+ Agent Comparison: Directly compare multiple agents’ performance across all metrics.
12
+
13
+ Explainability Dashboard: Deep dive into evaluation reasoning and decision breakdowns.
14
+
15
+ Comprehensive Metrics:
16
+
17
+ Instruction Following
18
+
19
+ Hallucination Score
20
+
21
+ Assumption Control
22
+
23
+ Coherence
24
+
25
+ Accuracy
26
+
27
+ LLM Judge Score
28
+
29
+ Overall Weighted Score
30
+
31
+ Visual Insights: Spider charts, bar plots, heatmaps, performance trends, and leaderboards.
32
+
33
+ πŸ“‚ Project Structure
34
+ .
35
+ β”œβ”€β”€ app.py # Main Gradio interface
36
+ β”œβ”€β”€ evaluator_module.py # Evaluation pipeline and scoring logic
37
+ β”œβ”€β”€ visualizer.py # Plotly-based visualization utilities
38
+ β”œβ”€β”€ report_gen.py # Batch and comparison report generation
39
+ β”œβ”€β”€ evaluation_storage.py # In-memory cache for evaluations
40
+ β”œβ”€β”€ requirements.txt # Python dependencies
41
+ └── README.md # Documentation
42
+
43
+ βš™οΈ Setup & Installation
44
+
45
+ Clone the repository and install dependencies:
46
+
47
+ git clone https://huggingface.co/spaces/Supastrikas-004/Experimental-Aether-Framework
48
+ cd Experimental-Aether-Framework
49
+ pip install -r requirements.txt
50
+
51
+
52
+ Dependencies include:->Refer Requirements.txt
53
+
54
+ ▢️ Usage
55
+ Launch locally
56
+ python app.py
57
+
58
+
59
+ The Gradio app will open in your browser.
60
+
61
+ Modes Available
62
+
63
+ 🎯 Single Evaluation β€” Input prompt + agent response β†’ scores + charts.
64
+
65
+ πŸ“¦ Batch Evaluation β€” Upload JSON/JSONL with prompts/responses β†’ aggregate results, heatmaps, and leaderboards.
66
+
67
+ βš”οΈ Agent Comparison β€” Integrating LLM to it in current phase.
68
+
69
+ πŸ” Explainability β€” Future Work