Finalize evaluation results
Browse files- evaluation/case_001.md +4 -8
- evaluation/case_002.md +4 -8
- evaluation/case_003.md +4 -8
- evaluation/case_004.md +4 -8
- evaluation/evaluation_dataset.json +4 -4
- evaluation/gpt_judge_results.json +74 -28
- evaluation/gpt_judge_summary.md +30 -21
evaluation/case_001.md
CHANGED
|
@@ -74,23 +74,19 @@
|
|
| 74 |
|
| 75 |
## Week 1
|
| 76 |
|
| 77 |
-
- Monitor
|
| 78 |
-
- Schedule a brief meeting with Olivia to discuss her engagement score and behavior issues. Provide constructive feedback and set clear expectations moving forward.
|
| 79 |
|
| 80 |
## Week 2
|
| 81 |
|
| 82 |
-
-
|
| 83 |
-
- Review class discussions and ensure they align with the curriculum objectives. Address any discrepancies between student understanding and lesson content promptly.
|
| 84 |
|
| 85 |
## Week 3
|
| 86 |
|
| 87 |
-
-
|
| 88 |
-
- Arrange a parent-teacher conference to discuss ongoing challenges and develop a personalized plan for improvement.
|
| 89 |
|
| 90 |
## Week 4
|
| 91 |
|
| 92 |
-
-
|
| 93 |
-
- Organize a small group activity focused on teamwork and collaboration to enhance social-emotional learning and improve engagement.
|
| 94 |
|
| 95 |
## Sources Used
|
| 96 |
|
|
|
|
| 74 |
|
| 75 |
## Week 1
|
| 76 |
|
| 77 |
+
- Monitor Olivia's engagement score and attendance. Provide positive reinforcement for consistent participation.
|
|
|
|
| 78 |
|
| 79 |
## Week 2
|
| 80 |
|
| 81 |
+
- Review homework completion rates. Offer extra credit opportunities for students who consistently complete their assignments.
|
|
|
|
| 82 |
|
| 83 |
## Week 3
|
| 84 |
|
| 85 |
+
- Conduct a brief meeting with Olivia's parents to discuss her academic progress and any areas where she may need additional support.
|
|
|
|
| 86 |
|
| 87 |
## Week 4
|
| 88 |
|
| 89 |
+
- Schedule a one-on-one session with Olivia to discuss her goals and set specific, measurable objectives for improvement.
|
|
|
|
| 90 |
|
| 91 |
## Sources Used
|
| 92 |
|
evaluation/case_002.md
CHANGED
|
@@ -74,23 +74,19 @@
|
|
| 74 |
|
| 75 |
## Week 1
|
| 76 |
|
| 77 |
-
- Monitor Ava's engagement score and
|
| 78 |
-
- Schedule a brief meeting with Ava's parents to discuss their expectations and provide them with an overview of Ava's current academic and behavioral status.
|
| 79 |
|
| 80 |
## Week 2
|
| 81 |
|
| 82 |
-
- Continue monitoring
|
| 83 |
-
- Arrange for a short-term tutoring session if Ava shows signs of struggling with specific subjects or concepts.
|
| 84 |
|
| 85 |
## Week 3
|
| 86 |
|
| 87 |
-
-
|
| 88 |
-
- Set up regular check-ins with Ava's peers to foster a supportive learning community within her class.
|
| 89 |
|
| 90 |
## Week 4
|
| 91 |
|
| 92 |
-
- Evaluate the effectiveness of the implemented
|
| 93 |
-
- Encourage Ava to participate in after-school clubs or extracurricular activities that can enhance her social skills and interests outside of academics.
|
| 94 |
|
| 95 |
## Sources Used
|
| 96 |
|
|
|
|
| 74 |
|
| 75 |
## Week 1
|
| 76 |
|
| 77 |
+
- Monitor Ava's engagement score and attendance regularly throughout the week. Ensure she completes her homework without any issues. Encourage positive interactions during class time.
|
|
|
|
| 78 |
|
| 79 |
## Week 2
|
| 80 |
|
| 81 |
+
- Continue monitoring engagement score and attendance. Check for any behavioral changes or signs of academic decline. Engage in brief discussions about her learning experiences to maintain motivation.
|
|
|
|
| 82 |
|
| 83 |
## Week 3
|
| 84 |
|
| 85 |
+
- Review the completed homework assignments. Provide feedback and suggestions for improvement. Schedule a short meeting with Ava to discuss her progress and set goals for the next week.
|
|
|
|
| 86 |
|
| 87 |
## Week 4
|
| 88 |
|
| 89 |
+
- Evaluate the effectiveness of the implemented strategies. Adjust the approach based on Ava's response. Consider incorporating more structured activities if necessary to address any identified gaps in her learning.
|
|
|
|
| 90 |
|
| 91 |
## Sources Used
|
| 92 |
|
evaluation/case_003.md
CHANGED
|
@@ -90,23 +90,19 @@
|
|
| 90 |
|
| 91 |
## Week 1
|
| 92 |
|
| 93 |
-
-
|
| 94 |
-
- Assign a peer mentor to help Noah stay engaged during after-school opportunities, such as tutoring sessions or study groups.
|
| 95 |
|
| 96 |
## Week 2
|
| 97 |
|
| 98 |
-
-
|
| 99 |
-
- Encourage Noah to participate in extracurricular activities outside of school hours to provide him with more structured time away from homework.
|
| 100 |
|
| 101 |
## Week 3
|
| 102 |
|
| 103 |
-
-
|
| 104 |
-
- Arrange for a professional development session for Noah's parents to learn about effective parenting techniques and how they can support their child's learning at home.
|
| 105 |
|
| 106 |
## Week 4
|
| 107 |
|
| 108 |
-
-
|
| 109 |
-
- Invite Noah to share his experiences and successes with peers during class discussions to boost his confidence and encourage social interaction.
|
| 110 |
|
| 111 |
## Sources Used
|
| 112 |
|
|
|
|
| 90 |
|
| 91 |
## Week 1
|
| 92 |
|
| 93 |
+
- Monitor Noah's attendance and academic performance closely. Ensure he completes his homework regularly. Encourage him to engage more actively in class discussions and group projects.
|
|
|
|
| 94 |
|
| 95 |
## Week 2
|
| 96 |
|
| 97 |
+
- Schedule a brief meeting with Noah's parents to discuss their expectations and any challenges they face regarding his education. Provide them with resources and strategies to help improve his engagement and behavior.
|
|
|
|
| 98 |
|
| 99 |
## Week 3
|
| 100 |
|
| 101 |
+
- Arrange for an extra session during lunchtime where Noah can work one-on-one with a tutor who specializes in helping students build confidence and focus. This will also provide him with additional practice time.
|
|
|
|
| 102 |
|
| 103 |
## Week 4
|
| 104 |
|
| 105 |
+
- Organize a small group activity focused on teamwork and problem-solving skills. This could be done through a game or project-based learning activity. Encourage Noah to participate actively and share his ideas.
|
|
|
|
| 106 |
|
| 107 |
## Sources Used
|
| 108 |
|
evaluation/case_004.md
CHANGED
|
@@ -92,23 +92,19 @@
|
|
| 92 |
|
| 93 |
## Week 1
|
| 94 |
|
| 95 |
-
- Monitor William Anderson's attendance and academic performance closely.
|
| 96 |
-
- Schedule a brief meeting with William's parents to discuss their child's engagement score and homework habits. Encourage them to set up after-school opportunities at home if they haven't done so yet.
|
| 97 |
|
| 98 |
## Week 2
|
| 99 |
|
| 100 |
-
- Continue monitoring
|
| 101 |
-
- Contact William's family again to check on their involvement in after-school opportunities. Offer to help them set up more structured activities if needed.
|
| 102 |
|
| 103 |
## Week 3
|
| 104 |
|
| 105 |
-
- Review William's recent test results and assignments.
|
| 106 |
-
- Arrange a parent-teacher conference to discuss William's progress and any challenges he may be facing. This will also give you an opportunity to reinforce positive behaviors at home.
|
| 107 |
|
| 108 |
## Week 4
|
| 109 |
|
| 110 |
-
- Evaluate William's
|
| 111 |
-
- Discuss potential future after-school opportunities with William's parents. Explore options such as extracurricular clubs or community service projects that can enhance his learning experience beyond the classroom.
|
| 112 |
|
| 113 |
## Sources Used
|
| 114 |
|
|
|
|
| 92 |
|
| 93 |
## Week 1
|
| 94 |
|
| 95 |
+
- Monitor William Anderson's attendance and academic performance closely. Ensure he completes his homework regularly. Encourage him to engage more actively in class discussions and group projects.
|
|
|
|
| 96 |
|
| 97 |
## Week 2
|
| 98 |
|
| 99 |
+
- Continue monitoring attendance, academic performance, and homework completion. Engage William in more structured learning activities such as study groups or one-on-one sessions with the teacher. Encourage participation in extracurricular clubs or sports to improve engagement.
|
|
|
|
| 100 |
|
| 101 |
## Week 3
|
| 102 |
|
| 103 |
+
- Review William's recent test results and assignments. Provide constructive feedback and set clear goals for improvement. Consider offering extra credit opportunities or alternative assessment methods to motivate him. Schedule regular check-ins with parents to discuss progress and challenges.
|
|
|
|
| 104 |
|
| 105 |
## Week 4
|
| 106 |
|
| 107 |
+
- Evaluate William's behavior during class and at home. Address any issues promptly through consistent communication and consequences. Encourage positive behaviors and provide incentives for good behavior. Arrange for a brief meeting with William's parents to discuss strategies for continued success.
|
|
|
|
| 108 |
|
| 109 |
## Sources Used
|
| 110 |
|
evaluation/evaluation_dataset.json
CHANGED
|
@@ -69,7 +69,7 @@
|
|
| 69 |
"reference_url": ""
|
| 70 |
}
|
| 71 |
],
|
| 72 |
-
"generated_ai_action_plan": "# Action Plan\n\n## Week 1\n\n- Monitor
|
| 73 |
},
|
| 74 |
{
|
| 75 |
"case_id": "case_002",
|
|
@@ -141,7 +141,7 @@
|
|
| 141 |
"reference_url": ""
|
| 142 |
}
|
| 143 |
],
|
| 144 |
-
"generated_ai_action_plan": "# Action Plan\n\n## Week 1\n\n- Monitor Ava's engagement score and
|
| 145 |
},
|
| 146 |
{
|
| 147 |
"case_id": "case_003",
|
|
@@ -235,7 +235,7 @@
|
|
| 235 |
"reference_url": "https://dropoutprevention.org/effective-strategies/family-engagement/"
|
| 236 |
}
|
| 237 |
],
|
| 238 |
-
"generated_ai_action_plan": "# Action Plan\n\n## Week 1\n\n-
|
| 239 |
},
|
| 240 |
{
|
| 241 |
"case_id": "case_004",
|
|
@@ -331,7 +331,7 @@
|
|
| 331 |
"reference_url": "https://dropoutprevention.org/effective-strategies/family-engagement/"
|
| 332 |
}
|
| 333 |
],
|
| 334 |
-
"generated_ai_action_plan": "# Action Plan\n\n## Week 1\n\n- Monitor William Anderson's attendance and academic performance closely.
|
| 335 |
},
|
| 336 |
{
|
| 337 |
"case_id": "case_005",
|
|
|
|
| 69 |
"reference_url": ""
|
| 70 |
}
|
| 71 |
],
|
| 72 |
+
"generated_ai_action_plan": "# Action Plan\n\n## Week 1\n\n- Monitor Olivia's engagement score and attendance. Provide positive reinforcement for consistent participation.\n\n## Week 2\n\n- Review homework completion rates. Offer extra credit opportunities for students who consistently complete their assignments.\n\n## Week 3\n\n- Conduct a brief meeting with Olivia's parents to discuss her academic progress and any areas where she may need additional support.\n\n## Week 4\n\n- Schedule a one-on-one session with Olivia to discuss her goals and set specific, measurable objectives for improvement.\n\n## Sources Used\n\n- Continue Monitoring"
|
| 73 |
},
|
| 74 |
{
|
| 75 |
"case_id": "case_002",
|
|
|
|
| 141 |
"reference_url": ""
|
| 142 |
}
|
| 143 |
],
|
| 144 |
+
"generated_ai_action_plan": "# Action Plan\n\n## Week 1\n\n- Monitor Ava's engagement score and attendance regularly throughout the week. Ensure she completes her homework without any issues. Encourage positive interactions during class time.\n\n## Week 2\n\n- Continue monitoring engagement score and attendance. Check for any behavioral changes or signs of academic decline. Engage in brief discussions about her learning experiences to maintain motivation.\n\n## Week 3\n\n- Review the completed homework assignments. Provide feedback and suggestions for improvement. Schedule a short meeting with Ava to discuss her progress and set goals for the next week.\n\n## Week 4\n\n- Evaluate the effectiveness of the implemented strategies. Adjust the approach based on Ava's response. Consider incorporating more structured activities if necessary to address any identified gaps in her learning.\n\n## Sources Used\n\n- Continue Monitoring"
|
| 145 |
},
|
| 146 |
{
|
| 147 |
"case_id": "case_003",
|
|
|
|
| 235 |
"reference_url": "https://dropoutprevention.org/effective-strategies/family-engagement/"
|
| 236 |
}
|
| 237 |
],
|
| 238 |
+
"generated_ai_action_plan": "# Action Plan\n\n## Week 1\n\n- Monitor Noah's attendance and academic performance closely. Ensure he completes his homework regularly. Encourage him to engage more actively in class discussions and group projects.\n\n## Week 2\n\n- Schedule a brief meeting with Noah's parents to discuss their expectations and any challenges they face regarding his education. Provide them with resources and strategies to help improve his engagement and behavior.\n\n## Week 3\n\n- Arrange for an extra session during lunchtime where Noah can work one-on-one with a tutor who specializes in helping students build confidence and focus. This will also provide him with additional practice time.\n\n## Week 4\n\n- Organize a small group activity focused on teamwork and problem-solving skills. This could be done through a game or project-based learning activity. Encourage Noah to participate actively and share his ideas.\n\n## Sources Used\n\n- After-School Opportunities\n- Family Engagement"
|
| 239 |
},
|
| 240 |
{
|
| 241 |
"case_id": "case_004",
|
|
|
|
| 331 |
"reference_url": "https://dropoutprevention.org/effective-strategies/family-engagement/"
|
| 332 |
}
|
| 333 |
],
|
| 334 |
+
"generated_ai_action_plan": "# Action Plan\n\n## Week 1\n\n- Monitor William Anderson's attendance and academic performance closely. Ensure he completes his homework regularly. Encourage him to engage more actively in class discussions and group projects.\n\n## Week 2\n\n- Continue monitoring attendance, academic performance, and homework completion. Engage William in more structured learning activities such as study groups or one-on-one sessions with the teacher. Encourage participation in extracurricular clubs or sports to improve engagement.\n\n## Week 3\n\n- Review William's recent test results and assignments. Provide constructive feedback and set clear goals for improvement. Consider offering extra credit opportunities or alternative assessment methods to motivate him. Schedule regular check-ins with parents to discuss progress and challenges.\n\n## Week 4\n\n- Evaluate William's behavior during class and at home. Address any issues promptly through consistent communication and consequences. Encourage positive behaviors and provide incentives for good behavior. Arrange for a brief meeting with William's parents to discuss strategies for continued success.\n\n## Sources Used\n\n- After-School Opportunities\n- Family Engagement"
|
| 335 |
},
|
| 336 |
{
|
| 337 |
"case_id": "case_005",
|
evaluation/gpt_judge_results.json
CHANGED
|
@@ -7,62 +7,67 @@
|
|
| 7 |
{
|
| 8 |
"case_id": "case_001",
|
| 9 |
"student_name": "Olivia Moore",
|
|
|
|
| 10 |
"scores": {
|
| 11 |
-
"relevance":
|
| 12 |
-
"correctness":
|
| 13 |
-
"root_cause_alignment":
|
| 14 |
-
"evidence_alignment":
|
| 15 |
-
"actionability":
|
| 16 |
"clarity": 4,
|
| 17 |
-
"overall_quality":
|
| 18 |
},
|
| 19 |
-
"justification": "The
|
| 20 |
},
|
| 21 |
{
|
| 22 |
"case_id": "case_002",
|
| 23 |
"student_name": "Ava Martinez",
|
|
|
|
| 24 |
"scores": {
|
| 25 |
-
"relevance":
|
| 26 |
-
"correctness":
|
| 27 |
-
"root_cause_alignment":
|
| 28 |
-
"evidence_alignment":
|
| 29 |
-
"actionability":
|
| 30 |
"clarity": 4,
|
| 31 |
-
"overall_quality":
|
| 32 |
},
|
| 33 |
-
"justification": "The plan is
|
| 34 |
},
|
| 35 |
{
|
| 36 |
"case_id": "case_003",
|
| 37 |
"student_name": "Noah Taylor",
|
|
|
|
| 38 |
"scores": {
|
| 39 |
-
"relevance":
|
| 40 |
"correctness": 2,
|
| 41 |
-
"root_cause_alignment":
|
| 42 |
"evidence_alignment": 3,
|
| 43 |
-
"actionability":
|
| 44 |
"clarity": 4,
|
| 45 |
-
"overall_quality":
|
| 46 |
},
|
| 47 |
-
"justification": "
|
| 48 |
},
|
| 49 |
{
|
| 50 |
"case_id": "case_004",
|
| 51 |
"student_name": "William Anderson",
|
|
|
|
| 52 |
"scores": {
|
| 53 |
"relevance": 3,
|
| 54 |
"correctness": 2,
|
| 55 |
"root_cause_alignment": 2,
|
| 56 |
"evidence_alignment": 3,
|
| 57 |
-
"actionability":
|
| 58 |
"clarity": 4,
|
| 59 |
"overall_quality": 3
|
| 60 |
},
|
| 61 |
-
"justification": "The plan is
|
| 62 |
},
|
| 63 |
{
|
| 64 |
"case_id": "case_005",
|
| 65 |
"student_name": "John Smith",
|
|
|
|
| 66 |
"scores": {
|
| 67 |
"relevance": 4,
|
| 68 |
"correctness": 4,
|
|
@@ -77,6 +82,7 @@
|
|
| 77 |
{
|
| 78 |
"case_id": "case_006",
|
| 79 |
"student_name": "Emma Brown",
|
|
|
|
| 80 |
"scores": {
|
| 81 |
"relevance": 4,
|
| 82 |
"correctness": 4,
|
|
@@ -91,6 +97,7 @@
|
|
| 91 |
{
|
| 92 |
"case_id": "case_007",
|
| 93 |
"student_name": "Sarah Johnson",
|
|
|
|
| 94 |
"scores": {
|
| 95 |
"relevance": 4,
|
| 96 |
"correctness": 4,
|
|
@@ -105,6 +112,7 @@
|
|
| 105 |
{
|
| 106 |
"case_id": "case_008",
|
| 107 |
"student_name": "Lily Green",
|
|
|
|
| 108 |
"scores": {
|
| 109 |
"relevance": 4,
|
| 110 |
"correctness": 4,
|
|
@@ -118,13 +126,51 @@
|
|
| 118 |
}
|
| 119 |
],
|
| 120 |
"averages": {
|
| 121 |
-
"relevance": 3.
|
| 122 |
-
"correctness":
|
| 123 |
-
"root_cause_alignment": 2.
|
| 124 |
-
"evidence_alignment": 3.
|
| 125 |
-
"actionability":
|
| 126 |
"clarity": 4.0,
|
| 127 |
-
"overall_quality": 3.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
},
|
| 129 |
-
"summary": "The
|
| 130 |
}
|
|
|
|
| 7 |
{
|
| 8 |
"case_id": "case_001",
|
| 9 |
"student_name": "Olivia Moore",
|
| 10 |
+
"risk_group": "Low",
|
| 11 |
"scores": {
|
| 12 |
+
"relevance": 3,
|
| 13 |
+
"correctness": 2,
|
| 14 |
+
"root_cause_alignment": 2,
|
| 15 |
+
"evidence_alignment": 3,
|
| 16 |
+
"actionability": 4,
|
| 17 |
"clarity": 4,
|
| 18 |
+
"overall_quality": 3
|
| 19 |
},
|
| 20 |
+
"justification": "The revised plan is lighter and correctly limits sources to Continue Monitoring. However, it still implies academic improvement needs, parent discussion, and measurable improvement objectives for a low-risk student with stable performance and strong participation."
|
| 21 |
},
|
| 22 |
{
|
| 23 |
"case_id": "case_002",
|
| 24 |
"student_name": "Ava Martinez",
|
| 25 |
+
"risk_group": "Low",
|
| 26 |
"scores": {
|
| 27 |
+
"relevance": 3,
|
| 28 |
+
"correctness": 2,
|
| 29 |
+
"root_cause_alignment": 2,
|
| 30 |
+
"evidence_alignment": 3,
|
| 31 |
+
"actionability": 4,
|
| 32 |
"clarity": 4,
|
| 33 |
+
"overall_quality": 3
|
| 34 |
},
|
| 35 |
+
"justification": "The revised plan is more proportional than the original and uses only Continue Monitoring. It still over-monitors behavior, academic decline, gaps in learning, and intervention effectiveness despite Ava's low-risk profile and positive teacher notes."
|
| 36 |
},
|
| 37 |
{
|
| 38 |
"case_id": "case_003",
|
| 39 |
"student_name": "Noah Taylor",
|
| 40 |
+
"risk_group": "Moderate",
|
| 41 |
"scores": {
|
| 42 |
+
"relevance": 3,
|
| 43 |
"correctness": 2,
|
| 44 |
+
"root_cause_alignment": 2,
|
| 45 |
"evidence_alignment": 3,
|
| 46 |
+
"actionability": 4,
|
| 47 |
"clarity": 4,
|
| 48 |
+
"overall_quality": 3
|
| 49 |
},
|
| 50 |
+
"justification": "The revised plan is clearer and more actionable, but it still expands beyond Noah's evidence by emphasizing behavior and engagement issues when the primary identified risk is moderate academic risk with stable trends. It also introduces tutoring even though that was not one of the retrieved sources."
|
| 51 |
},
|
| 52 |
{
|
| 53 |
"case_id": "case_004",
|
| 54 |
"student_name": "William Anderson",
|
| 55 |
+
"risk_group": "Moderate",
|
| 56 |
"scores": {
|
| 57 |
"relevance": 3,
|
| 58 |
"correctness": 2,
|
| 59 |
"root_cause_alignment": 2,
|
| 60 |
"evidence_alignment": 3,
|
| 61 |
+
"actionability": 4,
|
| 62 |
"clarity": 4,
|
| 63 |
"overall_quality": 3
|
| 64 |
},
|
| 65 |
+
"justification": "The revised plan is practical and uses the retrieved sources, but it still over-prescribes support for a moderate-risk student with stable root-cause trends. It adds engagement, extracurricular, parent check-in, and behavior-at-home actions that are only weakly supported by the case evidence."
|
| 66 |
},
|
| 67 |
{
|
| 68 |
"case_id": "case_005",
|
| 69 |
"student_name": "John Smith",
|
| 70 |
+
"risk_group": "High",
|
| 71 |
"scores": {
|
| 72 |
"relevance": 4,
|
| 73 |
"correctness": 4,
|
|
|
|
| 82 |
{
|
| 83 |
"case_id": "case_006",
|
| 84 |
"student_name": "Emma Brown",
|
| 85 |
+
"risk_group": "High",
|
| 86 |
"scores": {
|
| 87 |
"relevance": 4,
|
| 88 |
"correctness": 4,
|
|
|
|
| 97 |
{
|
| 98 |
"case_id": "case_007",
|
| 99 |
"student_name": "Sarah Johnson",
|
| 100 |
+
"risk_group": "Critical",
|
| 101 |
"scores": {
|
| 102 |
"relevance": 4,
|
| 103 |
"correctness": 4,
|
|
|
|
| 112 |
{
|
| 113 |
"case_id": "case_008",
|
| 114 |
"student_name": "Lily Green",
|
| 115 |
+
"risk_group": "Critical",
|
| 116 |
"scores": {
|
| 117 |
"relevance": 4,
|
| 118 |
"correctness": 4,
|
|
|
|
| 126 |
}
|
| 127 |
],
|
| 128 |
"averages": {
|
| 129 |
+
"relevance": 3.5,
|
| 130 |
+
"correctness": 3.0,
|
| 131 |
+
"root_cause_alignment": 2.88,
|
| 132 |
+
"evidence_alignment": 3.38,
|
| 133 |
+
"actionability": 4.0,
|
| 134 |
"clarity": 4.0,
|
| 135 |
+
"overall_quality": 3.5
|
| 136 |
+
},
|
| 137 |
+
"risk_group_averages": {
|
| 138 |
+
"Low": {
|
| 139 |
+
"relevance": 3.0,
|
| 140 |
+
"correctness": 2.0,
|
| 141 |
+
"root_cause_alignment": 2.0,
|
| 142 |
+
"evidence_alignment": 3.0,
|
| 143 |
+
"actionability": 4.0,
|
| 144 |
+
"clarity": 4.0,
|
| 145 |
+
"overall_quality": 3.0
|
| 146 |
+
},
|
| 147 |
+
"Moderate": {
|
| 148 |
+
"relevance": 3.0,
|
| 149 |
+
"correctness": 2.0,
|
| 150 |
+
"root_cause_alignment": 2.0,
|
| 151 |
+
"evidence_alignment": 3.0,
|
| 152 |
+
"actionability": 4.0,
|
| 153 |
+
"clarity": 4.0,
|
| 154 |
+
"overall_quality": 3.0
|
| 155 |
+
},
|
| 156 |
+
"High": {
|
| 157 |
+
"relevance": 4.0,
|
| 158 |
+
"correctness": 4.0,
|
| 159 |
+
"root_cause_alignment": 4.0,
|
| 160 |
+
"evidence_alignment": 4.0,
|
| 161 |
+
"actionability": 4.0,
|
| 162 |
+
"clarity": 4.0,
|
| 163 |
+
"overall_quality": 4.0
|
| 164 |
+
},
|
| 165 |
+
"Critical": {
|
| 166 |
+
"relevance": 4.0,
|
| 167 |
+
"correctness": 4.0,
|
| 168 |
+
"root_cause_alignment": 3.5,
|
| 169 |
+
"evidence_alignment": 3.5,
|
| 170 |
+
"actionability": 4.0,
|
| 171 |
+
"clarity": 4.0,
|
| 172 |
+
"overall_quality": 4.0
|
| 173 |
+
}
|
| 174 |
},
|
| 175 |
+
"summary": "The revised prompt improves Low and Moderate action plans from the first pass by reducing action density and limiting source accumulation. Overall quality rises to 3.50 across all 8 cases. The remaining weakness is that Low and Moderate plans still infer mild concerns from stable student profiles, so correctness and root-cause alignment remain lower than High and Critical cases."
|
| 176 |
}
|
evaluation/gpt_judge_summary.md
CHANGED
|
@@ -4,44 +4,53 @@
|
|
| 4 |
|
| 5 |
| Criterion | Average Score |
|
| 6 |
|---|---:|
|
| 7 |
-
| Relevance | 3.
|
| 8 |
-
| Correctness |
|
| 9 |
-
| Root Cause Alignment | 2.
|
| 10 |
-
| Evidence Alignment | 3.
|
| 11 |
-
| Actionability |
|
| 12 |
| Clarity | 4.00 |
|
| 13 |
-
| Overall Quality | 3.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
## Per-Case Scores
|
| 16 |
|
| 17 |
-
| Case | Student | Relevance | Correctness | Root Cause Alignment | Evidence Alignment | Actionability | Clarity | Overall Quality |
|
| 18 |
-
|---|---|---:|---:|---:|---:|---:|---:|---:|
|
| 19 |
-
| case_001 | Olivia Moore |
|
| 20 |
-
| case_002 | Ava Martinez |
|
| 21 |
-
| case_003 | Noah Taylor |
|
| 22 |
-
| case_004 | William Anderson | 3 | 2 | 2 | 3 |
|
| 23 |
-
| case_005 | John Smith | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
|
| 24 |
-
| case_006 | Emma Brown | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
|
| 25 |
-
| case_007 | Sarah Johnson | 4 | 4 | 4 | 3 | 4 | 4 | 4 |
|
| 26 |
-
| case_008 | Lily Green | 4 | 4 | 3 | 4 | 4 | 4 | 4 |
|
| 27 |
|
| 28 |
## Short Justifications
|
| 29 |
|
| 30 |
### case_001: Olivia Moore
|
| 31 |
|
| 32 |
-
The
|
| 33 |
|
| 34 |
### case_002: Ava Martinez
|
| 35 |
|
| 36 |
-
The plan is
|
| 37 |
|
| 38 |
### case_003: Noah Taylor
|
| 39 |
|
| 40 |
-
|
| 41 |
|
| 42 |
### case_004: William Anderson
|
| 43 |
|
| 44 |
-
The plan is
|
| 45 |
|
| 46 |
### case_005: John Smith
|
| 47 |
|
|
@@ -61,4 +70,4 @@ The plan addresses Lily's critical attendance, academic, homework, and behavior
|
|
| 61 |
|
| 62 |
## Overall Finding
|
| 63 |
|
| 64 |
-
The
|
|
|
|
| 4 |
|
| 5 |
| Criterion | Average Score |
|
| 6 |
|---|---:|
|
| 7 |
+
| Relevance | 3.50 |
|
| 8 |
+
| Correctness | 3.00 |
|
| 9 |
+
| Root Cause Alignment | 2.88 |
|
| 10 |
+
| Evidence Alignment | 3.38 |
|
| 11 |
+
| Actionability | 4.00 |
|
| 12 |
| Clarity | 4.00 |
|
| 13 |
+
| Overall Quality | 3.50 |
|
| 14 |
+
|
| 15 |
+
## Risk Group Averages
|
| 16 |
+
|
| 17 |
+
| Risk Group | Relevance | Correctness | Root Cause Alignment | Evidence Alignment | Actionability | Clarity | Overall Quality |
|
| 18 |
+
|---|---:|---:|---:|---:|---:|---:|---:|
|
| 19 |
+
| Low | 3.00 | 2.00 | 2.00 | 3.00 | 4.00 | 4.00 | 3.00 |
|
| 20 |
+
| Moderate | 3.00 | 2.00 | 2.00 | 3.00 | 4.00 | 4.00 | 3.00 |
|
| 21 |
+
| High | 4.00 | 4.00 | 4.00 | 4.00 | 4.00 | 4.00 | 4.00 |
|
| 22 |
+
| Critical | 4.00 | 4.00 | 3.50 | 3.50 | 4.00 | 4.00 | 4.00 |
|
| 23 |
|
| 24 |
## Per-Case Scores
|
| 25 |
|
| 26 |
+
| Case | Student | Risk Group | Relevance | Correctness | Root Cause Alignment | Evidence Alignment | Actionability | Clarity | Overall Quality |
|
| 27 |
+
|---|---|---|---:|---:|---:|---:|---:|---:|---:|
|
| 28 |
+
| case_001 | Olivia Moore | Low | 3 | 2 | 2 | 3 | 4 | 4 | 3 |
|
| 29 |
+
| case_002 | Ava Martinez | Low | 3 | 2 | 2 | 3 | 4 | 4 | 3 |
|
| 30 |
+
| case_003 | Noah Taylor | Moderate | 3 | 2 | 2 | 3 | 4 | 4 | 3 |
|
| 31 |
+
| case_004 | William Anderson | Moderate | 3 | 2 | 2 | 3 | 4 | 4 | 3 |
|
| 32 |
+
| case_005 | John Smith | High | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
|
| 33 |
+
| case_006 | Emma Brown | High | 4 | 4 | 4 | 4 | 4 | 4 | 4 |
|
| 34 |
+
| case_007 | Sarah Johnson | Critical | 4 | 4 | 4 | 3 | 4 | 4 | 4 |
|
| 35 |
+
| case_008 | Lily Green | Critical | 4 | 4 | 3 | 4 | 4 | 4 | 4 |
|
| 36 |
|
| 37 |
## Short Justifications
|
| 38 |
|
| 39 |
### case_001: Olivia Moore
|
| 40 |
|
| 41 |
+
The revised plan is lighter and correctly limits sources to Continue Monitoring. However, it still implies academic improvement needs, parent discussion, and measurable improvement objectives for a low-risk student with stable performance and strong participation.
|
| 42 |
|
| 43 |
### case_002: Ava Martinez
|
| 44 |
|
| 45 |
+
The revised plan is more proportional than the original and uses only Continue Monitoring. It still over-monitors behavior, academic decline, gaps in learning, and intervention effectiveness despite Ava's low-risk profile and positive teacher notes.
|
| 46 |
|
| 47 |
### case_003: Noah Taylor
|
| 48 |
|
| 49 |
+
The revised plan is clearer and more actionable, but it still expands beyond Noah's evidence by emphasizing behavior and engagement issues when the primary identified risk is moderate academic risk with stable trends. It also introduces tutoring even though that was not one of the retrieved sources.
|
| 50 |
|
| 51 |
### case_004: William Anderson
|
| 52 |
|
| 53 |
+
The revised plan is practical and uses the retrieved sources, but it still over-prescribes support for a moderate-risk student with stable root-cause trends. It adds engagement, extracurricular, parent check-in, and behavior-at-home actions that are only weakly supported by the case evidence.
|
| 54 |
|
| 55 |
### case_005: John Smith
|
| 56 |
|
|
|
|
| 70 |
|
| 71 |
## Overall Finding
|
| 72 |
|
| 73 |
+
The revised prompt improves Low and Moderate action plans from the first pass by reducing action density and limiting source accumulation. Overall quality rises to 3.50 across all 8 cases. The remaining weakness is that Low and Moderate plans still infer mild concerns from stable student profiles, so correctness and root-cause alignment remain lower than High and Critical cases.
|