name,layers,parameters_millions,weight_mib,kd_stage,mean_total_loss
24L upstream text teacher,24,752.393024,1435.1135559082031,,
8L task-free KD,8,420.318144,801.7042922973633,24to8,0.17622653172702485
6L task-free KD,6,377.207424,719.4742431640625,8to6,0.03382104352249371
4L task-free KD root,4,334.096704,637.2442016601562,6to4,0.047020394468859195