torchao int8 weight-only + flash-linear-attention (bnb int8 was ~4 tok/s) 3ebe40d Running verified UnstableLlama commited on Jul 16
int8 (bitsandbytes) on large slice; scale GPU duration to max_new_tokens a517d99 verified UnstableLlama commited on Jul 16