Instructions to use kernels-community/flash-attn2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Kernels
How to use kernels-community/flash-attn2 with Kernels:
# !pip install kernels from kernels import get_kernel kernel = get_kernel("kernels-community/flash-attn2") - Notebooks
- Google Colab
- Kaggle
File size: 477 Bytes
a7165c8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | // Copyright (c) 2024, Tri Dao.
// Splitting the different head dimensions to different files to speed up compilation.
// This file is auto-generated. See "generate_kernels.py"
#include "namespace_config.h"
#include "flash_bwd_launch_template.h"
namespace FLASH_NAMESPACE {
template<>
void run_mha_bwd_<cutlass::half_t, 64, true>(Flash_bwd_params ¶ms, cudaStream_t stream) {
run_mha_bwd_hdim64<cutlass::half_t, true>(params, stream);
}
} // namespace FLASH_NAMESPACE |