Hugging Face's logo

Intel
/
bert-large-uncased-sparse-80-1x4-block-pruneofa

Fill-Mask
Transformers
PyTorch
English
bert
pretraining
Model card Files
xet
 Community
1
How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("fill-mask", model="Intel/bert-large-uncased-sparse-80-1x4-block-pruneofa")
# Load model directly
from transformers import AutoTokenizer, AutoModelForPreTraining

tokenizer = AutoTokenizer.from_pretrained("Intel/bert-large-uncased-sparse-80-1x4-block-pruneofa")
model = AutoModelForPreTraining.from_pretrained("Intel/bert-large-uncased-sparse-80-1x4-block-pruneofa", device_map="auto")
Quick Links

80% 1x4 Block Sparse BERT-Large (uncased) Prune OFA

This model is was created using Prune OFA method described in Prune Once for All: Sparse Pre-Trained Language Models presented in ENLSP NeurIPS Workshop 2021.

For further details on the model and its result, see our paper and our implementation available here.

Downloads last month
11
Inference Providers NEW
Fill-Mask
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Datasets used to train Intel/bert-large-uncased-sparse-80-1x4-block-pruneofa

Collection including Intel/bert-large-uncased-sparse-80-1x4-block-pruneofa

Paper for Intel/bert-large-uncased-sparse-80-1x4-block-pruneofa