Bio-Medical-Llama-3.2-1B-MLX-4bit
This is a 4-bit quantized MLX version of ContactDoctor/Bio-Medical-Llama-3-2-1B-CoT-012025 for efficient inference on Apple Silicon.
Model Details
- Base Model: Bio-Medical-Llama-3-2-1B-CoT-012025
- Format: MLX (Apple Silicon optimized)
- Quantization: 4-bit (~4.5 bits per weight)
- Size: ~685 MB
- Parameters: 1B
- Use Case: Medical/Healthcare Q&A on iOS devices
Usage
from mlx_lm import load, generate
model, tokenizer = load("oscardeng/Bio-Medical-Llama-3.2-1B-MLX-4bit")
prompt = "What are the common symptoms of type 2 diabetes?"
messages = [{"role": "user", "content": prompt}]
formatted = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=formatted, max_tokens=256)
print(response)
Disclaimer
This model is for informational purposes only and should not be used as a substitute for professional medical advice, diagnosis, or treatment. Always consult with a qualified healthcare provider.
License
This model inherits the Llama 3.2 Community License from the base model.
- Downloads last month
- 19
Model size
0.2B params
Tensor type
F16
·
U32 ·
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for oscardeng/Bio-Medical-Llama-3.2-1B-MLX-4bit
Base model
meta-llama/Llama-3.2-1B-Instruct