Mistral-7B
Collection
5 items • Updated • 1
Text Generation
Transformers
Safetensors
English
mistral
mergekit
Merge
roleplay
Eval Results (legacy)
text-generation-inference
How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "FallenMerick/Chunky-Lemon-Cookie-11B" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "FallenMerick/Chunky-Lemon-Cookie-11B",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
Chunky-Lemon-Cookie-11B
This is a merge of pre-trained language models created using mergekit.
GGUF quants:
- https://huggingface.co/backyardai/Chunky-Lemon-Cookie-11B-GGUF
- https://huggingface.co/mradermacher/Chunky-Lemon-Cookie-11B-GGUF
Merge Details
Merge Method
This model was merged using the following methods:
- passthrough
- task arithmetic
Models Merged
The following models were included in the merge:
- SanjiWatsuki/Kunoichi-7B
- SanjiWatsuki/Silicon-Maid-7B
- KatyTheCutie/LemonadeRP-4.5.3
- Sao10K/Fimbulvetr-11B-v2
- mistralai/Mistral-7B-v0.1
Configuration
The following YAML configurations were used to produce this model:
slices:
- sources:
- model: mistralai/Mistral-7B-v0.1
layer_range: [0, 24]
- sources:
- model: mistralai/Mistral-7B-v0.1
layer_range: [8, 32]
merge_method: passthrough
dtype: float16
name: Mistral-11B
---
slices:
- sources:
- model: SanjiWatsuki/Kunoichi-7B
layer_range: [0, 24]
- sources:
- model: SanjiWatsuki/Silicon-Maid-7B
layer_range: [8, 24]
- sources:
- model: KatyTheCutie/LemonadeRP-4.5.3
layer_range: [24, 32]
merge_method: passthrough
dtype: float16
name: Big-Lemon-Cookie-11B
---
models:
- model: Big-Lemon-Cookie-11B
parameters:
weight: 0.85
- model: Sao10K/Fimbulvetr-11B-v2
parameters:
weight: 0.15
merge_method: task_arithmetic
base_model: Mistral-11B
dtype: float16
name: Chunky-Lemon-Cookie-11B
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 70.23 |
| AI2 Reasoning Challenge (25-Shot) | 69.62 |
| HellaSwag (10-Shot) | 86.55 |
| MMLU (5-Shot) | 65.35 |
| TruthfulQA (0-shot) | 61.59 |
| Winogrande (5-shot) | 79.79 |
| GSM8k (5-shot) | 58.45 |
- Downloads last month
- 6
Model tree for FallenMerick/Chunky-Lemon-Cookie-11B
Merge model
this model
Spaces using FallenMerick/Chunky-Lemon-Cookie-11B 8
Collection including FallenMerick/Chunky-Lemon-Cookie-11B
Paper for FallenMerick/Chunky-Lemon-Cookie-11B
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard69.620
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard86.550
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard65.350
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard61.590
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard79.790
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard58.450
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FallenMerick/Chunky-Lemon-Cookie-11B" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FallenMerick/Chunky-Lemon-Cookie-11B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'