Instructions to use MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6")
model = AutoModelForCausalLM.from_pretrained("MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6

SGLang

How to use MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6 with Docker Model Runner:
```
docker model run hf.co/MetaphoricalCode/Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6
```

Dans-PersonalityEngine-V1.2.0-24b-exl3-4bpw-hb6 / README.md

MetaphoricalCode

Upload 11 files

8f5bc64 verified about 1 year ago

preview code

raw

history blame contribute delete

15.5 kB

metadata

thumbnail: >-
  https://huggingface.co/PocketDoc/Dans-PersonalityEngine-V1.2.0-24b/resolve/main/resources/pe24.png
license: apache-2.0
tags:
  - general-purpose
  - roleplay
  - storywriting
  - chemistry
  - biology
  - code
  - climate
  - axolotl
  - text-generation-inference
  - finetune
datasets:
  - PocketDoc/Dans-MemoryCore-CoreCurriculum-Small
  - AquaV/US-Army-Survival-Sharegpt
  - AquaV/Multi-Environment-Operations-Sharegpt
  - AquaV/Resistance-Sharegpt
  - AquaV/Interrogation-Sharegpt
  - AquaV/Chemical-Biological-Safety-Applications-Sharegpt
  - AquaV/Energetic-Materials-Sharegpt
  - PocketDoc/Dans-Mathmaxx
  - PocketDoc/Dans-Mathmaxx-Numina-CoT
  - PJMixers/Math-Multiturn-1K-ShareGPT
  - PocketDoc/Dans-Benchmaxx-COT
  - PocketDoc/Dans-Codemaxx-LeetCode
  - PocketDoc/Dans-Codemaxx-CodeFeedback-Conversations
  - PocketDoc/Dans-Codemaxx-CodeFeedback-SingleTurn
  - PocketDoc/Dans-Codemaxx-Bigcode-SelfInstruct
  - PocketDoc/Dans-Taskmaxx
  - PocketDoc/Dans-Taskmaxx-DataPrepper
  - PocketDoc/Dans-Taskmaxx-ConcurrentQA-Reworked
  - PocketDoc/Dans-Taskmaxx-TableGPT
  - PocketDoc/Dans-Taskmaxx-SciRIFF
  - PocketDoc/Dans-Taskmaxx-Edit
  - PocketDoc/Dans-Toolmaxx-Agent
  - PocketDoc/Dans-Toolmaxx-ShellCommands
  - PocketDoc/Dans-Toolmaxx-Functions-Toolbench
  - PocketDoc/Dans-Toolmaxx-Functions-ToolACE
  - PocketDoc/Dans-ASCIIMaxx-Wordart
  - PocketDoc/Dans-Prosemaxx-Gutenberg
  - PocketDoc/Dans-Prosemaxx-Cowriter-3-XL
  - PocketDoc/Dans-Prosemaxx-Adventure
  - PocketDoc/Dans-Failuremaxx-Adventure-3
  - PocketDoc/Dans-Prosemaxx-InstructWriter-ZeroShot-2
  - PocketDoc/Dans-Prosemaxx-InstructWriter-Continue-2
  - PocketDoc/Dans-Assistantmaxx-Sharegpt
  - PocketDoc/Dans-Assistantmaxx-OpenAssistant2
  - PocketDoc/Dans-Assistantmaxx-Opus-Merge
  - PocketDoc/Dans-Assistantmaxx-sonnetorca-subset
  - PocketDoc/Dans-Assistantmaxx-sonnetorca-subset-2
  - PocketDoc/Dans-Assistantmaxx-NoRobots
  - PocketDoc/Dans-Assistantmaxx-Synthia
  - PocketDoc/Dans-Assistantmaxx-ASL
  - PocketDoc/Dans-Assistantmaxx-PersonaLLM-Opus
  - PocketDoc/Dans-Assistantmaxx-UnnaturalInstructions-GPT4
  - PocketDoc/Dans-Assistantmaxx-LongAlign
  - PocketDoc/Dans-Assistantmaxx-EvolKit
  - PocketDoc/Dans-Assistantmaxx-Camel-GPT4
  - PocketDoc/Dans-Assistantmaxx-OpenLeecher-Instruct
  - PocketDoc/Dans-Assistantmaxx-Tulu3-IF
  - PocketDoc/Dans-Systemmaxx
  - PocketDoc/Dans-Logicmaxx-Skunkworks
  - PocketDoc/Dans-Logicmaxx-FI-VeriMed
  - PocketDoc/Dans-Logicmaxx-SAT-AP
  - PocketDoc/Dans-Logicmaxx-Magpie-Ultra
  - PJMixers/grimulkan_theory-of-mind-ShareGPT
  - PJMixers/grimulkan_physical-reasoning-ShareGPT
  - PocketDoc/Dans-Personamaxx
  - PocketDoc/Dans-Personamaxx-Rainy
  - PocketDoc/Dans-Personamaxx-C1
  - PocketDoc/Dans-Personamaxx-VN
language:
  - en
base_model:
  - PocketDoc/Dans-PersonalityEngine-V1.2.0-24b
base_model_relation: quantized
pipeline_tag: text-generation
library_name: transformers

Quantized using the default exllamav3 (0.0.1) quantization process.

Original model: https://huggingface.co/PocketDoc/Dans-PersonalityEngine-V1.2.0-24b
exllamav3: https://github.com/turboderp-org/exllamav3

Dans-PersonalityEngine-V1.2.0-24b

  
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠀⠄⠀⡂⠀⠁⡄⢀⠁⢀⣈⡄⠌⠐⠠⠤⠄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⡄⠆⠀⢠⠀⠛⣸⣄⣶⣾⡷⡾⠘⠃⢀⠀⣴⠀⡄⠰⢆⣠⠘⠰⠀⡀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠃⠀⡋⢀⣤⡿⠟⠋⠁⠀⡠⠤⢇⠋⠀⠈⠃⢀⠀⠈⡡⠤⠀⠀⠁⢄⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠁⡂⠀⠀⣀⣔⣧⠟⠋⠀⢀⡄⠀⠪⣀⡂⢁⠛⢆⠀⠀⠀⢎⢀⠄⢡⠢⠛⠠⡀⠀⠄⠀⠀
⠀⠀⡀⠡⢑⠌⠈⣧⣮⢾⢏⠁⠀⠀⡀⠠⠦⠈⠀⠞⠑⠁⠀⠀⢧⡄⠈⡜⠷⠒⢸⡇⠐⠇⠿⠈⣖⠂⠀
⠀⢌⠀⠤⠀⢠⣞⣾⡗⠁⠀⠈⠁⢨⡼⠀⠀⠀⢀⠀⣀⡤⣄⠄⠈⢻⡇⠀⠐⣠⠜⠑⠁⠀⣀⡔⡿⠨⡄
⠈⠂⠀⠆⠀⣼⣾⠟⠀⠑⠀⡐⠗⠉⠀⠐⠶⣤⡵⠋⠀⠠⠹⡌⡀⠘⠇⢠⣾⡣⣀⡴⠋⠅⠈⢊⠠⡱⡀
⠪⠑⢌⠂⣼⣿⡟⠀⠀⠙⠀⠀⠀⡀⠀⠀⠐⡞⡐⠀⠀⡧⠀⢀⠠⠀⣁⠾⡇⠀⠙⡁⠀⠀⢀⣨⣄⡠⢱
⣸⠈⠊⠙⣛⣿⡧⠔⠚⠛⠳⣄⣀⡬⠤⠬⠼⡣⠃⠀⢀⡗⠀⡤⠞⠙⠄⠂⠃⢀⣠⣤⠶⠙⠅⠁⠃⠋⠈
⢋⠼⣀⠰⢯⢿⠁⠀⢢⠀⠀⢐⠋⡀⠀⠈⠁⠀⣀⣰⠏⠒⠙⠈⠀⣀⡤⠞⢁⣼⠏⠘⢀⣀⢤⢤⡐⢈⠂
⠀⠢⠀⠀⠸⣿⡄⠲⠚⠘⠚⠃⢀⠀⠈⢋⠶⠛⠉⠉⢃⣀⢤⢾⠋⣁⡤⡚⠁⢹⠁⠠⢛⠠⠬⠁⢬⠀⠀
⠀⠈⢳⣒⠋⠉⣿⢐⠠⣀⣃⠀⠀⠉⠂⢁⣀⣀⡤⢞⠩⢑⡨⠰⡞⠁⠁⢀⡠⠾⠎⡈⡌⡈⡓⡀⠄⠀⠀
⠀⠀⠀⠉⠘⠃⢻⡒⠦⢼⣿⣛⣻⣿⡷⢄⣀⣀⣠⣴⢾⣿⣆⣡⡄⣠⣪⡿⣷⣾⣷⣧⡡⠅⣇⠍⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠙⠒⠒⠛⠛⠓⠉⢹⠀⣷⠴⣻⣽⡻⢧⢻⡿⡏⣼⢿⣻⢾⣿⣿⣿⡿⢠ ⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠂⠻⠨⠰⢋⡅⠉⣑⡇⡗⣿⢂⣸⡿⣿⣛⠿⠃⠁ ⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠳⣌⣙⣸⢧⣿⣕⣼⣇⢹⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⣸⢧⢟⢟⡟⣾⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⢰⠙⣾⡟⣻⡕⣹⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⢸⢰⡏⢠⡿⠾⠋⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⢸⠸⡇⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠸⢸⢸⡇⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠘⠇⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

This model series is intended to be multifarious in its capabilities and should be quite capable at both co-writing and roleplay as well as find itself quite at home performing sentiment analysis or summarization as part of a pipeline.

It has been trained on a wide array of one shot instructions, multi turn instructions, tool use, role playing scenarios, text adventure games, co-writing, and much more.

Key Details

BASE MODEL: mistralai/Mistral-Small-24B-Base-2501
LICENSE: apache-2.0
LANGUAGE: English
CONTEXT LENGTH: 32768 tokens

Recommended Settings

TEMPERATURE: 1.0
TOP_P: 0.95
MIN_P: 0.05

Prompting Format

The model uses standard "ChatML" format:

<|im_start|>system
system prompt<|im_end|>
<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>

A word of caution: As of Feb 19 2025 backends can't seem to agree on automatically addind a "bos" token to the start, which they should! I'm investigating if there is a way I can change the config to mitigate this but for now if you have incoherent outputs not typical of a 24b model (verbatim repeating what you said back to you for instance) then try adding "<s>" to the very beginning of your context.

SillyTavern Templates

Context Template

{
    "story_string": "<|im_start|>system\n{{#if system}}{{system}}\n{{/if}}{{#if wiBefore}}{{wiBefore}}\n{{/if}}{{#if description}}{{description}}\n{{/if}}{{#if personality}}{{char}}'s personality: {{personality}}\n{{/if}}{{#if scenario}}Scenario: {{scenario}}\n{{/if}}{{#if wiAfter}}{{wiAfter}}\n{{/if}}{{#if persona}}{{persona}}\n{{/if}}{{trim}}<|im_end|>\n",
    "example_separator": "",
    "chat_start": "",
    "use_stop_strings": false,
    "allow_jailbreak": false,
    "always_force_name2": false,
    "trim_sentences": false,
    "include_newline": false,
    "single_line": false,
    "name": "Dan-ChatML"
}

Instruct Template

{
    "system_prompt": "Write {{char}}'s actions and dialogue, user will write {{user}}'s.",
    "input_sequence": "<|im_start|>user\n",
    "output_sequence": "<|im_start|>assistant\n",
    "first_output_sequence": "",
    "last_output_sequence": "",
    "system_sequence_prefix": "",
    "system_sequence_suffix": "",
    "stop_sequence": "<|im_end|>",
    "wrap": false,
    "macro": true,
    "names": false,
    "names_force_groups": false,
    "activation_regex": "",
    "skip_examples": false,
    "output_suffix": "<|im_end|>\n",
    "input_suffix": "<|im_end|>\n",
    "system_sequence": "<|im_start|>system\n",
    "system_suffix": "<|im_end|>\n",
    "user_alignment_message": "",
    "last_system_sequence": "",
    "system_same_as_user": false,
    "first_input_sequence": "",
    "last_input_sequence": "",
    "name": "Dan-ChatML"
}

A Chub.AI Sponsored Model

Character Hub supported this model with 65 hours on a 4x H200 144GB system. This is only some of what they've provided me for training and I am very grateful for their contributions, this model especially would have been difficult without it.

Character Hub has been supporting model development for quite a while now and they may be interested in your projects! Contact them through this google form.

Support Development

Development is limited by funding and resources. To help support:

- Contact on HF

- Email: visuallyadequate@gmail.com