Not-For-All-Audiences

Instructions to use crestf411/L3-70B-daybreak-abliterated-v0.4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use crestf411/L3-70B-daybreak-abliterated-v0.4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="crestf411/L3-70B-daybreak-abliterated-v0.4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("crestf411/L3-70B-daybreak-abliterated-v0.4")
model = AutoModelForCausalLM.from_pretrained("crestf411/L3-70B-daybreak-abliterated-v0.4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Local Apps Settings

vLLM

How to use crestf411/L3-70B-daybreak-abliterated-v0.4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "crestf411/L3-70B-daybreak-abliterated-v0.4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "crestf411/L3-70B-daybreak-abliterated-v0.4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/crestf411/L3-70B-daybreak-abliterated-v0.4

SGLang

How to use crestf411/L3-70B-daybreak-abliterated-v0.4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "crestf411/L3-70B-daybreak-abliterated-v0.4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "crestf411/L3-70B-daybreak-abliterated-v0.4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "crestf411/L3-70B-daybreak-abliterated-v0.4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "crestf411/L3-70B-daybreak-abliterated-v0.4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use crestf411/L3-70B-daybreak-abliterated-v0.4 with Docker Model Runner:
```
docker model run hf.co/crestf411/L3-70B-daybreak-abliterated-v0.4
```

L3-70B-daybreak-abliterated-v0.4 / huggingface-metadata.txt

crestf411

Upload folder using huggingface_hub

c3fbab0 verified about 2 years ago

Raw

History Blame Contribute Delete

3.27 kB

	url: https://huggingface.co/failspy/llama-3-70B-Instruct-abliterated
	branch: main
	download date: 2024-05-27 15:39:11
	sha256sum:
	68002021faf01e5972bcc7e7f8e6475f806dd012874ee454b3d162f6897ef057 model-00001-of-00030.safetensors
	821908ef377ad9e3e3de7d79bd7731c73821aee9e0b4f2a48fd6e7cb7ad74c76 model-00002-of-00030.safetensors
	a3a34ac6311cea29752257988579bbbb6f1bca98277347618f7e5e1e92dbd541 model-00003-of-00030.safetensors
	4ab2e5bc1693e95ef667a7b8ed427805373b473e5c74e46d459ab1ab297d5a7c model-00004-of-00030.safetensors
	4f4127aa8a6cd5252e341299362c05a4b40e96aaa7b633ada471157b1ca0b6d4 model-00005-of-00030.safetensors
	8d7fbc1594d74eb88c31aed140b3d78750bf70b8c1e1a6fde06bc9445240675e model-00006-of-00030.safetensors
	2ffd5faa16960c7f6cf9dbaabcfa9ffbc76a85b1caa234e7b8092fb6c1a50595 model-00007-of-00030.safetensors
	f5573b4b3475dedf53bb22655ba415bf9b786024c5767a28b247ebb60155cabd model-00008-of-00030.safetensors
	ccb170681dba48321ae6a97f60e2227cffc38c9f217e9571b63411b013861599 model-00009-of-00030.safetensors
	5c63f29d754c7ed8523dec422e7281198ec8264b635a75fcfbee53a5b218c451 model-00010-of-00030.safetensors
	1f5cde63459cd690a71e2bfc51c82d64b16e8d08cf01aa6dd9b97ce8e30e956d model-00011-of-00030.safetensors
	469cdafe6e0ae2b25c6e3b054ebd88d043f8fa897003e437bd6b2f7836a9e1bc model-00012-of-00030.safetensors
	d834a8d08ddf794633000d8f0467bb534dac10d990c2e4b93ae9a3cfe55c206a model-00013-of-00030.safetensors
	0366feafa8569d85ad35b6af686837abd5372a3504db7796889b1e72bcba6c82 model-00014-of-00030.safetensors
	7a0f2de0f24387abfa90a54cacc7f622ea87b64e03843e62d8d2424f46e12d42 model-00015-of-00030.safetensors
	480a2e13965123be156a026b9c93e68b746bbedb877050d567be2470c435c308 model-00016-of-00030.safetensors
	a3ca7e1d3988bc77d3c367bc8f9ea16ea6993c334cdec0942e042845a915cd59 model-00017-of-00030.safetensors
	dfdfbc03574cd8ad2dae7fe96e40b7a0501e0815298ea67398539f4fa6a76f4a model-00018-of-00030.safetensors
	3731dc19e7af2821c8b0200c60c8314ea9c1d0d18b2d13d9dd57a96c3491b079 model-00019-of-00030.safetensors
	d522d6f2186440c54f7e3b56a991dc8a200ea12f1486dfa8003bec60f9ddbd02 model-00020-of-00030.safetensors
	4d31091208b455608f6fbc6e43496eb41b3bfeb46caf35a6fa453e010e267fb1 model-00021-of-00030.safetensors
	8c3d16b91b1f4f2aabc0be7e2479a0ebe09b4a430a8fd2ae13f14af783c0f99d model-00022-of-00030.safetensors
	3fb5b337e9333fe16282f73ebb30a25ca285021f9a68b4e3d6eff84a3e6645cb model-00023-of-00030.safetensors
	c05e9cd1b9a62ef06b6f7b3d4014d25b2c2b8716277064397eea8e262a36879c model-00024-of-00030.safetensors
	1a3e22924e58df9f86382823c40098fb15589f5b8fa12ff740fa7a4f14794c30 model-00025-of-00030.safetensors
	748499e867537eaf3a2891d49f6c0580c13d0f657400ee509f408f2e2e60dfbe model-00026-of-00030.safetensors
	8f563045b3c4825589719870a0b8e6fc5ad077fed423b713369d42c4497f30ec model-00027-of-00030.safetensors
	3a7c691dd10e72b993f562b4b1fd318cd39727c3ee024aa691ab7d341fdfa5a7 model-00028-of-00030.safetensors
	b01c27dd6f543a1e8305e5cc1a371fcf1db0a4691e0ebceee802d3f5ec1a5d76 model-00029-of-00030.safetensors
	55298e57c1dfe47357f5b367863da3ad8aa7b009249b4d7146bd4cf22f004e89 model-00030-of-00030.safetensors
	ed026997d8ac568bcbce33544291ec0b361b08debb4a80432be131bfe9f121dc refusal_dir.pth