Instructions to use aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K")
model = AutoModelForCausalLM.from_pretrained("aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K")

Inference
Local Apps Settings

vLLM

How to use aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K

SGLang

How to use aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K with Docker Model Runner:
```
docker model run hf.co/aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K
```

aifeifei798 commited on Jul 1, 2024

Commit

0212e81

verified ·

1 Parent(s): fbef190

Upload 2 files

Browse files

Files changed (2) hide show

Model-Requests.txt +17 -0
README.md +8 -3

Model-Requests.txt ADDED Viewed

	@@ -0,0 +1,17 @@

+Request: aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-32K
+Model name:
+llama3-8B-DarkIdol-2.2-Uncensored-1048K
+Model link:
+https://huggingface.co/aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K
+Brief description:
+The module combination has been readjusted to better fulfill various roles and has been adapted for mobile phones.
+Uncensored 1048K
+An image/direct image link to represent the model (square shaped):
+![image/png](https://huggingface.co/aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K/resolve/main/llama3-8B-DarkIdol-2.2-Uncensored-1048K.png)
+---
+Thank you very much

README.md CHANGED Viewed

@@ -17,6 +17,11 @@ tags:
 ## Why 1048K?
 Due to the optimization of the preferred model, its performance is excellent across the range of 2000-1048K. Personal usage scenarios, such as 8186, 32K, etc., are insufficient. My primary role involves managing virtual idol Twitter accounts and assisting with singing, etc. A good conversation can be very lengthy, and sometimes even 32K is not enough. Imagine having a heated chat with your virtual girlfriend, only for it to abruptly cut off—that feeling is too painful. What kind of graphics card is needed? For investing in a girlfriend, would a 4090 or higher be more of a question?
 # Model Description:
 The module combination has been readjusted to better fulfill various roles and has been adapted for mobile phones.
@@ -35,7 +40,7 @@ The module combination has been readjusted to better fulfill various roles and h
 # Chang Log
 ### 2024-07-01
-- add writer
 - Optimize Japanese
 - Optimize logical processing
 - More humanized
@@ -240,7 +245,7 @@ This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522)
 ### Models Merged
 The following models were included in the merge:
-* [aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K](https://huggingface.co/aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K)
 * [TheBossLevel123/Llama3-Toxic-8B-Float16](https://huggingface.co/TheBossLevel123/Llama3-Toxic-8B-Float16)
 ### Configuration
@@ -251,7 +256,7 @@ The following YAML configuration was used to produce this model:
 models:
 #uncensored
   - model: TheBossLevel123/Llama3-Toxic-8B-Float16
-  - model: aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K
   - model: ./llama3-8B-DarkIdol-2.2-Uncensored-1048K-b
 merge_method: model_stock
 base_model: ./llama3-8B-DarkIdol-2.2-Uncensored-1048K-b

 ## Why 1048K?
 Due to the optimization of the preferred model, its performance is excellent across the range of 2000-1048K. Personal usage scenarios, such as 8186, 32K, etc., are insufficient. My primary role involves managing virtual idol Twitter accounts and assisting with singing, etc. A good conversation can be very lengthy, and sometimes even 32K is not enough. Imagine having a heated chat with your virtual girlfriend, only for it to abruptly cut off—that feeling is too painful. What kind of graphics card is needed? For investing in a girlfriend, would a 4090 or higher be more of a question?
+## my virtual idol Twitter
+- https://x.com/aifeifei799
+- https://x.com/aifeifei798
 # Model Description:
 The module combination has been readjusted to better fulfill various roles and has been adapted for mobile phones.
 # Chang Log
 ### 2024-07-01
+- add writer and more RP
 - Optimize Japanese
 - Optimize logical processing
 - More humanized
 ### Models Merged
 The following models were included in the merge:
+* [aifeifei798/llama3-8B-DarkIdol-2.1-Uncensored-1048K](https://huggingface.co/aifeifei798/llama3-8B-DarkIdol-2.1-Uncensored-1048K)
 * [TheBossLevel123/Llama3-Toxic-8B-Float16](https://huggingface.co/TheBossLevel123/Llama3-Toxic-8B-Float16)
 ### Configuration
 models:
 #uncensored
   - model: TheBossLevel123/Llama3-Toxic-8B-Float16
+  - model: aifeifei798/llama3-8B-DarkIdol-2.1-Uncensored-1048K
   - model: ./llama3-8B-DarkIdol-2.2-Uncensored-1048K-b
 merge_method: model_stock
 base_model: ./llama3-8B-DarkIdol-2.2-Uncensored-1048K-b