Instructions to use FINAL-Bench/AETHER-7B-7Attn-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FINAL-Bench/AETHER-7B-7Attn-base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FINAL-Bench/AETHER-7B-7Attn-base", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("FINAL-Bench/AETHER-7B-7Attn-base", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FINAL-Bench/AETHER-7B-7Attn-base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FINAL-Bench/AETHER-7B-7Attn-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/AETHER-7B-7Attn-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/FINAL-Bench/AETHER-7B-7Attn-base

SGLang

How to use FINAL-Bench/AETHER-7B-7Attn-base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FINAL-Bench/AETHER-7B-7Attn-base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/AETHER-7B-7Attn-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FINAL-Bench/AETHER-7B-7Attn-base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/AETHER-7B-7Attn-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use FINAL-Bench/AETHER-7B-7Attn-base with Docker Model Runner:
```
docker model run hf.co/FINAL-Bench/AETHER-7B-7Attn-base
```

AETHER-7B-7Attn-base / latin_square.svg

SeaWolf-AI

Upload latin_square.svg with huggingface_hub

b4e1b81 verified 10 days ago

preview code

Raw

History Blame Contribute Delete

16.1 kB

	<svg xmlns="http://www.w3.org/2000/svg" width="634" height="700" viewBox="0 0 634 700" font-family="Inter,Segoe UI,sans-serif">
	<rect width="634" height="700" fill="#0d1117"/>
	<text x="317.0" y="42" fill="#e6edf3" font-size="25" font-weight="700" text-anchor="middle">AETHER-7B-7Attn — Attention Layout</text>
	<text x="317.0" y="70" fill="#8b949e" font-size="14" text-anchor="middle">7 heterogeneous mechanisms · 7x7 Latin square · 49 layers</text>
	<text x="44" y="138.0" fill="#8b949e" font-size="12" text-anchor="end">R1</text>
	<text x="95.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C1</text>
	<text x="44" y="212.0" fill="#8b949e" font-size="12" text-anchor="end">R2</text>
	<text x="169.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C2</text>
	<text x="44" y="286.0" fill="#8b949e" font-size="12" text-anchor="end">R3</text>
	<text x="243.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C3</text>
	<text x="44" y="360.0" fill="#8b949e" font-size="12" text-anchor="end">R4</text>
	<text x="317.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C4</text>
	<text x="44" y="434.0" fill="#8b949e" font-size="12" text-anchor="end">R5</text>
	<text x="391.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C5</text>
	<text x="44" y="508.0" fill="#8b949e" font-size="12" text-anchor="end">R6</text>
	<text x="465.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C6</text>
	<text x="44" y="582.0" fill="#8b949e" font-size="12" text-anchor="end">R7</text>
	<text x="539.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C7</text>
	<rect x="61" y="99" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
	<text x="95.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
	<text x="95.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L1</text>
	<rect x="135" y="99" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
	<text x="169.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
	<text x="169.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L2</text>
	<rect x="209" y="99" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
	<text x="243.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
	<text x="243.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L3</text>
	<rect x="283" y="99" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
	<text x="317.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
	<text x="317.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L4</text>
	<rect x="357" y="99" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
	<text x="391.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
	<text x="391.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L5</text>
	<rect x="431" y="99" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
	<text x="465.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
	<text x="465.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L6</text>
	<rect x="505" y="99" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
	<text x="539.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
	<text x="539.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L7</text>
	<rect x="61" y="173" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
	<text x="95.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
	<text x="95.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L8</text>
	<rect x="135" y="173" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
	<text x="169.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
	<text x="169.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L9</text>
	<rect x="209" y="173" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
	<text x="243.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
	<text x="243.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L10</text>
	<rect x="283" y="173" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
	<text x="317.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
	<text x="317.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L11</text>
	<rect x="357" y="173" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
	<text x="391.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
	<text x="391.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L12</text>
	<rect x="431" y="173" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
	<text x="465.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
	<text x="465.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L13</text>
	<rect x="505" y="173" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
	<text x="539.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
	<text x="539.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L14</text>
	<rect x="61" y="247" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
	<text x="95.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
	<text x="95.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L15</text>
	<rect x="135" y="247" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
	<text x="169.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
	<text x="169.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L16</text>
	<rect x="209" y="247" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
	<text x="243.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
	<text x="243.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L17</text>
	<rect x="283" y="247" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
	<text x="317.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
	<text x="317.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L18</text>
	<rect x="357" y="247" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
	<text x="391.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
	<text x="391.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L19</text>
	<rect x="431" y="247" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
	<text x="465.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
	<text x="465.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L20</text>
	<rect x="505" y="247" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
	<text x="539.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
	<text x="539.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L21</text>
	<rect x="61" y="321" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
	<text x="95.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
	<text x="95.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L22</text>
	<rect x="135" y="321" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
	<text x="169.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
	<text x="169.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L23</text>
	<rect x="209" y="321" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
	<text x="243.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
	<text x="243.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L24</text>
	<rect x="283" y="321" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
	<text x="317.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
	<text x="317.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L25</text>
	<rect x="357" y="321" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
	<text x="391.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
	<text x="391.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L26</text>
	<rect x="431" y="321" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
	<text x="465.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
	<text x="465.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L27</text>
	<rect x="505" y="321" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
	<text x="539.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
	<text x="539.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L28</text>
	<rect x="61" y="395" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
	<text x="95.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
	<text x="95.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L29</text>
	<rect x="135" y="395" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
	<text x="169.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
	<text x="169.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L30</text>
	<rect x="209" y="395" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
	<text x="243.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
	<text x="243.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L31</text>
	<rect x="283" y="395" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
	<text x="317.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
	<text x="317.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L32</text>
	<rect x="357" y="395" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
	<text x="391.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
	<text x="391.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L33</text>
	<rect x="431" y="395" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
	<text x="465.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
	<text x="465.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L34</text>
	<rect x="505" y="395" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
	<text x="539.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
	<text x="539.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L35</text>
	<rect x="61" y="469" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
	<text x="95.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
	<text x="95.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L36</text>
	<rect x="135" y="469" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
	<text x="169.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
	<text x="169.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L37</text>
	<rect x="209" y="469" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
	<text x="243.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
	<text x="243.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L38</text>
	<rect x="283" y="469" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
	<text x="317.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
	<text x="317.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L39</text>
	<rect x="357" y="469" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
	<text x="391.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
	<text x="391.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L40</text>
	<rect x="431" y="469" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
	<text x="465.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
	<text x="465.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L41</text>
	<rect x="505" y="469" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
	<text x="539.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
	<text x="539.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L42</text>
	<rect x="61" y="543" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
	<text x="95.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
	<text x="95.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L43</text>
	<rect x="135" y="543" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
	<text x="169.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
	<text x="169.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L44</text>
	<rect x="209" y="543" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
	<text x="243.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
	<text x="243.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L45</text>
	<rect x="283" y="543" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
	<text x="317.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
	<text x="317.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L46</text>
	<rect x="357" y="543" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
	<text x="391.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
	<text x="391.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L47</text>
	<rect x="431" y="543" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
	<text x="465.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
	<text x="465.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L48</text>
	<rect x="505" y="543" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
	<text x="539.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
	<text x="539.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L49</text>
	<text x="58" y="644" fill="#8b949e" font-size="13">Each of the 7 mechanisms (A-G) appears exactly once per row and per column.</text>
	<text x="58" y="666" fill="#6e7681" font-size="12">L1-L49 = layer index. Mechanism identities and assignment recipe are proprietary.</text>
	</svg>