Instructions to use yobo2u/DeepSeek-V4-Flash-0731-A100 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yobo2u/DeepSeek-V4-Flash-0731-A100 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="yobo2u/DeepSeek-V4-Flash-0731-A100")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("yobo2u/DeepSeek-V4-Flash-0731-A100")
model = AutoModelForCausalLM.from_pretrained("yobo2u/DeepSeek-V4-Flash-0731-A100", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use yobo2u/DeepSeek-V4-Flash-0731-A100 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "yobo2u/DeepSeek-V4-Flash-0731-A100"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yobo2u/DeepSeek-V4-Flash-0731-A100",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/yobo2u/DeepSeek-V4-Flash-0731-A100

SGLang

How to use yobo2u/DeepSeek-V4-Flash-0731-A100 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "yobo2u/DeepSeek-V4-Flash-0731-A100" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yobo2u/DeepSeek-V4-Flash-0731-A100",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "yobo2u/DeepSeek-V4-Flash-0731-A100" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yobo2u/DeepSeek-V4-Flash-0731-A100",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use yobo2u/DeepSeek-V4-Flash-0731-A100 with Docker Model Runner:
```
docker model run hf.co/yobo2u/DeepSeek-V4-Flash-0731-A100
```

DeepSeek-V4-Flash-0731-A100

Converted DeepSeek-V4-Flash-0731 weights for deployment on NVIDIA A100 / A800 (SM80) GPUs.

For deployment configuration, installation steps, launch scripts and benchmarks, see:
GitHub: yobo2u/DeepSeek-V4-Flash-0731-A100

SM90+ users (H100 / H800 / H20) should use the official upstream weights directly — this repository is not needed.

About This Conversion

Base model: deepseek-ai/DeepSeek-V4-Flash-0731

The SM80 architecture used by A100/A800 does not support the native FP8 execution path that the original model targets on newer GPUs. The weights in this repository were converted offline so they can run on SM80 GPUs together with a dedicated SGLang 0.5.16 monkeypatch:

Non-expert weights: converted to BF16
MoE expert weights: kept in MXFP4
Shards: 48 safetensors files
Total size: approximately 173 GB (decimal)
Context length: up to 1,048,576 tokens
Recommended inference framework: SGLang 0.5.16
Recommended hardware: 8× A100 80GB or 8× A800 80GB, TP=8
Speculative decoding: DSpark supported

This repository provides converted model weights only. The full runtime environment, pinned versions, monkeypatch source, launch arguments and benchmark scripts are maintained in the GitHub configuration repository.

Deployment

Do not reuse launch commands written for stock vLLM or newer-architecture GPUs. SM80 deployment requires a pinned SGLang version plus the A100 monkeypatch:

Full deployment guide: https://github.com/yobo2u/DeepSeek-V4-Flash-0731-A100
Launch script: https://github.com/yobo2u/DeepSeek-V4-Flash-0731-A100/blob/main/launch.sh
Benchmark script: https://github.com/yobo2u/DeepSeek-V4-Flash-0731-A100/blob/main/benchmark_dspark_full.py
Benchmark reports: https://github.com/yobo2u/DeepSeek-V4-Flash-0731-A100/tree/main/benchmarks
A100 monkeypatch: https://github.com/yaleyoou/deepseek-v4-a100-sglang-v0516
SGLang: 0.5.16, pinned commit fdebc938f7f4d16fe6b9f55dcd9a767cf0899ea1

Download

hf download yobo2u/DeepSeek-V4-Flash-0731-A100 \
  --local-dir /path/to/models/DeepSeek-V4-Flash-0731-A100

For a model of this size, use low download concurrency and resumable transfers, then verify that all 48 shards referenced by model.safetensors.index.json are present.

The same weights are also published on ModelScope: yobo2u/DeepSeek-V4-Flash-0731-A100

Verified Configuration

Item	Value
GPU	8× NVIDIA A800-SXM4-80GB (SM80)
Tensor parallel	8
SGLang	0.5.16 (commit `fdebc938`)
Dtype	BF16 + MXFP4 experts
Serving port	8082
Context length	1,048,576
`--mem-fraction-static`	0.85
`--chunked-prefill-size`	32,768
`--max-running-requests`	16
Speculative decoding	DSpark

Measured Performance

Measured on the verified configuration above (config C), covering 1K / 4K / 16K / 64K / 128K context × concurrency 1 / 8 / 16 × 2 full repeats = 30 groups, 900 requests, 0 errors and 0 timeouts:

Metric	Value
Single-stream decode (concurrency 1, 1000/TPOT)	~217 tok/s
Single-stream aggregate throughput (concurrency 1)	~205 tok/s (166–222)
Aggregate throughput (concurrency 16)	~1,232 tok/s (peak 1,334)
DSpark accept rate	~60% (0.40–0.86, increases with context)
DSpark accept length	~4.01 (3.01–5.30)
TTFT (concurrency 1)	~296 ms
Peak VRAM	~51.9 GB/GPU

Accept rate and accept length rise substantially with context length: about 0.40 / 3.0 at 1K, and about 0.82 / 5.1 at 128K. The table reports means across all 30 groups; per-group values and raw data are in the JSON files under the GitHub benchmarks directory.

Tuning Findings

A three-way factor decomposition over mem-fraction-static and chunked-prefill-size (30 groups / 900 requests each):

Config	mem-fraction	chunked-prefill	Mean throughput	Peak VRAM
A	0.85	16,384	746.2 tok/s	48.58 GiB
B	0.90	32,768	759.8 tok/s	53.80 GiB
C (recommended)	0.85	32,768	766.4 tok/s	51.92 GiB

Keep mem-fraction-static at 0.85. Raising it to 0.90 squeezes the KV cache headroom, dropping the 128K accept rate from 0.80 to 0.69 and costing roughly 11% throughput at long context.
Raising chunked-prefill-size from 16,384 to 32,768 helps at short-to-medium context and higher concurrency, and is not the cause of the 128K regression.
Full factor-decomposition report (HTML and PDF) is available in the GitHub benchmarks directory.

Performance varies with input length, concurrency, driver, CUDA version, interconnect topology and the specific A100/A800 SKU. The GitHub repository is the source of truth for methodology and results.

Limitations and Notes

These are community weights converted for SM80 deployment — not an official DeepSeek model release.
Running them depends on a dedicated monkeypatch. Downloading the weights alone does not make them loadable by Transformers, vLLM, or unmodified SGLang.
--quantization fp8 is part of how the current SGLang monkeypatch path is wired up; it does not mean every weight is stored in native FP8.
Follow the version matrix and launch arguments in the GitHub documentation to avoid SGLang API and kernel incompatibilities.
For model capabilities, recommended sampling parameters, chat template format and usage policy, refer to the original model repository.

Credits

Original model and architecture: DeepSeek-AI
SM80 monkeypatch: yaleyoou/deepseek-v4-a100-sglang-v0516
Original patch: Qeeweew/deepseek-v4-a100-sglang
Inference framework: SGLang
Deployment configuration: yobo2u/DeepSeek-V4-Flash-0731-A100

License

Released under the MIT License, following the original repository. Users must also comply with the upstream model repository's license terms and applicable laws and regulations.

Downloads last month: 288

Safetensors

Model size

304B params

Tensor type

BF16

F32

I64

Model tree for yobo2u/DeepSeek-V4-Flash-0731-A100

Base model

deepseek-ai/DeepSeek-V4-Flash-0731

Quantized

(194)

this model