Tidalluna/DeepSeek-V4-Flash-0731-Tina-CQ4KM

DeepSeek-V4-Flash-0731 Tina CQ4KM

This repository contains the Tina CQ4KM quantized main model derived from deepseek-ai/DeepSeek-V4-Flash-0731. It targets low-latency, single-sequence local inference with Tina Engine on an single NVIDIA V100 32 GiB GPU (SM70) and CUDA 12.8.

Compatibility

The weight shards use Tina's custom .cqtensors container. They are not Safetensors or GGUF files and cannot be loaded directly by Transformers, llama.cpp, vLLM, SGLang, or TensorRT-LLM. A CQTensors-compatible Tina Engine release is required.

llama.cpp / GGUF incompatibility

This artifact cannot currently be used by llama.cpp. Renaming .cqtensors to .gguf, .bin, or another extension does not make it compatible. There are three independent incompatibility layers:

1. Container and quantization layout: CQTensors stores Tina's role-specific Q4_K/Q5_K/Q6_K assignments, tensor metadata and placement identity. It is neither a GGUF container nor a llama.cpp quantization layout.
2. Routing semantics: the first three MoE layers use the official ffn.gate.tid2eid token-ID-to-expert table (num_hash_layers = 3) instead of an ordinary learned top-k router. The remaining layers use the model's sqrt-softplus, bias-assisted Top-6 routing contract.
3. Model graph: DeepSeek V4 Flash uses four-way Hyper-Connections with Sinkhorn mixing (hc_mult = 4) and a layer-dependent MLA schedule with raw, Ratio4 and Ratio128 compression. Tina's kernels and KV owner implement these exact semantics; a conventional Llama/DeepSeek GGUF graph does not.

A real llama.cpp port would therefore require a new DeepSeek V4 architecture implementation, tensor mapping, KV/cache logic and kernels. A header converter or filename change alone cannot work.

The included inference/ directory is DeepSeek's upstream reference implementation for the original checkpoint. It is retained for model-semantic reference and does not load these .cqtensors shards. The included encoding/ directory is the upstream message encoder and output parser and is the authoritative chat-format reference for this model family.

Artifact Layout

• model-00001-of-00048.cqtensors through model-00045-of-00048.cqtensors: 45 main-model shards
• model-manifest.json: tensor inventory, formats, extents, numerical samples, provenance, and per-shard SHA-256 values
• SHA256SUMS: independent file verification list
• Official tokenizer, generation configuration, encoding code, and reference inference code from the upstream 0731 release

The main artifact contains 34,244 tensors and 173,543,251,036 logical payload bytes. Official shard numbering is deliberately retained. Source shards 46-48 contain the optional DSpark/MTP speculative-decoding module and are not part of this initial main-model publication.

Quantization Scope

The two shard ranges have deliberately different qualification histories:

• Shards 1-45 (included here): calibration-guided CQ4KM conversion. The official 0731 packed-FP4/E8M0 and FP8/E8M0 tensors were decoded to a finite floating-point reference. Deterministically selected source blocks were then compared against Q4_K, Q5_K and Q6_K candidates before the role-based precision plan was frozen. No scale, route, output, or performance result was reused from the preview model.
• Shards 46-48 (not included yet): source-bound direct conversion. These official DSpark/MTP tensors are separately full-hash-bound and converted with a frozen CQ4KM role mapping, without applying the sampled 1-45 delta- calibration gate. "Direct" means conversion from the official 0731 source, not a byte-for-byte copy: packed FP4/FP8 values are still decoded and re-encoded into the target runtime formats.

The 30 token/s result in this model card covers only calibrated main-model shards 1-45. It makes no DSpark/MTP acceptance-rate or speculative-throughput claim for shards 46-48.

Provenance

• Source model: deepseek-ai/DeepSeek-V4-Flash-0731
• Source revision: 7872f01b1d1fe23eabc4c98b48bffcef5a386062
• Quantization policy: DeepSeek-V4-Flash-0731-CQ4KM-V100-v1
• Container: CQTensors, version 1, magic CQTNS1\0\0
• Artifact content identity: de2aaf1431ff79ef55126a9c9f51cbe4fb51677825faea25e39a398f8444d231
• Manifest file SHA-256: 7d388b0351caee57193f1725f682a79bb607b379b89b80c386363227e5b004e5

This derivative is not affiliated with or endorsed by DeepSeek.

Current Qualification

The result above was produced by Tina Engine's native Rust/CUDA 12.8 fast decode path, not by Transformers and not by an aggregate-batching estimate. The qualification used:

• GPU: one NVIDIA V100 32 GiB / SM70 (single GPU)
• CPU: AMD Ryzen 7 7800X3D (8 cores / 16 threads)
• Host memory: 32 GiB DDR5-6000 (6000 MT/s effective rate)
• Model storage: 1 TB MLC NVMe SSD (local artifact storage)
• CUDA: 12.8
• Native Tina Engine release build compiled for sm_70
• Main-model shards 1-45 only; DSpark/MTP was disabled
• Batch size 1 and one active sequence; no continuous batching
• A sealed 1,158-slot routed-expert placement installed before timed decode
• Fixed 1530 MHz graphics clock and observed 1107 MHz memory clock
• Three independent 32-sample processes
• 33 generated tokens per process, yielding 32 steady inter-token samples
• Throughput: 30.063488, 30.534844, and 30.457034 token/s
• Process CV: 1.576007%, 2.036693%, and 1.966784%
• Identical 33-token chain and execution receipt across all three processes

These are local hardware-specific engineering measurements, not a portable performance guarantee. Aggregate batching is excluded. Fast mode prioritizes user-visible latency and does not claim bit-exact parity with the source checkpoint. Broad quality, perplexity, and downstream-task results are not yet published.

Tina Engine

Tina Engine is the custom Rust/CUDA inference runtime used for the qualification above. Its release tree is currently undergoing packaging, architecture and unsafe-boundary convergence. The inference-engine source is planned for public open-source release soon; this weight publication intentionally arrives first.

Download

hf download Pure156/DeepSeek-V4-Flash-0731-Tina-CQ4KM `
  --local-dir ./DeepSeek-V4-Flash-0731-Tina-CQ4KM

Keep all shard names unchanged and verify the download against SHA256SUMS or the hashes embedded in model-manifest.json.

License

The upstream model, weights, encoder, and reference inference code are redistributed under the included MIT License. Retain the copyright and permission notice when redistributing substantial portions of this artifact.