Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,43 @@
+---
+license: apache-2.0
+base_model: scottyjmp5/Legal-Qwen3.6-27B-Abliterated
+datasets:
+- scottyjmp5/courtlistener-legal-corpus
+language:
+- en
+pipeline_tag: image-text-to-text
+tags:
+- legal
+- caselaw
+- abliterated
+- vision
+- tool-calling
+- fp8
+- compressed-tensors
+- vllm
+---
+
+# Legal-Qwen3.6-27B-Abliterated-FP8
+
+FP8 W8A8 (dynamic per-token activation) quantization of
+[Legal-Qwen3.6-27B-Abliterated](https://huggingface.co/scottyjmp5/Legal-Qwen3.6-27B-Abliterated),
+produced with [llm-compressor](https://github.com/vllm-project/llm-compressor).
+
+- Text Linear layers are FP8; the vision tower, embeddings, and lm_head are kept full precision,
+  so vision, tool calling, and thinking mode are unaffected.
+- 29 GB (vs 52 GB bf16). Serves on a single 40GB+ GPU with vLLM (compressed-tensors is
+  detected automatically). Fits 2x24GB with tensor parallelism.
+- FP8 W8A8 uses precompiled CUTLASS kernels in vLLM: no custom toolchain needed.
+
+```bash
+vllm serve scottyjmp5/Legal-Qwen3.6-27B-Abliterated-FP8 \
+  --trust-remote-code --max-model-len 8192
+```
+
+Tip observed in testing: FP8's speed advantage over bf16 appears when CUDA graphs are enabled
+(the vLLM default). With enforce_eager it can be slightly slower than bf16; with graphs it was
+1.67x faster in our single-stream tests.
+
+See the [main model card](https://huggingface.co/scottyjmp5/Legal-Qwen3.6-27B-Abliterated) for
+training details, the retrieval (RAG) recommendation, and warnings (abliterated base; not legal
+advice).