Add model card

README.md

@@ -0,0 +1,236 @@
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- Qwen/Qwen3.6-27B
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- climate
+- climate-change
+- wind-energy
+- climate-discourse
+- opposition-detection
+- classification
+- qwen3
+- fine-tuned
+- cards
+- joint
+- quantized
+- fp8
+- image-text-to-text
+- multimodal
+- vision-language
+datasets:
+- C3DS/cards_sft_dataset
+---
+
+# CARDS-Wind-Qwen3.6-27B-FP8
+
+**Joint** climate-discourse classifier — a single Qwen3.6-27B backbone fine-tuned on **CARDS + Wind concatenated**, then FP8-dynamic quantized. One model handles both tasks:
+
+- **CARDS** — classification of climate-contrarian claims under the [Coan et al. (2025)](https://doi.org/10.1038/s44458-025-00029-z) hierarchical taxonomy.
+- **Wind** — three-level wind-energy opposition classification (detection / frames / claims).
+
+The model picks the right task from the system prompt: pass the CARDS system prompt for CARDS-style output, the Wind system prompt for Wind-style output. Same weights, same chat template.
+
+This is the FP8 deployment variant — ~27 GB on disk, fits on a single A100/H100/H200.
+
+## Why a joint model
+
+- **One checkpoint, two tasks.** Saves disk + VRAM + ops complexity if you serve both classifiers in production.
+- **Same recipe, additive cost.** Trained with `cards/ft/train.py --joint` — Unsloth + LoRA r=16 α=16, 3 epochs, lr=2e-4 — identical to the cards FT models, with the wind dataset added to the training mix.
+- **Wind paper is forthcoming.** This model accompanies a forthcoming C3DS paper on wind-energy opposition discourse — research-preview status until the paper is out.
+
+## Results (Wind test set)
+
+Evaluated on the wind-opposition test set (773 rows, 436 opposition-positive). Compared with the wind-only sibling, the BF16 joint model, and frontier APIs:
+
+### Detection (binary)
+
+| Metric | Windy-27B FP8 | CARDS-Wind-27B (BF16) | **CARDS-Wind-27B (FP8 — this model)** | Claude Opus 4.7 | GPT-5.5 |
+|---|---|---|---|---|---|
+| Precision | 0.877 | 0.863 | **0.866** | 0.896 | **0.927** |
+| Recall | 0.920 | 0.911 | **0.917** | 0.890 | 0.846 |
+| F1 | **0.898** | 0.886 | **0.891** | 0.893 | 0.885 |
+
+### Samples F1 (multi-label frame / claim accuracy)
+
+| View | Windy-27B FP8 | CARDS-Wind-27B (BF16) | **CARDS-Wind-27B (FP8)** | Claude Opus 4.7 | GPT-5.5 |
+|---|---|---|---|---|---|
+| Frames — all rows | 0.787 | 0.770 | **0.772** | 0.791 | 0.792 |
+| Frames — opposition only | 0.751 | 0.736 | **0.739** | 0.734 | 0.697 |
+| Claims — all rows | 0.755 | 0.733 | **0.738** | 0.754 | 0.745 |
+| Claims — opposition only | 0.694 | 0.668 | **0.677** | 0.667 | 0.614 |
+
+- **Joint training costs ~0.007 detection F1** vs the wind-only `Windy-27B-FP8` (0.891 vs 0.898) — small but real.
+- Still **ties or beats Claude Opus 4.7 on detection F1** (0.891 vs 0.893) and beats it on frames-opposition-only and claims-opposition-only samples F1.
+- **FP8 ≈ BF16** on every metric — quantization is effectively free here.
+- Zero parse failures on 773 test items.
+
+### CARDS test set
+
+CARDS-side metrics for the joint model are **not separately reported** in this release — the cards-only sibling [`C3DS/CARDS-Qwen3.6-27B`](https://huggingface.co/C3DS/CARDS-Qwen3.6-27B) is the canonical reference for CARDS test-set numbers (samples F1 = 0.893 at L1, ties Opus 4.6). The joint model uses an identical CARDS training setup; expect comparable performance, but treat as approximate until reproduced.
+
+## Usage
+
+### Routing between the two tasks
+
+The model picks its behaviour from the system prompt. Both prompts are bundled in this repo:
+
+- [`cards_prompts.json`](./cards_prompts.json) — `slim_system_instruction` (CARDS) + `cot_trigger`
+- [`wind_prompts.json`](./wind_prompts.json) — `slim_system_instruction` (Wind)
+
+### With vLLM
+
+```bash
+vllm serve C3DS/CARDS-Wind-Qwen3.6-27B-FP8 \
+  --port 8000 \
+  --max-model-len 4096 \
+  --enable-prefix-caching \
+  --kv-cache-dtype fp8 \
+  --served-model-name CARDS-Wind-Qwen3.6-27B
+```
+
+```python
+import json
+from huggingface_hub import hf_hub_download
+from openai import OpenAI
+
+cards = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Qwen3.6-27B-FP8", "cards_prompts.json")))
+wind  = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Qwen3.6-27B-FP8", "wind_prompts.json")))
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")
+
+def classify_cards(text):
+    resp = client.chat.completions.create(
+        model="CARDS-Wind-Qwen3.6-27B",
+        messages=[
+            {"role": "system", "content": cards["slim_system_instruction"]},
+            {"role": "user",   "content": f"### Text:\n{text}\n\n{cards['cot_trigger']}"},
+        ],
+        temperature=0, max_tokens=4000,
+    )
+    return resp.choices[0].message.content
+
+def classify_wind(text):
+    resp = client.chat.completions.create(
+        model="CARDS-Wind-Qwen3.6-27B",
+        messages=[
+            {"role": "system", "content": wind["slim_system_instruction"]},
+            {"role": "user",   "content": text},
+        ],
+        temperature=0, max_tokens=4000,
+    )
+    return resp.choices[0].message.content
+```
+
+Both modes produce a `<think>…</think>` reasoning trace followed by a YAML block. CARDS output is `categories: [...]`; Wind output is `opposition_detected`, `frames`, `claims`.
+
+
+### Multimodal — image + text
+
+The base Qwen3.6-27B supports image inputs via the OpenAI-compatible
+`image_url` content part, and this fine-tune preserves that capability for
+both tasks. Switch tasks by switching the system prompt — CARDS prompt for
+CARDS-style output, Wind prompt for Wind-style output — and pass an image
+(with or without caption text) alongside.
+
+Serve vLLM with multimodal flags enabled:
+
+```bash
+vllm serve C3DS/CARDS-Wind-Qwen3.6-27B-FP8 \
+  --port 8000 \
+  --max-model-len 8192 \
+  --trust-remote-code \
+  --limit-mm-per-prompt image=4 \
+  --enable-prefix-caching \
+  --kv-cache-dtype fp8 \
+  --served-model-name CARDS-Wind-Qwen3.6-27B
+```
+
+```python
+import base64, json, mimetypes
+from pathlib import Path
+from huggingface_hub import hf_hub_download
+from openai import OpenAI
+
+cards = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Qwen3.6-27B-FP8", "cards_prompts.json")))
+wind  = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Qwen3.6-27B-FP8", "wind_prompts.json")))
+
+def image_part(path):
+    p = Path(path)
+    mime = mimetypes.guess_type(p)[0] or "image/png"
+    b64 = base64.b64encode(p.read_bytes()).decode()
+    return {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{b64}"}}
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")
+
+# Wind classification on an image:
+resp = client.chat.completions.create(
+    model="CARDS-Wind-Qwen3.6-27B",
+    messages=[
+        {"role": "system", "content": wind["slim_system_instruction"]},
+        {"role": "user", "content": [
+            {"type": "text", "text": "Read the image and any caption below; classify the wind-opposition framing depicted."},
+            image_part("screenshot.png"),
+            {"type": "text", "text": "### Caption:\n<optional caption>"},
+        ]},
+    ],
+    temperature=0,
+    max_tokens=4000,
+)
+print(resp.choices[0].message.content)
+```
+
+## Training & Quantization
+
+### Joint fine-tuning
+
+- **Base model:** `Qwen/Qwen3.6-27B`
+- **Method:** LoRA (rank 16, α 16, dropout 0) on `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`, then merged into base weights
+- **Datasets:** [`C3DS/cards_sft_dataset`](https://huggingface.co/datasets/C3DS/cards_sft_dataset) (CARDS RECoT messages) + the wind RECoT messages corpus (forthcoming release)
+- **Mix:** datasets concatenated row-wise — no balancing or task-token; the model learns to pick its task from the system prompt
+- **Framework:** Unsloth + TRL `SFTTrainer`, invoked via `cards/ft/train.py --joint`
+- **Hyperparameters:** 3 epochs, `per_device_train_batch_size=1`, `gradient_accumulation_steps=8`, `lr=2e-4`, cosine schedule, 10 warmup steps, `max_seq_length=8192`, `adamw_8bit`, `bf16`
+
+### FP8 quantization
+
+- **Scheme:** `fp8_e4m3` dynamic per-channel quantization (weights only). Activations stay in BF16; no calibration data required.
+- **Targets:** linear layers in transformer blocks; `lm_head` left in BF16.
+- **Tool:** `llmcompressor` with `QuantizationModifier(targets="Linear", scheme="FP8_DYNAMIC")` applied to the merged BF16 joint checkpoint.
+
+## Limitations
+
+- **Forthcoming Wind paper.** Wind-side methodology, codebook, and dataset details are pending publication.
+- **CARDS-side metrics not separately re-evaluated** for the joint model — refer to the cards-only sibling for the canonical CARDS test-set numbers.
+- **Joint training trade-off.** Detection F1 on Wind is marginally lower than the wind-only `Windy-Qwen3.5-27B-FP8` (0.891 vs 0.898). Use the dedicated wind model if absolute wind detection F1 is the priority; use this one if you need both tasks from a single backbone.
+- **Thinking tokens.** Training used `enable_thinking=True`. Parse output after `</think>` or disable thinking at inference.
+
+## Related models
+
+- [`C3DS/CARDS-Qwen3.6-27B`](https://huggingface.co/C3DS/CARDS-Qwen3.6-27B) — CARDS-only Qwen3.6-27B sibling (same backbone, CARDS-only training).
+- [`C3DS/Windy-Qwen3.5-27B-FP8`](https://huggingface.co/C3DS/Windy-Qwen3.5-27B-FP8) — Wind-only 27B FP8 sibling (Qwen3.5 backbone).
+
+## Citation
+
+For the CARDS side of this model, please cite:
+
+```bibtex
+@article{coan2025cards,
+  title   = {Large language model reveals an increase in climate contrarian speech in the United States Congress},
+  author  = {Coan, Travis G. and Malla, Ranadheer and Nanko, Mirjam O. and Kattrup, William and Roberts, J. Timmons and Cook, John and Boussalis, Constantine},
+  journal = {Communications Sustainability},
+  volume  = {1},
+  pages   = {37},
+  year    = {2025},
+  doi     = {10.1038/s44458-025-00029-z}
+}
+```
+
+A wind-side citation will be added when the corresponding paper is published.
+
+## License
+
+Apache 2.0, inherited from Qwen3.6-27B.