rhoahndur/retrosynthesis-qwen3-4b-gguf

Retrosynthesis Qwen3-4B GGUF

Q4_K_M quantized version of rhoahndur/retrosynthesis-qwen3-4b for CPU inference via llama.cpp.

Size: 2.5 GB Quantization: Q4_K_M

Usage with llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="rhoahndur/retrosynthesis-qwen3-4b-gguf",
    filename="retrosynthesis-qwen3-4b-Q4_K_M.gguf",
    n_ctx=512,
)
output = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a retrosynthesis expert. Output ONLY reactant SMILES separated by dots."},
        {"role": "user", "content": "Predict the reactants for: CC(=O)Oc1ccccc1C(=O)O"}
    ],
    max_tokens=256,
    temperature=0.7,
)
print(output["choices"][0]["message"]["content"])

Demo

Retrosynthesis AI