CyberGPT-Q4_K_M
CyberGPT is a cybersecurity-focused language model based on Qwen3-4B, fine-tuned to provide accurate, educational, and technically clear responses to cybersecurity-related questions.
This repository contains the Q4_K_M GGUF quantized version of CyberGPT for efficient local inference using llama.cpp.
Model Details
| Property | Value |
|---|---|
| Model | CyberGPT |
| Base Model | Qwen3-4B |
| Architecture | Qwen3ForCausalLM |
| Parameters | ~4B |
| Format | GGUF |
| Quantization | Q4_K_M |
| Model Size | ~2.5 GB |
| Primary Domain | Cybersecurity |
| Language | English |
| Inference Framework | llama.cpp |
| GPU Backend Tested | Vulkan |
About CyberGPT
CyberGPT is designed as a specialized cybersecurity assistant for educational and technical use.
The model is intended to help with topics including:
- Network security
- Cryptography
- Authentication and authorization
- Web security
- Malware concepts
- Vulnerability concepts
- Security monitoring
- Incident response
- Network attacks and defenses
- General cybersecurity concepts
The goal of the project is to investigate how domain-specific fine-tuning can improve the usefulness of a general-purpose language model for cybersecurity-related tasks.
Base Model
CyberGPT is based on:
Qwen3-4B
The base model was downloaded from the Qwen3 model family and converted to GGUF format before quantization.
The original model architecture contains:
- 36 transformer layers
- Hidden size: 2560
- 32 attention heads
- 8 key/value heads
- Vocabulary size: 151,936
- Context capability: up to 262,144 tokens in the original configuration
For local deployment, this repository provides the quantized GGUF version.
Quantization
The released model uses:
Q4_K_M
Quantization significantly reduces model size compared with the original F16 representation while making the model practical for local inference.
The project produced:
Qwen3-4B-f16.gguf
↓
Qwen3-4B-Q4_K_M.gguf
The resulting CyberGPT Q4_K_M model is approximately 2.5 GB.
Running with llama.cpp
1. Download llama.cpp
Clone and build llama.cpp with Vulkan support:
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -S . -B build-vulkan \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_VULKAN=ON
cmake --build build-vulkan \
--config Release \
-j2 \
--target llama-cli llama-server
2. Download the model
Download the GGUF file from this repository and place it somewhere convenient.
For example:
~/llm-work/CyberGPT-Q4_K_M.gguf
3. Run CyberGPT from the command line
./build-vulkan/bin/llama-cli \
-m ~/llm-work/CyberGPT-Q4_K_M.gguf \
-c 2048 \
-n 256 \
-ngl 99
-ngl 99 allows llama.cpp to offload as many model layers as possible to the available GPU backend.
Running as an API Server
CyberGPT was tested using the llama.cpp server:
./build-vulkan/bin/llama-server \
-m ~/llm-work/CyberGPT-Q4_K_M.gguf \
-c 4096 \
-ngl 99 \
--host 127.0.0.1 \
--port 8080
The API becomes available at:
http://127.0.0.1:8080
For example, a compatible chat-completions endpoint can be accessed through:
/v1/chat/completions
Example
Prompt
What is the difference between symmetric and asymmetric encryption?
CyberGPT
Symmetric encryption uses the same key for both encryption and decryption,
while asymmetric encryption uses two different keys: a public key and a
private key.
Symmetric encryption is generally faster and is commonly used for encrypting
large amounts of data. Asymmetric cryptography is computationally more
expensive and is commonly used for key exchange, authentication, and digital
signatures.
Evaluation
CyberGPT was evaluated using a set of 20 cybersecurity questions covering fundamental security concepts.
The evaluation questions included topics such as:
- Symmetric vs asymmetric encryption
- Firewalls
- SQL injection
- Cross-site scripting (XSS)
- Authentication vs authorization
- Principle of least privilege
- Denial-of-service attacks
- IDS vs IPS
- Man-in-the-middle attacks
- Multi-factor authentication
- Hashing vs encryption
- Zero-day vulnerabilities
- Phishing
- VPNs
- TCP vs UDP
- Privilege escalation
- Malware
- Network segmentation
- Security logging and monitoring
- Incident response
Generation Results
CyberGPT successfully generated responses for:
Questions: 20
Successful: 20
Generation success: 100%
The evaluation pipeline stored the generated responses in JSON format for further analysis.
Important evaluation note
An automated LLM-as-a-judge scoring step was also performed.
The judge successfully produced parseable scores for 17 of the 20 questions. Three judge responses were truncated or returned invalid JSON.
Therefore, the current project does not claim a definitive 100% quality score from the automated judge.
The 20/20 figure above represents successful model response generation, not a 100% correctness score.
Performance
CyberGPT was tested locally using llama.cpp with Vulkan acceleration.
Example observed generation performance:
Generation: approximately 15–19 tokens/second
Actual performance depends on:
- GPU
- CPU
- Vulkan driver
- RAM/VRAM
- Context size
- Number of GPU-offloaded layers
- Prompt length
- Generation length
Project Pipeline
The overall development pipeline was:
Qwen3-4B
│
▼
Cybersecurity Fine-Tuning
│
▼
Fine-Tuned CyberGPT
│
▼
F16 GGUF Conversion
│
▼
Q4_K_M Quantization
│
▼
llama.cpp + Vulkan
│
▼
Local CyberGPT Inference
│
▼
20-Question Evaluation
│
▼
Evaluation Results
│
▼
Hugging Face Model Release
Intended Use
CyberGPT is intended primarily for:
- Cybersecurity education
- Learning security concepts
- Security terminology explanations
- Defensive security research
- Development and experimentation
- Local LLM experimentation
- Cybersecurity-focused AI research
Limitations
CyberGPT is a relatively small approximately 4B parameter model.
It may:
- Produce incorrect or incomplete information
- Hallucinate technical details
- Misinterpret ambiguous security questions
- Provide outdated information
- Perform worse on highly specialized security topics
- Require additional verification for security-critical decisions
Model outputs should therefore be treated as informational assistance rather than authoritative security guidance.
For production security decisions, generated information should be validated against trusted technical documentation, security advisories, standards, and other authoritative sources.
Responsible Use
Cybersecurity knowledge can be used for both defensive and offensive purposes.
CyberGPT should be used responsibly for:
- Education
- Authorized security testing
- Defensive security research
- Security engineering
- CTF and laboratory environments
- Vulnerability research conducted with appropriate authorization
Do not use the model to conduct unauthorized attacks, compromise systems, steal information, or cause harm.
Technical Stack
The project uses:
- Qwen3-4B
- GGUF
- llama.cpp
- Vulkan
- Python
- Hugging Face Hub
Files
The repository currently contains:
CyberGPT-Q4_K_M.gguf
This is the quantized model intended for local inference.
Future Work
Potential improvements include:
- Larger cybersecurity instruction datasets
- More comprehensive evaluation benchmarks
- Human evaluation
- Improved automated evaluation reliability
- Comparison against additional cybersecurity LLMs
- RAG integration with security documentation
- Tool-augmented cybersecurity assistance
- Further quantization and performance optimization
- Deployment through a web interface or API
Acknowledgements
This project builds upon the open-source Qwen3 model family and the llama.cpp inference ecosystem.
Thanks to the developers and open-source communities that make local LLM experimentation possible.
Disclaimer
This is an experimental cybersecurity-focused language model. The authors do not guarantee that generated information is complete, accurate, or suitable for security-critical applications.
Always verify security-related information before applying it to real systems.
- Downloads last month
- 66
4-bit