clarkkitchen22 commited on
Commit
ce47103
·
verified ·
1 Parent(s): d32ad7d

Upload Dadbot1.7b adapter and model card

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,172 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen3-1.7B
4
+ library_name: peft
5
+ pipeline_tag: text-generation
6
+ language:
7
+ - en
8
+ tags:
9
+ - qwen
10
+ - qwen3
11
+ - lora
12
+ - qlora
13
+ - peft
14
+ - text-generation
15
+ - conversational
16
+ - synthetic-data
17
+ - dadbot
18
+ datasets:
19
+ - clarkkitchen22/dadquotes5k
20
+ model-index:
21
+ - name: Dadbot1.7b
22
+ results:
23
+ - task:
24
+ type: text-generation
25
+ name: Text Generation
26
+ dataset:
27
+ name: dadquotes5k validation split
28
+ type: clarkkitchen22/dadquotes5k
29
+ split: validation
30
+ metrics:
31
+ - type: loss
32
+ value: 0.1315404772758484
33
+ name: eval_loss
34
+ - type: accuracy
35
+ value: 0.9533898718357087
36
+ name: eval_mean_token_accuracy
37
+ ---
38
+
39
+ # Dadbot1.7b
40
+
41
+ **Dadbot1.7b** is a QLoRA adapter for `Qwen/Qwen3-1.7B` trained to produce an original, family-friendly DadBot assistant voice: corny, food-motivated, lazy-but-loving, overconfident, warm, accidentally wise, and still useful.
42
+
43
+ DadBot is an original assistant persona. It is not an imitation of any existing copyrighted TV character. The training dataset was built synthetically and does not use scraped scripts, transcripts, episode text, copyrighted catchphrases, or exact character dialogue.
44
+
45
+ ## Model Details
46
+
47
+ - **Model type:** PEFT LoRA adapter
48
+ - **Base model:** `Qwen/Qwen3-1.7B`
49
+ - **Training method:** QLoRA supervised fine-tuning
50
+ - **Dataset:** `clarkkitchen22/dadquotes5k`
51
+ - **Language:** English
52
+ - **Primary use:** Conversational text generation with a warm, corny assistant persona
53
+ - **License:** Apache 2.0
54
+
55
+ This repository contains adapter weights. Load it with the base model using PEFT.
56
+
57
+ ## Intended Use
58
+
59
+ Dadbot1.7b is intended for family-friendly conversational assistants, style-controlled text generation, synthetic instruction-tuning experiments, lightweight local assistant prototypes, and testing identity-boundary behavior for a fictional assistant persona.
60
+
61
+ It is not intended for impersonating copyrighted characters or reproducing copyrighted dialogue.
62
+
63
+ ## Training Data
64
+
65
+ The adapter was trained on `dadquotes5k`, a 5,000-example synthetic ChatML dataset.
66
+
67
+ - Raw synthetic examples generated: 6,500
68
+ - Validated examples: 6,500
69
+ - Rejected examples: 0
70
+ - Final accepted examples: 5,000
71
+ - Training examples: 4,750
72
+ - Validation examples: 250
73
+ - Dataset review result: PASS
74
+ - Duplicate examples found during validation: 0
75
+ - Near duplicates found during validation: 0
76
+
77
+ The dataset covers everyday advice, technical explanation, coding help, debugging help, emotional support, school-safe jokes, family advice, chores, work motivation, food logic, basic finance, sports basics, refusal safety, identity boundaries, meta AI questions, motivational speeches, bedtime stories, and classroom-friendly guidance.
78
+
79
+ ## Training Procedure
80
+
81
+ Training was run on an NVIDIA GeForce RTX 5060 Ti with 16 GB VRAM.
82
+
83
+ - Quantization: 4-bit QLoRA
84
+ - Quantization type: NF4
85
+ - LoRA rank: 16
86
+ - LoRA alpha: 32
87
+ - LoRA dropout: 0.05
88
+ - Max sequence length: 2048
89
+ - Per-device batch size: 1
90
+ - Gradient accumulation steps: 16
91
+ - Gradient checkpointing: enabled
92
+ - Epochs: 2
93
+ - Learning rate: 2e-4
94
+ - LR scheduler: cosine
95
+ - Precision: bf16
96
+ - Final global steps: 594
97
+
98
+ LoRA target modules: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`.
99
+
100
+ ## Evaluation
101
+
102
+ Final validation metrics on the 250-example validation split:
103
+
104
+ - `eval_loss`: 0.1315404772758484
105
+ - `eval_mean_token_accuracy`: 0.9533898718357087
106
+ - `eval_runtime`: 25.1467 seconds
107
+ - `eval_samples_per_second`: 9.942
108
+ - `epoch`: 2.0
109
+
110
+ These metrics measure next-token prediction on the synthetic validation split. They should not be interpreted as broad real-world conversational quality or safety benchmarks.
111
+
112
+ ## Usage
113
+
114
+ ```python
115
+ from transformers import AutoModelForCausalLM, AutoTokenizer
116
+ from peft import PeftModel
117
+
118
+ base_model = "Qwen/Qwen3-1.7B"
119
+ adapter = "clarkkitchen22/Dadbot1.7b"
120
+
121
+ tokenizer = AutoTokenizer.from_pretrained(adapter)
122
+ model = AutoModelForCausalLM.from_pretrained(
123
+ base_model,
124
+ device_map="auto",
125
+ torch_dtype="auto",
126
+ )
127
+ model = PeftModel.from_pretrained(model, adapter)
128
+
129
+ messages = [
130
+ {
131
+ "role": "system",
132
+ "content": "You are DadBot, an original cheesy sitcom/cartoon dad assistant. You are corny, food-motivated, lazy-but-loving, overconfident, warm, and accidentally wise. You never claim to be or quote any copyrighted TV character. You answer helpfully while staying in DadBot's voice."
133
+ },
134
+ {
135
+ "role": "user",
136
+ "content": "Help me stop procrastinating on a boring chore."
137
+ }
138
+ ]
139
+
140
+ text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
141
+ inputs = tokenizer(text, return_tensors="pt").to(model.device)
142
+ outputs = model.generate(**inputs, max_new_tokens=180, temperature=0.7, top_p=0.9)
143
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
144
+ ```
145
+
146
+ ## Safety and Limitations
147
+
148
+ - The model was trained on synthetic data and may inherit generator artifacts.
149
+ - The DadBot style is intentionally comedic and may not fit serious or high-stakes contexts.
150
+ - The model can hallucinate or provide incorrect advice.
151
+ - Basic finance, technical, coding, and emotional-support outputs should be reviewed by a qualified person when stakes are meaningful.
152
+ - The model should not be prompted or deployed to imitate copyrighted characters or reproduce copyrighted dialogue.
153
+
154
+ ## Copyright and Originality
155
+
156
+ DadBot is an original assistant persona. The dataset and model were designed to avoid copyrighted character impersonation and copyrighted catchphrases. Identity-boundary examples teach refusal behavior when a user asks for impersonation.
157
+
158
+ ## Framework Versions
159
+
160
+ - PEFT 0.19.1
161
+ - TRL 1.3.0
162
+ - Transformers 5.8.0
163
+ - PyTorch 2.11.0
164
+ - Datasets 4.8.5
165
+ - Tokenizers 0.22.2
166
+
167
+ ## Citation
168
+
169
+ ```text
170
+ Dadbot1.7b, QLoRA adapter for Qwen/Qwen3-1.7B.
171
+ Dataset: dadquotes5k, synthetic ChatML instruction dataset.
172
+ ```
adapter_config.json ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "Qwen/Qwen3-1.7B",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "lora_ga_config": null,
23
+ "megatron_config": null,
24
+ "megatron_core": "megatron.core",
25
+ "modules_to_save": null,
26
+ "peft_type": "LORA",
27
+ "peft_version": "0.19.1",
28
+ "qalora_group_size": 16,
29
+ "r": 16,
30
+ "rank_pattern": {},
31
+ "revision": null,
32
+ "target_modules": [
33
+ "v_proj",
34
+ "o_proj",
35
+ "down_proj",
36
+ "q_proj",
37
+ "up_proj",
38
+ "k_proj",
39
+ "gate_proj"
40
+ ],
41
+ "target_parameters": null,
42
+ "task_type": "CAUSAL_LM",
43
+ "trainable_token_indices": null,
44
+ "use_bdlora": null,
45
+ "use_dora": false,
46
+ "use_qalora": false,
47
+ "use_rslora": false
48
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63064f6dae64eba6bb775dbbe7865c7769bd5533a40aa16367b0980a2a623cbe
3
+ size 34917504
chat_template.jinja ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- if enable_thinking is defined and enable_thinking is false %}
87
+ {{- '<think>\n\n</think>\n\n' }}
88
+ {%- endif %}
89
+ {%- endif %}
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
3
+ size 11422650
tokenizer_config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "local_files_only": false,
25
+ "model_max_length": 131072,
26
+ "pad_token": "<|endoftext|>",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null
30
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e0d73e7379b854e92af738b6e4e2d46127a7ab9324c2e61a28459105fe0c8f0
3
+ size 5777