Hugging Face's logo

ysn-rfd
/
Refact-1_6B-fim-GGUF

Text Generation
Transformers
GGUF
English
code
llama-cpp
matrixportal
Eval Results (legacy)
Model card Files
xet
 Community
ysn-rfd commited on
Commit
c104108
·
verified ·
1 Parent(s): f85612b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +347 -0
README.md ADDED
@@ -0,0 +1,347 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: smallcloudai/Refact-1_6B-fim
3
+ datasets:
4
+ - bigcode/the-stack-dedup
5
+ - rombodawg/2XUNCENSORED_MegaCodeTraining188k
6
+ - bigcode/commitpackft
7
+ language:
8
+ - en
9
+ library_name: transformers
10
+ license: bigscience-openrail-m
11
+ metrics:
12
+ - code_eval
13
+ pipeline_tag: text-generation
14
+ tags:
15
+ - code
16
+ - llama-cpp
17
+ - matrixportal
18
+ inference: true
19
+ widget:
20
+ - text: 'def print_hello_world():'
21
+ example_title: Hello world
22
+ group: Python
23
+ pretrain-datasets:
24
+ - books
25
+ - arxiv
26
+ - c4
27
+ - falcon-refinedweb
28
+ - wiki
29
+ - github-issues
30
+ - stack_markdown
31
+ - self-made dataset of permissive github code
32
+ model-index:
33
+ - name: Refact-1.6B
34
+ results:
35
+ - task:
36
+ type: text-generation
37
+ dataset:
38
+ name: HumanEval
39
+ type: openai_humaneval
40
+ metrics:
41
+ - type: pass@1
42
+ value: 32.0
43
+ name: pass@1 (T=0.01)
44
+ verified: false
45
+ - type: pass@1
46
+ value: 31.5
47
+ name: pass@1 (T=0.2)
48
+ verified: false
49
+ - type: pass@10
50
+ value: 53.0
51
+ name: pass@10 (T=0.8)
52
+ verified: false
53
+ - type: pass@100
54
+ value: 76.9
55
+ name: pass@100 (T=0.8)
56
+ verified: false
57
+ - task:
58
+ type: text-generation
59
+ dataset:
60
+ name: HumanEvalSynthesize Python
61
+ type: bigcode/humanevalpack
62
+ metrics:
63
+ - type: pass@1
64
+ value: 35.8
65
+ name: pass@1 (T=0.2)
66
+ verified: false
67
+ - type: pass@1
68
+ value: 31.6
69
+ name: pass@1 (T=0.2)
70
+ verified: false
71
+ - type: pass@1
72
+ value: 29.1
73
+ name: pass@1 (T=0.2)
74
+ verified: false
75
+ - type: pass@1
76
+ value: -1
77
+ name: pass@1 (T=0.2)
78
+ verified: false
79
+ - type: pass@1
80
+ value: 26.3
81
+ name: pass@1 (T=0.2)
82
+ verified: false
83
+ - type: pass@1
84
+ value: -1
85
+ name: pass@1 (T=0.2)
86
+ verified: false
87
+ - type: pass@1
88
+ value: -1
89
+ name: pass@1 (T=0.2)
90
+ verified: false
91
+ - type: pass@1
92
+ value: 18.38
93
+ name: pass@1 (T=0.2)
94
+ verified: false
95
+ - type: pass@1
96
+ value: 12.28
97
+ name: pass@1 (T=0.2)
98
+ verified: false
99
+ - type: pass@1
100
+ value: 15.12
101
+ name: pass@1 (T=0.2)
102
+ verified: false
103
+ - type: pass@1
104
+ value: -1
105
+ name: pass@1 (T=0.2)
106
+ verified: false
107
+ - type: pass@1
108
+ value: 13.17
109
+ name: pass@1 (T=0.2)
110
+ verified: false
111
+ - type: pass@1
112
+ value: 2.8
113
+ name: pass@1 (T=0.2)
114
+ verified: false
115
+ - type: pass@1
116
+ value: -1
117
+ name: pass@1 (T=0.2)
118
+ verified: false
119
+ - type: pass@1
120
+ value: 26.92
121
+ name: pass@1 (T=0.2)
122
+ verified: false
123
+ - type: pass@1
124
+ value: 26.85
125
+ name: pass@1 (T=0.2)
126
+ verified: false
127
+ - type: pass@1
128
+ value: 30.76
129
+ name: pass@1 (T=0.2)
130
+ verified: false
131
+ - type: pass@1
132
+ value: -1
133
+ name: pass@1 (T=0.2)
134
+ verified: false
135
+ - type: pass@1
136
+ value: 25.94
137
+ name: pass@1 (T=0.2)
138
+ verified: false
139
+ - type: pass@1
140
+ value: 8.44
141
+ name: pass@1 (T=0.2)
142
+ verified: false
143
+ - type: pass@1
144
+ value: -1
145
+ name: pass@1 (T=0.2)
146
+ verified: false
147
+ - type: pass@1
148
+ value: 26.46
149
+ name: pass@1 (T=0.2)
150
+ verified: false
151
+ - type: pass@1
152
+ value: 17.86
153
+ name: pass@1 (T=0.2)
154
+ verified: false
155
+ - type: pass@1
156
+ value: 20.94
157
+ name: pass@1 (T=0.2)
158
+ verified: false
159
+ - type: pass@1
160
+ value: -1
161
+ name: pass@1 (T=0.2)
162
+ verified: false
163
+ - type: pass@1
164
+ value: 18.78
165
+ name: pass@1 (T=0.2)
166
+ verified: false
167
+ - type: pass@1
168
+ value: -1
169
+ name: pass@1 (T=0.2)
170
+ verified: false
171
+ - type: pass@1
172
+ value: -1
173
+ name: pass@1 (T=0.2)
174
+ verified: false
175
+ - task:
176
+ type: text-generation
177
+ dataset:
178
+ name: MBPP
179
+ type: mbpp
180
+ metrics:
181
+ - type: pass@1
182
+ value: 31.15
183
+ name: pass@1 (T=0.01)
184
+ verified: false
185
+ - task:
186
+ type: text-generation
187
+ dataset:
188
+ name: DS-1000 (Overall Completion)
189
+ type: ds1000
190
+ metrics:
191
+ - type: pass@1
192
+ value: 10.1
193
+ name: pass@1 (T=0.2)
194
+ verified: false
195
+ - task:
196
+ type: text-generation
197
+ dataset:
198
+ name: MultiPL-HumanEval (C++)
199
+ type: nuprl/MultiPL-E
200
+ metrics:
201
+ - type: pass@1
202
+ value: 21.61
203
+ name: pass@1 (T=0.2)
204
+ verified: false
205
+ - type: pass@1
206
+ value: 13.91
207
+ name: pass@1 (T=0.2)
208
+ verified: false
209
+ - type: pass@1
210
+ value: 9.5
211
+ name: pass@1 (T=0.2)
212
+ verified: false
213
+ - type: pass@1
214
+ value: 53.57
215
+ name: pass@1 (T=0.2)
216
+ verified: false
217
+ - type: pass@1
218
+ value: 21.58
219
+ name: pass@1 (T=0.2)
220
+ verified: false
221
+ - type: pass@1
222
+ value: 13.75
223
+ name: pass@1 (T=0.2)
224
+ verified: false
225
+ - type: pass@1
226
+ value: 26.88
227
+ name: pass@1 (T=0.2)
228
+ verified: false
229
+ - type: pass@1
230
+ value: 15.26
231
+ name: pass@1 (T=0.2)
232
+ verified: false
233
+ - type: pass@1
234
+ value: 23.04
235
+ name: pass@1 (T=0.2)
236
+ verified: false
237
+ - type: pass@1
238
+ value: 12.1
239
+ name: pass@1 (T=0.2)
240
+ verified: false
241
+ - type: pass@1
242
+ value: 29.6
243
+ name: pass@1 (T=0.2)
244
+ verified: false
245
+ - type: pass@1
246
+ value: 13.77
247
+ name: pass@1 (T=0.2)
248
+ verified: false
249
+ - type: pass@1
250
+ value: 12.68
251
+ name: pass@1 (T=0.2)
252
+ verified: false
253
+ - type: pass@1
254
+ value: 4.29
255
+ name: pass@1 (T=0.2)
256
+ verified: false
257
+ - type: pass@1
258
+ value: 19.54
259
+ name: pass@1 (T=0.2)
260
+ verified: false
261
+ - type: pass@1
262
+ value: 18.33
263
+ name: pass@1 (T=0.2)
264
+ verified: false
265
+ - type: pass@1
266
+ value: 5.7
267
+ name: pass@1 (T=0.2)
268
+ verified: false
269
+ - type: pass@1
270
+ value: 17.68
271
+ name: pass@1 (T=0.2)
272
+ verified: false
273
+ - type: pass@1
274
+ value: 25
275
+ name: pass@1 (T=0.2)
276
+ verified: false
277
+ ---
278
+
279
+ # ysn-rfd/Refact-1_6B-fim-GGUF
280
+ This model was converted to GGUF format from [`smallcloudai/Refact-1_6B-fim`](https://huggingface.co/smallcloudai/Refact-1_6B-fim) using llama.cpp via the ggml.ai's [all-gguf-same-where](https://huggingface.co/spaces/matrixportal/all-gguf-same-where) space.
281
+ Refer to the [original model card](https://huggingface.co/smallcloudai/Refact-1_6B-fim) for more details on the model.
282
+
283
+ ## ✅ Quantized Models Download List
284
+
285
+ ### 🔍 Recommended Quantizations
286
+ - **✨ General CPU Use:** [`Q4_K_M`](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q4_k_m.gguf) (Best balance of speed/quality)
287
+ - **📱 ARM Devices:** [`Q4_0`](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q4_0.gguf) (Optimized for ARM CPUs)
288
+ - **🏆 Maximum Quality:** [`Q8_0`](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q8_0.gguf) (Near-original quality)
289
+
290
+ ### 📦 Full Quantization Options
291
+ | 🚀 Download | 🔢 Type | 📝 Notes |
292
+ |:---------|:-----|:------|
293
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q2_k.gguf) | ![Q2_K](https://img.shields.io/badge/Q2_K-1A73E8) | Basic quantization |
294
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q3_k_s.gguf) | ![Q3_K_S](https://img.shields.io/badge/Q3_K_S-34A853) | Small size |
295
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q3_k_m.gguf) | ![Q3_K_M](https://img.shields.io/badge/Q3_K_M-FBBC05) | Balanced quality |
296
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q3_k_l.gguf) | ![Q3_K_L](https://img.shields.io/badge/Q3_K_L-4285F4) | Better quality |
297
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q4_0.gguf) | ![Q4_0](https://img.shields.io/badge/Q4_0-EA4335) | Fast on ARM |
298
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q4_k_s.gguf) | ![Q4_K_S](https://img.shields.io/badge/Q4_K_S-673AB7) | Fast, recommended |
299
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q4_k_m.gguf) | ![Q4_K_M](https://img.shields.io/badge/Q4_K_M-673AB7) ⭐ | Best balance |
300
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q5_0.gguf) | ![Q5_0](https://img.shields.io/badge/Q5_0-FF6D01) | Good quality |
301
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q5_k_s.gguf) | ![Q5_K_S](https://img.shields.io/badge/Q5_K_S-0F9D58) | Balanced |
302
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q5_k_m.gguf) | ![Q5_K_M](https://img.shields.io/badge/Q5_K_M-0F9D58) | High quality |
303
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q6_k.gguf) | ![Q6_K](https://img.shields.io/badge/Q6_K-4285F4) 🏆 | Very good quality |
304
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-q8_0.gguf) | ![Q8_0](https://img.shields.io/badge/Q8_0-EA4335) ⚡ | Fast, best quality |
305
+ | [Download](https://huggingface.co/ysn-rfd/Refact-1_6B-fim-GGUF/resolve/main/refact-1_6b-fim-f16.gguf) | ![F16](https://img.shields.io/badge/F16-000000) | Maximum accuracy |
306
+
307
+ 💡 **Tip:** Use `F16` for maximum precision when quality is critical
308
+
309
+
310
+ ---
311
+ # 🚀 Applications and Tools for Locally Quantized LLMs
312
+ ## 🖥️ Desktop Applications
313
+
314
+ | Application | Description | Download Link |
315
+ |-----------------|----------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------|
316
+ | **Llama.cpp** | A fast and efficient inference engine for GGUF models. | [GitHub Repository](https://github.com/ggml-org/llama.cpp) |
317
+ | **Ollama** | A streamlined solution for running LLMs locally. | [Website](https://ollama.com/) |
318
+ | **AnythingLLM** | An AI-powered knowledge management tool. | [GitHub Repository](https://github.com/Mintplex-Labs/anything-llm) |
319
+ | **Open WebUI** | A user-friendly web interface for running local LLMs. | [GitHub Repository](https://github.com/open-webui/open-webui) |
320
+ | **GPT4All** | A user-friendly desktop application supporting various LLMs, compatible with GGUF models. | [GitHub Repository](https://github.com/nomic-ai/gpt4all) |
321
+ | **LM Studio** | A desktop application designed to run and manage local LLMs, supporting GGUF format. | [Website](https://lmstudio.ai/) |
322
+ | **GPT4All Chat**| A chat application compatible with GGUF models for local, offline interactions. | [GitHub Repository](https://github.com/nomic-ai/gpt4all) |
323
+
324
+ ---
325
+
326
+ ## 📱 Mobile Applications
327
+
328
+ | Application | Description | Download Link |
329
+ |-------------------|----------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------|
330
+ | **ChatterUI** | A simple and lightweight LLM app for mobile devices. | [GitHub Repository](https://github.com/Vali-98/ChatterUI) |
331
+ | **Maid** | Mobile Artificial Intelligence Distribution for running AI models on mobile devices. | [GitHub Repository](https://github.com/Mobile-Artificial-Intelligence/maid) |
332
+ | **PocketPal AI** | A mobile AI assistant powered by local models. | [GitHub Repository](https://github.com/a-ghorbani/pocketpal-ai) |
333
+ | **Layla** | A flexible platform for running various AI models on mobile devices. | [Website](https://www.layla-network.ai/) |
334
+
335
+ ---
336
+
337
+ ## 🎨 Image Generation Applications
338
+
339
+ | Application | Description | Download Link |
340
+ |-------------------------------------|----------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------|
341
+ | **Stable Diffusion** | An open-source AI model for generating images from text. | [GitHub Repository](https://github.com/CompVis/stable-diffusion) |
342
+ | **Stable Diffusion WebUI** | A web application providing access to Stable Diffusion models via a browser interface. | [GitHub Repository](https://github.com/AUTOMATIC1111/stable-diffusion-webui) |
343
+ | **Local Dream** | Android Stable Diffusion with Snapdragon NPU acceleration. Also supports CPU inference. | [GitHub Repository](https://github.com/xororz/local-dream) |
344
+ | **Stable-Diffusion-Android (SDAI)** | An open-source AI art application for Android devices, enabling digital art creation. | [GitHub Repository](https://github.com/ShiftHackZ/Stable-Diffusion-Android) |
345
+
346
+ ---
347
+