Instructions to use mofosyne/TinyLLama-v0-5M-F16-llamafile with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mofosyne/TinyLLama-v0-5M-F16-llamafile with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="mofosyne/TinyLLama-v0-5M-F16-llamafile",
	filename="TinyLLama-4.6M-v0.0-F16.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mofosyne/TinyLLama-v0-5M-F16-llamafile with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mofosyne/TinyLLama-v0-5M-F16-llamafile:F16
# Run inference directly in the terminal:
llama cli -hf mofosyne/TinyLLama-v0-5M-F16-llamafile:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mofosyne/TinyLLama-v0-5M-F16-llamafile:F16
# Run inference directly in the terminal:
llama cli -hf mofosyne/TinyLLama-v0-5M-F16-llamafile:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mofosyne/TinyLLama-v0-5M-F16-llamafile:F16
# Run inference directly in the terminal:
./llama-cli -hf mofosyne/TinyLLama-v0-5M-F16-llamafile:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mofosyne/TinyLLama-v0-5M-F16-llamafile:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mofosyne/TinyLLama-v0-5M-F16-llamafile:F16

Use Docker

docker model run hf.co/mofosyne/TinyLLama-v0-5M-F16-llamafile:F16

LM Studio
Jan

vLLM

How to use mofosyne/TinyLLama-v0-5M-F16-llamafile with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mofosyne/TinyLLama-v0-5M-F16-llamafile"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mofosyne/TinyLLama-v0-5M-F16-llamafile",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/mofosyne/TinyLLama-v0-5M-F16-llamafile:F16

Ollama
How to use mofosyne/TinyLLama-v0-5M-F16-llamafile with Ollama:
```
ollama run hf.co/mofosyne/TinyLLama-v0-5M-F16-llamafile:F16
```

Unsloth Studio

How to use mofosyne/TinyLLama-v0-5M-F16-llamafile with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mofosyne/TinyLLama-v0-5M-F16-llamafile to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mofosyne/TinyLLama-v0-5M-F16-llamafile to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for mofosyne/TinyLLama-v0-5M-F16-llamafile to start chatting

Atomic Chat new
Docker Model Runner
How to use mofosyne/TinyLLama-v0-5M-F16-llamafile with Docker Model Runner:
```
docker model run hf.co/mofosyne/TinyLLama-v0-5M-F16-llamafile:F16
```

Lemonade

How to use mofosyne/TinyLLama-v0-5M-F16-llamafile with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mofosyne/TinyLLama-v0-5M-F16-llamafile:F16

Run and chat with the model

lemonade run user.TinyLLama-v0-5M-F16-llamafile-F16

List all available models

lemonade list

mofosyne commited on Jul 20, 2024

Commit

8f0017d

1 Parent(s): 7fb699e

detailed licence kv metadata was missing

Browse files

Files changed (5) hide show

Tinyllama-4.6M-v0.0-F16.dump.md +104 -101
Tinyllama-4.6M-v0.0-F16.gguf +2 -2
Tinyllama-4.6M-v0.0-F16.llamafile +1 -1
llamafile-creation.sh +1 -1
maykeye_tinyllama-metadata.json +4 -3

Tinyllama-4.6M-v0.0-F16.dump.md CHANGED Viewed

@@ -4,13 +4,13 @@
 ## Key Value Metadata Store
-There are 36 key-value pairs in this file
 | POS | TYPE      | Count | Key                                    | Value                                                                            |
 |----:|:----------|------:|:---------------------------------------|:---------------------------------------------------------------------------------|
 |   1 | UINT32    |     1 | GGUF.version                           | 3                                                                                |
 |   2 | UINT64    |     1 | GGUF.tensor_count                      | 75                                                                               |
-|   3 | UINT64    |     1 | GGUF.kv_count                          | 33                                                                               |
 |   4 | STRING    |     1 | general.architecture                   | `llama`                                                                          |
 |   5 | STRING    |     1 | general.type                           | `model`                                                                          |
 |   6 | STRING    |     1 | general.name                           | `TinyLLama`                                                                      |
@@ -20,30 +20,33 @@ There are 36 key-value pairs in this file
 |  10 | STRING    |     1 | general.quantized_by                   | `Mofosyne`                                                                       |
 |  11 | STRING    |     1 | general.size_label                     | `4.6M`                                                                           |
 |  12 | STRING    |     1 | general.license                        | `apache-2.0`                                                                     |
-|  13 | STRING    |     1 | general.url                            | `https://huggingface.co/mofosyne/TinyLLama-v0-llamafile`                         |
-|  14 | STRING    |     1 | general.source.url                     | `https://huggingface.co/Maykeye/TinyLLama-v0`                                    |
-|  15 | [STRING]  |     5 | general.tags                           | [ `text generation`, `transformer`, `llama`, `tiny`, `tiny model` ]              |
-|  16 | [STRING]  |     1 | general.languages                      | [ `en` ]                                                                         |
-|  17 | [STRING]  |     2 | general.datasets                       | [ `https://hugging`...`-GPT4-train.txt`, `https://hugging`...`-GPT4-valid.txt` ] |
-|  18 | UINT32    |     1 | llama.block_count                      | 8                                                                                |
-|  19 | UINT32    |     1 | llama.context_length                   | 2048                                                                             |
-|  20 | UINT32    |     1 | llama.embedding_length                 | 64                                                                               |
-|  21 | UINT32    |     1 | llama.feed_forward_length              | 256                                                                              |
-|  22 | UINT32    |     1 | llama.attention.head_count             | 16                                                                               |
-|  23 | FLOAT32   |     1 | llama.attention.layer_norm_rms_epsilon | 1e-06                                                                            |
-|  24 | UINT32    |     1 | general.file_type                      | 1                                                                                |
-|  25 | UINT32    |     1 | llama.vocab_size                       | 32000                                                                            |
-|  26 | UINT32    |     1 | llama.rope.dimension_count             | 4                                                                                |
-|  27 | STRING    |     1 | tokenizer.ggml.model                   | `llama`                                                                          |
-|  28 | STRING    |     1 | tokenizer.ggml.pre                     | `default`                                                                        |
-|  29 | [STRING]  | 32000 | tokenizer.ggml.tokens                  | [ `<unk>`, `<s>`, `</s>`, `<0x00>`, `<0x01>`, ... ]                              |
-|  30 | [FLOAT32] | 32000 | tokenizer.ggml.scores                  | [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ]                                       |
-|  31 | [INT32]   | 32000 | tokenizer.ggml.token_type              | [ 2, 3, 3, 6, 6, 6, 6, ... ]                                                     |
-|  32 | UINT32    |     1 | tokenizer.ggml.bos_token_id            | 1                                                                                |
-|  33 | UINT32    |     1 | tokenizer.ggml.eos_token_id            | 2                                                                                |
-|  34 | UINT32    |     1 | tokenizer.ggml.unknown_token_id        | 0                                                                                |
-|  35 | UINT32    |     1 | tokenizer.ggml.padding_token_id        | 0                                                                                |
-|  36 | UINT32    |     1 | general.quantization_version           | 2                                                                                |
 ## Tensors Overview ~5M Elements
@@ -65,81 +68,81 @@ This table contains the offset and data segment relative to start of file
 | T_ID | Tensor Layer Name        |  Data Offset (B) |    Data Size (B) |
 |-----:|:-------------------------|-----------------:|-----------------:|
-|    0 | output.weight            |          0xba760 |         0x3e8000 |
-|    1 | token_embd.weight        |         0x4a2760 |         0x3e8000 |
-|    2 | blk.0.attn_norm.weight   |         0x88a760 |            0x100 |
-|    3 | blk.0.ffn_down.weight    |         0x88a860 |           0x8000 |
-|    4 | blk.0.ffn_gate.weight    |         0x892860 |           0x8000 |
-|    5 | blk.0.ffn_up.weight      |         0x89a860 |           0x8000 |
-|    6 | blk.0.ffn_norm.weight    |         0x8a2860 |            0x100 |
-|    7 | blk.0.attn_k.weight      |         0x8a2960 |           0x2000 |
-|    8 | blk.0.attn_output.weight |         0x8a4960 |           0x2000 |
-|    9 | blk.0.attn_q.weight      |         0x8a6960 |           0x2000 |
-|   10 | blk.0.attn_v.weight      |         0x8a8960 |           0x2000 |
-|   11 | blk.1.attn_norm.weight   |         0x8aa960 |            0x100 |
-|   12 | blk.1.ffn_down.weight    |         0x8aaa60 |           0x8000 |
-|   13 | blk.1.ffn_gate.weight    |         0x8b2a60 |           0x8000 |
-|   14 | blk.1.ffn_up.weight      |         0x8baa60 |           0x8000 |
-|   15 | blk.1.ffn_norm.weight    |         0x8c2a60 |            0x100 |
-|   16 | blk.1.attn_k.weight      |         0x8c2b60 |           0x2000 |
-|   17 | blk.1.attn_output.weight |         0x8c4b60 |           0x2000 |
-|   18 | blk.1.attn_q.weight      |         0x8c6b60 |           0x2000 |
-|   19 | blk.1.attn_v.weight      |         0x8c8b60 |           0x2000 |
-|   20 | blk.2.attn_norm.weight   |         0x8cab60 |            0x100 |
-|   21 | blk.2.ffn_down.weight    |         0x8cac60 |           0x8000 |
-|   22 | blk.2.ffn_gate.weight    |         0x8d2c60 |           0x8000 |
-|   23 | blk.2.ffn_up.weight      |         0x8dac60 |           0x8000 |
-|   24 | blk.2.ffn_norm.weight    |         0x8e2c60 |            0x100 |
-|   25 | blk.2.attn_k.weight      |         0x8e2d60 |           0x2000 |
-|   26 | blk.2.attn_output.weight |         0x8e4d60 |           0x2000 |
-|   27 | blk.2.attn_q.weight      |         0x8e6d60 |           0x2000 |
-|   28 | blk.2.attn_v.weight      |         0x8e8d60 |           0x2000 |
-|   29 | blk.3.attn_norm.weight   |         0x8ead60 |            0x100 |
-|   30 | blk.3.ffn_down.weight    |         0x8eae60 |           0x8000 |
-|   31 | blk.3.ffn_gate.weight    |         0x8f2e60 |           0x8000 |
-|   32 | blk.3.ffn_up.weight      |         0x8fae60 |           0x8000 |
-|   33 | blk.3.ffn_norm.weight    |         0x902e60 |            0x100 |
-|   34 | blk.3.attn_k.weight      |         0x902f60 |           0x2000 |
-|   35 | blk.3.attn_output.weight |         0x904f60 |           0x2000 |
-|   36 | blk.3.attn_q.weight      |         0x906f60 |           0x2000 |
-|   37 | blk.3.attn_v.weight      |         0x908f60 |           0x2000 |
-|   38 | blk.4.attn_norm.weight   |         0x90af60 |            0x100 |
-|   39 | blk.4.ffn_down.weight    |         0x90b060 |           0x8000 |
-|   40 | blk.4.ffn_gate.weight    |         0x913060 |           0x8000 |
-|   41 | blk.4.ffn_up.weight      |         0x91b060 |           0x8000 |
-|   42 | blk.4.ffn_norm.weight    |         0x923060 |            0x100 |
-|   43 | blk.4.attn_k.weight      |         0x923160 |           0x2000 |
-|   44 | blk.4.attn_output.weight |         0x925160 |           0x2000 |
-|   45 | blk.4.attn_q.weight      |         0x927160 |           0x2000 |
-|   46 | blk.4.attn_v.weight      |         0x929160 |           0x2000 |
-|   47 | blk.5.attn_norm.weight   |         0x92b160 |            0x100 |
-|   48 | blk.5.ffn_down.weight    |         0x92b260 |           0x8000 |
-|   49 | blk.5.ffn_gate.weight    |         0x933260 |           0x8000 |
-|   50 | blk.5.ffn_up.weight      |         0x93b260 |           0x8000 |
-|   51 | blk.5.ffn_norm.weight    |         0x943260 |            0x100 |
-|   52 | blk.5.attn_k.weight      |         0x943360 |           0x2000 |
-|   53 | blk.5.attn_output.weight |         0x945360 |           0x2000 |
-|   54 | blk.5.attn_q.weight      |         0x947360 |           0x2000 |
-|   55 | blk.5.attn_v.weight      |         0x949360 |           0x2000 |
-|   56 | blk.6.attn_norm.weight   |         0x94b360 |            0x100 |
-|   57 | blk.6.ffn_down.weight    |         0x94b460 |           0x8000 |
-|   58 | blk.6.ffn_gate.weight    |         0x953460 |           0x8000 |
-|   59 | blk.6.ffn_up.weight      |         0x95b460 |           0x8000 |
-|   60 | blk.6.ffn_norm.weight    |         0x963460 |            0x100 |
-|   61 | blk.6.attn_k.weight      |         0x963560 |           0x2000 |
-|   62 | blk.6.attn_output.weight |         0x965560 |           0x2000 |
-|   63 | blk.6.attn_q.weight      |         0x967560 |           0x2000 |
-|   64 | blk.6.attn_v.weight      |         0x969560 |           0x2000 |
-|   65 | blk.7.attn_norm.weight   |         0x96b560 |            0x100 |
-|   66 | blk.7.ffn_down.weight    |         0x96b660 |           0x8000 |
-|   67 | blk.7.ffn_gate.weight    |         0x973660 |           0x8000 |
-|   68 | blk.7.ffn_up.weight      |         0x97b660 |           0x8000 |
-|   69 | blk.7.ffn_norm.weight    |         0x983660 |            0x100 |
-|   70 | blk.7.attn_k.weight      |         0x983760 |           0x2000 |
-|   71 | blk.7.attn_output.weight |         0x985760 |           0x2000 |
-|   72 | blk.7.attn_q.weight      |         0x987760 |           0x2000 |
-|   73 | blk.7.attn_v.weight      |         0x989760 |           0x2000 |
-|   74 | output_norm.weight       |         0x98b760 |            0x100 |
 ### <a name="base">Base Tensor Group : ~4M Elements</a>

 ## Key Value Metadata Store
+There are 39 key-value pairs in this file
 | POS | TYPE      | Count | Key                                    | Value                                                                            |
 |----:|:----------|------:|:---------------------------------------|:---------------------------------------------------------------------------------|
 |   1 | UINT32    |     1 | GGUF.version                           | 3                                                                                |
 |   2 | UINT64    |     1 | GGUF.tensor_count                      | 75                                                                               |
+|   3 | UINT64    |     1 | GGUF.kv_count                          | 36                                                                               |
 |   4 | STRING    |     1 | general.architecture                   | `llama`                                                                          |
 |   5 | STRING    |     1 | general.type                           | `model`                                                                          |
 |   6 | STRING    |     1 | general.name                           | `TinyLLama`                                                                      |
 |  10 | STRING    |     1 | general.quantized_by                   | `Mofosyne`                                                                       |
 |  11 | STRING    |     1 | general.size_label                     | `4.6M`                                                                           |
 |  12 | STRING    |     1 | general.license                        | `apache-2.0`                                                                     |
+|  13 | STRING    |     1 | general.license.name                   | `Apache License Version 2.0, January 2004`                                       |
+|  14 | STRING    |     1 | general.license.link                   | `https://huggingface.co/dataset`...`ob/main/markdown/apache-2.0.md`              |
+|  15 | STRING    |     1 | general.url                            | `https://huggingface.co/mofosyne/TinyLLama-v0-llamafile`                         |
+|  16 | STRING    |     1 | general.source.url                     | `https://huggingface.co/Maykeye/TinyLLama-v0`                                    |
+|  17 | STRING    |     1 | general.source.repo_url                | `https://huggingface.co/Maykeye/TinyLLama-v0`                                    |
+|  18 | [STRING]  |     5 | general.tags                           | [ `text generation`, `transformer`, `llama`, `tiny`, `tiny model` ]              |
+|  19 | [STRING]  |     1 | general.languages                      | [ `en` ]                                                                         |
+|  20 | [STRING]  |     2 | general.datasets                       | [ `https://hugging`...`-GPT4-train.txt`, `https://hugging`...`-GPT4-valid.txt` ] |
+|  21 | UINT32    |     1 | llama.block_count                      | 8                                                                                |
+|  22 | UINT32    |     1 | llama.context_length                   | 2048                                                                             |
+|  23 | UINT32    |     1 | llama.embedding_length                 | 64                                                                               |
+|  24 | UINT32    |     1 | llama.feed_forward_length              | 256                                                                              |
+|  25 | UINT32    |     1 | llama.attention.head_count             | 16                                                                               |
+|  26 | FLOAT32   |     1 | llama.attention.layer_norm_rms_epsilon | 1e-06                                                                            |
+|  27 | UINT32    |     1 | general.file_type                      | 1                                                                                |
+|  28 | UINT32    |     1 | llama.vocab_size                       | 32000                                                                            |
+|  29 | UINT32    |     1 | llama.rope.dimension_count             | 4                                                                                |
+|  30 | STRING    |     1 | tokenizer.ggml.model                   | `llama`                                                                          |
+|  31 | STRING    |     1 | tokenizer.ggml.pre                     | `default`                                                                        |
+|  32 | [STRING]  | 32000 | tokenizer.ggml.tokens                  | [ `<unk>`, `<s>`, `</s>`, `<0x00>`, `<0x01>`, ... ]                              |
+|  33 | [FLOAT32] | 32000 | tokenizer.ggml.scores                  | [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ]                                       |
+|  34 | [INT32]   | 32000 | tokenizer.ggml.token_type              | [ 2, 3, 3, 6, 6, 6, 6, ... ]                                                     |
+|  35 | UINT32    |     1 | tokenizer.ggml.bos_token_id            | 1                                                                                |
+|  36 | UINT32    |     1 | tokenizer.ggml.eos_token_id            | 2                                                                                |
+|  37 | UINT32    |     1 | tokenizer.ggml.unknown_token_id        | 0                                                                                |
+|  38 | UINT32    |     1 | tokenizer.ggml.padding_token_id        | 0                                                                                |
+|  39 | UINT32    |     1 | general.quantization_version           | 2                                                                                |
 ## Tensors Overview ~5M Elements
 | T_ID | Tensor Layer Name        |  Data Offset (B) |    Data Size (B) |
 |-----:|:-------------------------|-----------------:|-----------------:|
+|    0 | output.weight            |          0xba880 |         0x3e8000 |
+|    1 | token_embd.weight        |         0x4a2880 |         0x3e8000 |
+|    2 | blk.0.attn_norm.weight   |         0x88a880 |            0x100 |
+|    3 | blk.0.ffn_down.weight    |         0x88a980 |           0x8000 |
+|    4 | blk.0.ffn_gate.weight    |         0x892980 |           0x8000 |
+|    5 | blk.0.ffn_up.weight      |         0x89a980 |           0x8000 |
+|    6 | blk.0.ffn_norm.weight    |         0x8a2980 |            0x100 |
+|    7 | blk.0.attn_k.weight      |         0x8a2a80 |           0x2000 |
+|    8 | blk.0.attn_output.weight |         0x8a4a80 |           0x2000 |
+|    9 | blk.0.attn_q.weight      |         0x8a6a80 |           0x2000 |
+|   10 | blk.0.attn_v.weight      |         0x8a8a80 |           0x2000 |
+|   11 | blk.1.attn_norm.weight   |         0x8aaa80 |            0x100 |
+|   12 | blk.1.ffn_down.weight    |         0x8aab80 |           0x8000 |
+|   13 | blk.1.ffn_gate.weight    |         0x8b2b80 |           0x8000 |
+|   14 | blk.1.ffn_up.weight      |         0x8bab80 |           0x8000 |
+|   15 | blk.1.ffn_norm.weight    |         0x8c2b80 |            0x100 |
+|   16 | blk.1.attn_k.weight      |         0x8c2c80 |           0x2000 |
+|   17 | blk.1.attn_output.weight |         0x8c4c80 |           0x2000 |
+|   18 | blk.1.attn_q.weight      |         0x8c6c80 |           0x2000 |
+|   19 | blk.1.attn_v.weight      |         0x8c8c80 |           0x2000 |
+|   20 | blk.2.attn_norm.weight   |         0x8cac80 |            0x100 |
+|   21 | blk.2.ffn_down.weight    |         0x8cad80 |           0x8000 |
+|   22 | blk.2.ffn_gate.weight    |         0x8d2d80 |           0x8000 |
+|   23 | blk.2.ffn_up.weight      |         0x8dad80 |           0x8000 |
+|   24 | blk.2.ffn_norm.weight    |         0x8e2d80 |            0x100 |
+|   25 | blk.2.attn_k.weight      |         0x8e2e80 |           0x2000 |
+|   26 | blk.2.attn_output.weight |         0x8e4e80 |           0x2000 |
+|   27 | blk.2.attn_q.weight      |         0x8e6e80 |           0x2000 |
+|   28 | blk.2.attn_v.weight      |         0x8e8e80 |           0x2000 |
+|   29 | blk.3.attn_norm.weight   |         0x8eae80 |            0x100 |
+|   30 | blk.3.ffn_down.weight    |         0x8eaf80 |           0x8000 |
+|   31 | blk.3.ffn_gate.weight    |         0x8f2f80 |           0x8000 |
+|   32 | blk.3.ffn_up.weight      |         0x8faf80 |           0x8000 |
+|   33 | blk.3.ffn_norm.weight    |         0x902f80 |            0x100 |
+|   34 | blk.3.attn_k.weight      |         0x903080 |           0x2000 |
+|   35 | blk.3.attn_output.weight |         0x905080 |           0x2000 |
+|   36 | blk.3.attn_q.weight      |         0x907080 |           0x2000 |
+|   37 | blk.3.attn_v.weight      |         0x909080 |           0x2000 |
+|   38 | blk.4.attn_norm.weight   |         0x90b080 |            0x100 |
+|   39 | blk.4.ffn_down.weight    |         0x90b180 |           0x8000 |
+|   40 | blk.4.ffn_gate.weight    |         0x913180 |           0x8000 |
+|   41 | blk.4.ffn_up.weight      |         0x91b180 |           0x8000 |
+|   42 | blk.4.ffn_norm.weight    |         0x923180 |            0x100 |
+|   43 | blk.4.attn_k.weight      |         0x923280 |           0x2000 |
+|   44 | blk.4.attn_output.weight |         0x925280 |           0x2000 |
+|   45 | blk.4.attn_q.weight      |         0x927280 |           0x2000 |
+|   46 | blk.4.attn_v.weight      |         0x929280 |           0x2000 |
+|   47 | blk.5.attn_norm.weight   |         0x92b280 |            0x100 |
+|   48 | blk.5.ffn_down.weight    |         0x92b380 |           0x8000 |
+|   49 | blk.5.ffn_gate.weight    |         0x933380 |           0x8000 |
+|   50 | blk.5.ffn_up.weight      |         0x93b380 |           0x8000 |
+|   51 | blk.5.ffn_norm.weight    |         0x943380 |            0x100 |
+|   52 | blk.5.attn_k.weight      |         0x943480 |           0x2000 |
+|   53 | blk.5.attn_output.weight |         0x945480 |           0x2000 |
+|   54 | blk.5.attn_q.weight      |         0x947480 |           0x2000 |
+|   55 | blk.5.attn_v.weight      |         0x949480 |           0x2000 |
+|   56 | blk.6.attn_norm.weight   |         0x94b480 |            0x100 |
+|   57 | blk.6.ffn_down.weight    |         0x94b580 |           0x8000 |
+|   58 | blk.6.ffn_gate.weight    |         0x953580 |           0x8000 |
+|   59 | blk.6.ffn_up.weight      |         0x95b580 |           0x8000 |
+|   60 | blk.6.ffn_norm.weight    |         0x963580 |            0x100 |
+|   61 | blk.6.attn_k.weight      |         0x963680 |           0x2000 |
+|   62 | blk.6.attn_output.weight |         0x965680 |           0x2000 |
+|   63 | blk.6.attn_q.weight      |         0x967680 |           0x2000 |
+|   64 | blk.6.attn_v.weight      |         0x969680 |           0x2000 |
+|   65 | blk.7.attn_norm.weight   |         0x96b680 |            0x100 |
+|   66 | blk.7.ffn_down.weight    |         0x96b780 |           0x8000 |
+|   67 | blk.7.ffn_gate.weight    |         0x973780 |           0x8000 |
+|   68 | blk.7.ffn_up.weight      |         0x97b780 |           0x8000 |
+|   69 | blk.7.ffn_norm.weight    |         0x983780 |            0x100 |
+|   70 | blk.7.attn_k.weight      |         0x983880 |           0x2000 |
+|   71 | blk.7.attn_output.weight |         0x985880 |           0x2000 |
+|   72 | blk.7.attn_q.weight      |         0x987880 |           0x2000 |
+|   73 | blk.7.attn_v.weight      |         0x989880 |           0x2000 |
+|   74 | output_norm.weight       |         0x98b880 |            0x100 |
 ### <a name="base">Base Tensor Group : ~4M Elements</a>

Tinyllama-4.6M-v0.0-F16.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dc7881ea9ea6c8022ea0d09092a6bc5c7d88d013ea96d81baec8884c04ed7759
-size 10008672

 version https://git-lfs.github.com/spec/v1
+oid sha256:4b8df674f3486214ade8412f6f0bc42168ad37ee086ebe3b1888832a8d8bca7c
+size 10008960

Tinyllama-4.6M-v0.0-F16.llamafile CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:55c2298a0a38f076b60b0a56de8df869aafbf73903545a045c26054457a3d119
 size 21509955

 version https://git-lfs.github.com/spec/v1
+oid sha256:d583b00bc88fc261078db9c5979ea65d2a4a5be4d7bf5c1662c5678ea9e72c28
 size 21509955

llamafile-creation.sh CHANGED Viewed

@@ -6,7 +6,7 @@ METADATA_FILE="maykeye_tinyllama-metadata.json"
 ###############################################################################
 # Pull both model folder, llamafile (for the engine) and llama.cpp (for the conversion script)
 echo == Prep Enviroment ==
-#git submodule update --init
 ###############################################################################
 echo == Build and prep the llamafile engine execuable ==

 ###############################################################################
 # Pull both model folder, llamafile (for the engine) and llama.cpp (for the conversion script)
 echo == Prep Enviroment ==
+git submodule update --init
 ###############################################################################
 echo == Build and prep the llamafile engine execuable ==

maykeye_tinyllama-metadata.json CHANGED Viewed

@@ -4,12 +4,13 @@
     "general.author": "Maykeye",
     "general.quantized_by": "Mofosyne",
     "general.url": "https://huggingface.co/mofosyne/TinyLLama-v0-llamafile",
     "general.description": "This gguf is ported from a first version of Maykeye attempt at recreating roneneldan/TinyStories-1M but using Llama architecture",
     "general.license": "apache-2.0",
-    "general.license_name": "Apache License Version 2.0, January 2004",
-    "general.license_link": "https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md",
     "general.source.url": "https://huggingface.co/Maykeye/TinyLLama-v0",
-    "general.source.source_repo_url": "https://huggingface.co/Maykeye/TinyLLama-v0",
     "general.tags": ["text generation", "transformer", "llama", "tiny", "tiny model"],
     "general.languages": ["en"],
     "general.datasets": ["https://huggingface.co/datasets/roneneldan/TinyStories/blob/main/TinyStoriesV2-GPT4-train.txt", "https://huggingface.co/datasets/roneneldan/TinyStories/blob/main/TinyStoriesV2-GPT4-valid.txt"]

     "general.author": "Maykeye",
     "general.quantized_by": "Mofosyne",
     "general.url": "https://huggingface.co/mofosyne/TinyLLama-v0-llamafile",
+    "general.repo_url": "https://huggingface.co/mofosyne/TinyLLama-v0-llamafile",
     "general.description": "This gguf is ported from a first version of Maykeye attempt at recreating roneneldan/TinyStories-1M but using Llama architecture",
     "general.license": "apache-2.0",
+    "general.license.name": "Apache License Version 2.0, January 2004",
+    "general.license.link": "https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md",
     "general.source.url": "https://huggingface.co/Maykeye/TinyLLama-v0",
+    "general.source.repo_url": "https://huggingface.co/Maykeye/TinyLLama-v0",
     "general.tags": ["text generation", "transformer", "llama", "tiny", "tiny model"],
     "general.languages": ["en"],
     "general.datasets": ["https://huggingface.co/datasets/roneneldan/TinyStories/blob/main/TinyStoriesV2-GPT4-train.txt", "https://huggingface.co/datasets/roneneldan/TinyStories/blob/main/TinyStoriesV2-GPT4-valid.txt"]