Instructions to use AliceThirty/Step-3.5-Flash-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AliceThirty/Step-3.5-Flash-gguf with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="AliceThirty/Step-3.5-Flash-gguf",
	filename="Q2_K/Step-3.5-Flash-Q2_K-00001-of-00002.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use AliceThirty/Step-3.5-Flash-gguf with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M

Use Docker

docker model run hf.co/AliceThirty/Step-3.5-Flash-gguf:Q4_K_M

LM Studio
Jan
Ollama
How to use AliceThirty/Step-3.5-Flash-gguf with Ollama:
```
ollama run hf.co/AliceThirty/Step-3.5-Flash-gguf:Q4_K_M
```

Unsloth Studio

How to use AliceThirty/Step-3.5-Flash-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AliceThirty/Step-3.5-Flash-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AliceThirty/Step-3.5-Flash-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AliceThirty/Step-3.5-Flash-gguf to start chatting

How to use AliceThirty/Step-3.5-Flash-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AliceThirty/Step-3.5-Flash-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AliceThirty/Step-3.5-Flash-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf AliceThirty/Step-3.5-Flash-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AliceThirty/Step-3.5-Flash-gguf:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use AliceThirty/Step-3.5-Flash-gguf with Docker Model Runner:
```
docker model run hf.co/AliceThirty/Step-3.5-Flash-gguf:Q4_K_M
```

Lemonade

How to use AliceThirty/Step-3.5-Flash-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AliceThirty/Step-3.5-Flash-gguf:Q4_K_M

Run and chat with the model

lemonade run user.Step-3.5-Flash-gguf-Q4_K_M

List all available models

lemonade list

AliceThirty commited on Feb 4

Commit

1a870e7

verified ·

1 Parent(s): 4b78952

Upload folder using huggingface_hub

Browse files

Files changed (21) hide show

.gitattributes +20 -0
Q2_K/Step-3.5-Flash-Q2_K-00001-of-00002.gguf +3 -0
Q2_K/Step-3.5-Flash-Q2_K-00002-of-00002.gguf +3 -0
Q3_K/Step-3.5-Flash-Q3_K-00001-of-00002.gguf +3 -0
Q3_K/Step-3.5-Flash-Q3_K-00002-of-00002.gguf +3 -0
Q3_K_L/Step-3.5-Flash-Q3_K_L-00001-of-00003.gguf +3 -0
Q3_K_L/Step-3.5-Flash-Q3_K_L-00002-of-00003.gguf +3 -0
Q3_K_L/Step-3.5-Flash-Q3_K_L-00003-of-00003.gguf +3 -0
Q3_K_M/Step-3.5-Flash-Q3_K_M-00001-of-00002.gguf +3 -0
Q3_K_M/Step-3.5-Flash-Q3_K_M-00002-of-00002.gguf +3 -0
Q3_K_S/Step-3.5-Flash-Q3_K_S-00001-of-00002.gguf +3 -0
Q3_K_S/Step-3.5-Flash-Q3_K_S-00002-of-00002.gguf +3 -0
Q4_K/Step-3.5-Flash-Q4_K-00001-of-00003.gguf +3 -0
Q4_K/Step-3.5-Flash-Q4_K-00002-of-00003.gguf +3 -0
Q4_K/Step-3.5-Flash-Q4_K-00003-of-00003.gguf +3 -0
Q4_K_M/Step-3.5-Flash-Q4_K_M-00001-of-00003.gguf +3 -0
Q4_K_M/Step-3.5-Flash-Q4_K_M-00002-of-00003.gguf +3 -0
Q4_K_M/Step-3.5-Flash-Q4_K_M-00003-of-00003.gguf +3 -0
Q4_K_S/Step-3.5-Flash-Q4_K_S-00001-of-00003.gguf +3 -0
Q4_K_S/Step-3.5-Flash-Q4_K_S-00002-of-00003.gguf +3 -0
Q4_K_S/Step-3.5-Flash-Q4_K_S-00003-of-00003.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,23 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Q2_K/Step-3.5-Flash-Q2_K-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Q2_K/Step-3.5-Flash-Q2_K-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K/Step-3.5-Flash-Q3_K-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K/Step-3.5-Flash-Q3_K-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_L/Step-3.5-Flash-Q3_K_L-00001-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_L/Step-3.5-Flash-Q3_K_L-00002-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_L/Step-3.5-Flash-Q3_K_L-00003-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Step-3.5-Flash-Q3_K_M-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Step-3.5-Flash-Q3_K_M-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_S/Step-3.5-Flash-Q3_K_S-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_S/Step-3.5-Flash-Q3_K_S-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Q4_K/Step-3.5-Flash-Q4_K-00001-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q4_K/Step-3.5-Flash-Q4_K-00002-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q4_K/Step-3.5-Flash-Q4_K-00003-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q4_K_M/Step-3.5-Flash-Q4_K_M-00001-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q4_K_M/Step-3.5-Flash-Q4_K_M-00002-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q4_K_M/Step-3.5-Flash-Q4_K_M-00003-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q4_K_S/Step-3.5-Flash-Q4_K_S-00001-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q4_K_S/Step-3.5-Flash-Q4_K_S-00002-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Q4_K_S/Step-3.5-Flash-Q4_K_S-00003-of-00003.gguf filter=lfs diff=lfs merge=lfs -text

Q2_K/Step-3.5-Flash-Q2_K-00001-of-00002.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1647795d9aebdbe73acc1addb507bc1d228fb96158bf3b085356cc28f8baff84
+size 49888011264

Q2_K/Step-3.5-Flash-Q2_K-00002-of-00002.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fbba1b5c0ebaf6ea151141610399368351222e268edb95ff6cf88fb8350a49f3
+size 21917852448

Q3_K/Step-3.5-Flash-Q3_K-00001-of-00002.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:af823fca80d82aaa126cae7af53b2cbf3356e4b2d922bdcf3472b6b959d0536d
+size 49670491904

Q3_K/Step-3.5-Flash-Q3_K-00002-of-00002.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8890f8af845a91836b078320cfa8aaa0b577157af484b4722261ddbd8e942386
+size 44130952192

Q3_K_L/Step-3.5-Flash-Q3_K_L-00001-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c5d9c79fb0d269de3337a6f4967319f27585dd80e03f84952ac33ae032f2883c
+size 49204013472

Q3_K_L/Step-3.5-Flash-Q3_K_L-00002-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d7ee2c65a65e1bee721a4694263c38763039ee441322ebf64f8328315f36fc8c
+size 49796539456

Q3_K_L/Step-3.5-Flash-Q3_K_L-00003-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a73ed65a7b61eeec044a5639db9f0e95b6f71d3c1a441d43800a85822c3b979b
+size 3041912256

Q3_K_M/Step-3.5-Flash-Q3_K_M-00001-of-00002.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:88b38a6b7723210f2ebb1f899e313e597c7c02f79bc4b660708eaf583da482ac
+size 49670491904

Q3_K_M/Step-3.5-Flash-Q3_K_M-00002-of-00002.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:02f4004c1dd5faaff3658549ce07eb457d5dc05071fc66f239b0c64589def128
+size 44130952192

Q3_K_S/Step-3.5-Flash-Q3_K_S-00001-of-00002.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c3048cf97993f3e35f38bc3a09552293dcf63aed05fce854cd06592730ce7d69
+size 49700614400

Q3_K_S/Step-3.5-Flash-Q3_K_S-00002-of-00002.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ee038250a8e40465c2b72169ef58826ab9a96854b4201c2d5a70a57acb9abaf4
+size 35318792736

Q4_K/Step-3.5-Flash-Q4_K-00001-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7b261cce520631091ae2f33f21e81745382603e08933039868635b14fee0f40b
+size 49329290240

Q4_K/Step-3.5-Flash-Q4_K-00002-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:662901e41b07e0545eb1e593cfe6b797531a3bdeb990986137ae175a4a4c4eca
+size 49492190528

Q4_K/Step-3.5-Flash-Q4_K-00003-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:540a1de66a926271bd3a7e7cd0d85d08acbcb0475a8c22547871b4180eb2bb8a
+size 19760959040

Q4_K_M/Step-3.5-Flash-Q4_K_M-00001-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2764fdbdad8f41c010244a70d7760655a3f52e76cff2e0c4cbbbf36bb75da631
+size 49329290240

Q4_K_M/Step-3.5-Flash-Q4_K_M-00002-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:208021a33b531d57f802c7000a0ddec75f810c7ffda415e9aca394a07a129935
+size 49492190528

Q4_K_M/Step-3.5-Flash-Q4_K_M-00003-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e01c3d1a3749eac972f44d6bde579cdcba54402094d73dfec6a4eac9d68018c1
+size 19760959040

Q4_K_S/Step-3.5-Flash-Q4_K_S-00001-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:976341048546e83023dc152070319aae4675518911c2eaa950f2992567e45d44
+size 49569745088

Q4_K_S/Step-3.5-Flash-Q4_K_S-00002-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c7a5c039c9c7ad5c576df9f684331d77da9dedf09155a5f1e3fdcf0eddc67946
+size 49746145760

Q4_K_S/Step-3.5-Flash-Q4_K_S-00003-of-00003.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a72f40d84b757e7d260d61d2f86c7fef81b77a58f5b37d91730c8a4e9cc7718f
+size 12183196896