Instructions to use Priyansu19/rr-llama-1b-v2_gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use Priyansu19/rr-llama-1b-v2_gguf with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="Priyansu19/rr-llama-1b-v2_gguf", filename="model.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Priyansu19/rr-llama-1b-v2_gguf with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Priyansu19/rr-llama-1b-v2_gguf # Run inference directly in the terminal: llama-cli -hf Priyansu19/rr-llama-1b-v2_gguf
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Priyansu19/rr-llama-1b-v2_gguf # Run inference directly in the terminal: llama-cli -hf Priyansu19/rr-llama-1b-v2_gguf
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Priyansu19/rr-llama-1b-v2_gguf # Run inference directly in the terminal: ./llama-cli -hf Priyansu19/rr-llama-1b-v2_gguf
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Priyansu19/rr-llama-1b-v2_gguf # Run inference directly in the terminal: ./build/bin/llama-cli -hf Priyansu19/rr-llama-1b-v2_gguf
Use Docker
docker model run hf.co/Priyansu19/rr-llama-1b-v2_gguf
- LM Studio
- Jan
- Ollama
How to use Priyansu19/rr-llama-1b-v2_gguf with Ollama:
ollama run hf.co/Priyansu19/rr-llama-1b-v2_gguf
- Unsloth Studio
How to use Priyansu19/rr-llama-1b-v2_gguf with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Priyansu19/rr-llama-1b-v2_gguf to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Priyansu19/rr-llama-1b-v2_gguf to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Priyansu19/rr-llama-1b-v2_gguf to start chatting
- Docker Model Runner
How to use Priyansu19/rr-llama-1b-v2_gguf with Docker Model Runner:
docker model run hf.co/Priyansu19/rr-llama-1b-v2_gguf
- Lemonade
How to use Priyansu19/rr-llama-1b-v2_gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Priyansu19/rr-llama-1b-v2_gguf
Run and chat with the model
lemonade run user.rr-llama-1b-v2_gguf-{{QUANT_TAG}}List all available models
lemonade list
| LLAMA 3.2 COMMUNITY LICENSE AGREEMENT | |
| Llama 3.2 Version Release Date: September 25, 2024 | |
| “Agreement” means the terms and conditions for use, reproduction, distribution | |
| and modification of the Llama Materials set forth herein. | |
| “Documentation” means the specifications, manuals and documentation accompanying Llama 3.2 | |
| distributed by Meta at https://llama.meta.com/doc/overview. | |
| “Licensee” or “you” means you, or your employer or any other person or entity (if you are | |
| entering into this Agreement on such person or entity’s behalf), of the age required under | |
| applicable laws, rules or regulations to provide legal consent and that has legal authority | |
| to bind your employer or such other person or entity if you are entering in this Agreement | |
| on their behalf. | |
| “Llama 3.2” means the foundational large language models and software and algorithms, including | |
| machine-learning model code, trained model weights, inference-enabling code, training-enabling code, | |
| fine-tuning enabling code and other elements of the foregoing distributed by Meta at | |
| https://www.llama.com/llama-downloads. | |
| “Llama Materials” means, collectively, Meta’s proprietary Llama 3.2 and Documentation (and | |
| any portion thereof) made available under this Agreement. | |
| “Meta” or “we” means Meta Platforms Ireland Limited (if you are located in or, | |
| if you are an entity, your principal place of business is in the EEA or Switzerland) | |
| and Meta Platforms, Inc. (if you are located outside of the EEA or Switzerland). | |
| By clicking “I Accept” below or by using or distributing any portion or element of the Llama Materials, | |
| you agree to be bound by this Agreement. | |
| 1. License Rights and Redistribution. | |
| a. Grant of Rights. You are granted a non-exclusive, worldwide, | |
| non-transferable and royalty-free limited license under Meta’s intellectual property or other rights | |
| owned by Meta embodied in the Llama Materials to use, reproduce, distribute, copy, create derivative works | |
| of, and make modifications to the Llama Materials. | |
| b. Redistribution and Use. | |
| i. If you distribute or make available the Llama Materials (or any derivative works thereof), | |
| or a product or service (including another AI model) that contains any of them, you shall (A) provide | |
| a copy of this Agreement with any such Llama Materials; and (B) prominently display “Built with Llama” | |
| on a related website, user interface, blogpost, about page, or product documentation. If you use the | |
| Llama Materials or any outputs or results of the Llama Materials to create, train, fine tune, or | |
| otherwise improve an AI model, which is distributed or made available, you shall also include “Llama” | |
| at the beginning of any such AI model name. | |
| ii. If you receive Llama Materials, or any derivative works thereof, from a Licensee as part | |
| of an integrated end user product, then Section 2 of this Agreement will not apply to you. | |
| iii. You must retain in all copies of the Llama Materials that you distribute the | |
| following attribution notice within a “Notice” text file distributed as a part of such copies: | |
| “Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, | |
| Inc. All Rights Reserved.” | |
| iv. Your use of the Llama Materials must comply with applicable laws and regulations | |
| (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for | |
| the Llama Materials (available at https://www.llama.com/llama3_2/use-policy), which is hereby | |
| incorporated by reference into this Agreement. | |
| 2. Additional Commercial Terms. If, on the Llama 3.2 version release date, the monthly active users | |
| of the products or services made available by or for Licensee, or Licensee’s affiliates, | |
| is greater than 700 million monthly active users in the preceding calendar month, you must request | |
| a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to | |
| exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights. | |
| 3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE LLAMA MATERIALS AND ANY OUTPUT AND | |
| RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND META DISCLAIMS | |
| ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES | |
| OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE | |
| FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE LLAMA MATERIALS AND ASSUME ANY RISKS ASSOCIATED | |
| WITH YOUR USE OF THE LLAMA MATERIALS AND ANY OUTPUT AND RESULTS. | |
| 4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, | |
| WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, | |
| FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN | |
| IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING. | |
| 5. Intellectual Property. | |
| a. No trademark licenses are granted under this Agreement, and in connection with the Llama Materials, | |
| neither Meta nor Licensee may use any name or mark owned by or associated with the other or any of its affiliates, | |
| except as required for reasonable and customary use in describing and redistributing the Llama Materials or as | |
| set forth in this Section 5(a). Meta hereby grants you a license to use “Llama” (the “Mark”) solely as required | |
| to comply with the last sentence of Section 1.b.i. You will comply with Meta’s brand guidelines (currently accessible | |
| at https://about.meta.com/brand/resources/meta/company-brand/). All goodwill arising out of your use of the Mark | |
| will inure to the benefit of Meta. | |
| b. Subject to Meta’s ownership of Llama Materials and derivatives made by or for Meta, with respect to any | |
| derivative works and modifications of the Llama Materials that are made by you, as between you and Meta, | |
| you are and will be the owner of such derivative works and modifications. | |
| c. If you institute litigation or other proceedings against Meta or any entity (including a cross-claim or | |
| counterclaim in a lawsuit) alleging that the Llama Materials or Llama 3.2 outputs or results, or any portion | |
| of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable | |
| by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or | |
| claim is filed or instituted. You will indemnify and hold harmless Meta from and against any claim by any third | |
| party arising out of or related to your use or distribution of the Llama Materials. | |
| 6. Term and Termination. The term of this Agreement will commence upon your acceptance of this Agreement or access | |
| to the Llama Materials and will continue in full force and effect until terminated in accordance with the terms | |
| and conditions herein. Meta may terminate this Agreement if you are in breach of any term or condition of this | |
| Agreement. Upon termination of this Agreement, you shall delete and cease use of the Llama Materials. Sections 3, | |
| 4 and 7 shall survive the termination of this Agreement. | |
| 7. Governing Law and Jurisdiction. This Agreement will be governed and construed under the laws of the State of | |
| California without regard to choice of law principles, and the UN Convention on Contracts for the International | |
| Sale of Goods does not apply to this Agreement. The courts of California shall have exclusive jurisdiction of | |
| any dispute arising out of this Agreement. | |