Update usage example with infinity (#13)
Browse files- Update usage example with infinity (3d61b474c3e21454d937ce470ef7f402dbff9559)
Co-authored-by: Michael <michaelfeil@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -2713,6 +2713,14 @@ const similarities = document_embeddings.map(x => 100 * dot(source_embeddings, x
|
|
| 2713 |
console.log(similarities); // [34.504930869007296, 64.03973265120138, 19.520042686034362]
|
| 2714 |
```
|
| 2715 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2716 |
## Training Details
|
| 2717 |
|
| 2718 |
### Training Data
|
|
|
|
| 2713 |
console.log(similarities); // [34.504930869007296, 64.03973265120138, 19.520042686034362]
|
| 2714 |
```
|
| 2715 |
|
| 2716 |
+
Use with infinity:
|
| 2717 |
+
[Infinity](https://github.com/michaelfeil/infinity) is a MIT licensed server for OpenAI-compatible deployment.
|
| 2718 |
+
```
|
| 2719 |
+
docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
|
| 2720 |
+
michaelf34/infinity:0.0.68 \
|
| 2721 |
+
v2 --model-id Alibaba-NLP/gte-base-en-v1.5 --revision "4c742dc2b781e4ab062a4a77f4f7cbad4bdee970" --dtype bfloat16 --batch-size 32 --device cuda --engine torch --port 7997
|
| 2722 |
+
```
|
| 2723 |
+
|
| 2724 |
## Training Details
|
| 2725 |
|
| 2726 |
### Training Data
|