Tonykip commited on
Commit
5925b98
·
verified ·
1 Parent(s): 747c19a

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ audio/featured_268bc1833e83.wav filter=lfs diff=lfs merge=lfs -text
37
+ audio/featured_46c861c190e2.wav filter=lfs diff=lfs merge=lfs -text
38
+ audio/featured_df287dfc1406.wav filter=lfs diff=lfs merge=lfs -text
DEMO.md ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Say It Back — recording + submission kit
2
+
3
+ **Space:** https://huggingface.co/spaces/Tonykip/say-it-back
4
+
5
+ ## Before you hit record (speed)
6
+ 1. Open the Space, let it fully load.
7
+ 2. Click **2–3 featured phrases once** to warm everything (they're instant after).
8
+ 3. Type one live phrase and run it once to warm the translate endpoint (~20s first time, fast after — it caches).
9
+ Then record. On camera everything will feel snappy.
10
+
11
+ ## Demo video (~60–90s)
12
+ 1. **The why (10s):** "Kalenjin is fading in my family — my wife understands a little but thinks in English. I built a tool so she can just say a word in English and hear how to say it in Kalenjin."
13
+ 2. **Instant phrases (15s):** click **Thank you → _Kongoi_**, **Water → _Peek_**, **I love you → _achamin_** — each plays back in the Cheps voice immediately.
14
+ 3. **The live moment (25s):** click the mic, say **"Good morning, how are you?"** → it transcribes → shows **_Amunee karon?_** → speaks it back in Kalenjin. That's the wow.
15
+ 4. **The human close (15s):** your wife (or you) using it for real — that footage IS the Backyard-track story.
16
+
17
+ ## Social post draft (edit freely)
18
+ > Kalenjin is slipping away between generations in my own family. So this weekend I built **Say It Back** 🗣️ — say a word in English, hear it spoken back in Kalenjin, in a fine-tuned Kalenjin voice.
19
+ >
20
+ > Whisper (ASR) → EN→Kalenjin translation → Cheps TTS, wired up in Gradio.
21
+ > Built for #BuildSmall (@huggingface × @Gradio). Models on @modal. 🇰🇪
22
+ >
23
+ > Try it: https://huggingface.co/spaces/Tonykip/say-it-back
24
+
25
+ ## Submission checklist
26
+ - [ ] Gradio Space public + working ✅ (this Space)
27
+ - [ ] Demo video recorded
28
+ - [ ] Social post published (tag HF, Gradio, Modal)
29
+ - [ ] Submit the Space link via the hackathon form / channel
30
+ - [ ] (Optional) ask org admins to add you as writer on `build-small-hackathon` if org-hosting is required, then duplicate the Space there.
README.md CHANGED
@@ -1,13 +1,41 @@
1
  ---
2
- title: Say It Back
3
- emoji: 📊
4
- colorFrom: pink
5
- colorTo: yellow
6
  sdk: gradio
7
- sdk_version: 6.18.0
8
- python_version: '3.13'
9
  app_file: app.py
10
  pinned: false
 
 
 
 
 
 
 
 
 
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Say It Back in Kalenjin
3
+ emoji: 🗣️
4
+ colorFrom: yellow
5
+ colorTo: red
6
  sdk: gradio
7
+ sdk_version: 5.49.1
 
8
  app_file: app.py
9
  pinned: false
10
+ short_description: Say a word in English, hear it in Kalenjin
11
+ tags:
12
+ - build-small-hackathon
13
+ - backyard-ai
14
+ - tiny-titan
15
+ - best-demo
16
+ - speech
17
+ - low-resource-languages
18
+ - kalenjin
19
  ---
20
 
21
+ # 🗣️ Say It Back in Kalenjin
22
+
23
+ Say a word in English. Hear it in Kalenjin, in a real Kalenjin voice.
24
+
25
+ Kalenjin is slipping away in my family. My wife thinks in English and wants to
26
+ learn it; this lets her say a word and hear how to say it back in Kalenjin.
27
+
28
+ ## How it works
29
+ Speak (or type) English → Whisper transcribes it → an English→Swahili→Kalenjin
30
+ cascade translates it → **Cheps** (a Kalenjin TTS I fine-tuned) speaks it.
31
+ Common phrases are pre-rendered so they play instantly.
32
+
33
+ ## Build Small (Hugging Face × Gradio)
34
+ - **Track:** 🏡 Backyard AI.
35
+ - **🐣 Tiny Titan:** every model is ≤4B — Whisper 74M (ASR), NLLB 600M ×2 in the
36
+ cascade (translation), Orpheus 3B + LoRA (the Cheps voice). Small weights, real
37
+ impact for a language with almost no tooling.
38
+ - **Demo video:** _TODO: link_
39
+ - **Social post:** _TODO: link_
40
+
41
+ Models run on Modal; this Gradio Space ties them together.
app.py ADDED
@@ -0,0 +1,173 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Walutik (Say It Back) — say a word in English, hear it in Kalenjin.
2
+
3
+ English speech --(Whisper)--> English text
4
+ --(Kalenjin cascade, Modal)--> Kalenjin text
5
+ --(Cheps TTS, Modal)--> Kalenjin audio (auto-plays)
6
+
7
+ Built for the HF × Gradio "Build Small" hackathon — Backyard AI track.
8
+ """
9
+
10
+ from __future__ import annotations
11
+
12
+ import functools
13
+ import hashlib
14
+ import json
15
+ import os
16
+
17
+ import gradio as gr
18
+ import requests
19
+
20
+ HERE = os.path.dirname(os.path.abspath(__file__))
21
+
22
+ TRANSLATE_URL = os.environ.get(
23
+ "MODAL_TRANSLATE_URL",
24
+ "https://tonykipkemboi--kalenjin-cascade-api-textcascade-translate.modal.run",
25
+ )
26
+ TTS_URL = os.environ.get(
27
+ "MODAL_SYNTHESIZE_URL",
28
+ "https://tonykipkemboi--kalenjin-tts-serve-kalenjintts-fastapi-app.modal.run/synthesize",
29
+ )
30
+ INFERENCE_TOKEN = os.environ.get("INFERENCE_TOKEN", "") # Space secret; never committed
31
+
32
+ AUDIO_DIR = os.path.join(HERE, "audio")
33
+ os.makedirs(AUDIO_DIR, exist_ok=True)
34
+
35
+ try:
36
+ with open(os.path.join(HERE, "phrasebook.json"), encoding="utf-8") as f:
37
+ PHRASEBOOK = json.load(f)
38
+ except FileNotFoundError:
39
+ PHRASEBOOK = {}
40
+ FEATURED = list(PHRASEBOOK.keys()) or [
41
+ "Thank you", "Good morning", "How are you?", "I love you",
42
+ "Water", "Milk", "My child", "Come and eat",
43
+ ]
44
+
45
+ # ── English ASR (small Whisper, loaded once) ─────────────────────────────────
46
+ _whisper = None
47
+
48
+
49
+ def _get_whisper():
50
+ global _whisper
51
+ if _whisper is None:
52
+ from faster_whisper import WhisperModel
53
+
54
+ _whisper = WhisperModel("base.en", device="cpu", compute_type="int8")
55
+ return _whisper
56
+
57
+
58
+ def transcribe_english(audio_path):
59
+ if not audio_path:
60
+ return ""
61
+ segments, _ = _get_whisper().transcribe(audio_path, language="en", beam_size=1)
62
+ return " ".join(s.text for s in segments).strip()
63
+
64
+
65
+ # ── translate + synthesize (cached for instant repeats) ──────────────────────
66
+ @functools.lru_cache(maxsize=512)
67
+ def translate_en_to_kal(text):
68
+ r = requests.post(TRANSLATE_URL, json={"text": text}, timeout=70)
69
+ r.raise_for_status()
70
+ d = r.json()
71
+ return (d.get("kalenjin", "").strip(), d.get("swahili", "").strip())
72
+
73
+
74
+ @functools.lru_cache(maxsize=512)
75
+ def synthesize_kalenjin(text):
76
+ headers = {"Content-Type": "application/json"}
77
+ if INFERENCE_TOKEN:
78
+ headers["Authorization"] = f"Bearer {INFERENCE_TOKEN}"
79
+ r = requests.post(TTS_URL, json={"text": text}, headers=headers, timeout=150)
80
+ r.raise_for_status()
81
+ key = hashlib.sha256(text.encode()).hexdigest()[:16]
82
+ path = os.path.join(AUDIO_DIR, f"tts_{key}.wav")
83
+ with open(path, "wb") as f:
84
+ f.write(r.content)
85
+ return path
86
+
87
+
88
+ def say_it_back(typed_text, audio_path):
89
+ english = (typed_text or "").strip()
90
+ if not english and audio_path:
91
+ yield "listening…", "", "", None
92
+ english = transcribe_english(audio_path)
93
+ if not english:
94
+ yield "Say or type something in English.", "", "", None
95
+ return
96
+
97
+ if english in PHRASEBOOK:
98
+ e = PHRASEBOOK[english]
99
+ yield english, e["kalenjin"], f"Swahili: {e['swahili']}", os.path.join(HERE, e["wav"])
100
+ return
101
+
102
+ yield english, "translating…", "", None
103
+ try:
104
+ kalenjin, swahili = translate_en_to_kal(english)
105
+ except Exception as e: # noqa: BLE001
106
+ yield english, f"translation failed: {e}", "", None
107
+ return
108
+ if not kalenjin:
109
+ yield english, "no translation came back.", "", None
110
+ return
111
+
112
+ yield english, kalenjin, f"Swahili: {swahili}", None
113
+ try:
114
+ wav = synthesize_kalenjin(kalenjin)
115
+ except Exception as e: # noqa: BLE001
116
+ yield english, kalenjin, f"Swahili: {swahili} · voice failed: {e}", None
117
+ return
118
+ yield english, kalenjin, f"Swahili: {swahili}", wav
119
+
120
+
121
+ def featured_click(phrase):
122
+ e = PHRASEBOOK.get(phrase)
123
+ if e:
124
+ return phrase, e["kalenjin"], f"Swahili: {e['swahili']}", os.path.join(HERE, e["wav"])
125
+ kalenjin, swahili = translate_en_to_kal(phrase)
126
+ return phrase, kalenjin, f"Swahili: {swahili}", synthesize_kalenjin(kalenjin)
127
+
128
+
129
+ # ── UI (mobile-first single column) ──────────────────────────────────────────
130
+ CSS = """
131
+ .gradio-container { max-width: 640px !important; margin: auto !important; }
132
+ .kal-out textarea { font-size: 32px !important; font-weight: 700 !important; color:#b34700 !important; text-align:center; }
133
+ .en-out textarea { font-size: 18px !important; }
134
+ footer { visibility: hidden; }
135
+ """
136
+
137
+ with gr.Blocks(title="Say It Back in Kalenjin", theme=gr.themes.Soft(primary_hue="orange"), css=CSS) as demo:
138
+ gr.Markdown(
139
+ "# 🗣️ Say It Back in Kalenjin\n"
140
+ "**Say a word in English. Hear it in Kalenjin.**\n\n"
141
+ "Kalenjin is slipping away in my family, so I built this to help us learn "
142
+ "it back. The voice is Cheps, a Kalenjin TTS I fine-tuned."
143
+ )
144
+
145
+ with gr.Group():
146
+ mic = gr.Audio(sources=["microphone"], type="filepath", label="🎤 Say it in English")
147
+ txt = gr.Textbox(label="…or type English", placeholder="e.g. good morning, how are you?")
148
+ go = gr.Button("Say it back →", variant="primary", size="lg")
149
+
150
+ with gr.Group():
151
+ kal_out = gr.Textbox(label="Kalenjin", interactive=False, elem_classes="kal-out")
152
+ audio_out = gr.Audio(label="🔊 Kalenjin voice (Cheps)", autoplay=True, interactive=False)
153
+ en_out = gr.Textbox(label="You said", interactive=False, elem_classes="en-out")
154
+ via = gr.Markdown("")
155
+
156
+ gr.Markdown("**Tap a phrase to hear it:**")
157
+ with gr.Row():
158
+ for _phrase in FEATURED:
159
+ gr.Button(_phrase, size="sm").click(
160
+ featured_click, inputs=gr.State(_phrase),
161
+ outputs=[en_out, kal_out, via, audio_out],
162
+ )
163
+
164
+ gr.Markdown(
165
+ "<sub>Tiny by design — every model is small: Whisper 74M (speech), "
166
+ "NLLB 600M (translation), Cheps / Orpheus 3B (voice). All under 4B.</sub>"
167
+ )
168
+
169
+ go.click(say_it_back, inputs=[txt, mic], outputs=[en_out, kal_out, via, audio_out])
170
+ txt.submit(say_it_back, inputs=[txt, mic], outputs=[en_out, kal_out, via, audio_out])
171
+
172
+ if __name__ == "__main__":
173
+ demo.queue().launch()
audio/featured_268bc1833e83.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b82e864c0cd618cc33f00e82581bb203709bee54ece8398ca97856b9c5c52002
3
+ size 121596
audio/featured_46c861c190e2.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af4c9018e27b416b1d898ca960617a3f31dfd2d7c15bbc57dea926b63b81cd0d
3
+ size 155950
audio/featured_7ca7dea90680.wav ADDED
Binary file (22.6 kB). View file
 
audio/featured_8189b8409b18.wav ADDED
Binary file (28.7 kB). View file
 
audio/featured_90a90a48e23d.wav ADDED
Binary file (95.4 kB). View file
 
audio/featured_c33084feaa65.wav ADDED
Binary file (42.7 kB). View file
 
audio/featured_df287dfc1406.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6518e24cf758b481c9c8e670bc9ac55a1394367f0b75bd4d3dc349f399c3ede3
3
+ size 105684
audio/featured_ea4c0f6bd91d.wav ADDED
Binary file (95.9 kB). View file
 
phrasebook.json ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "Thank you": {
3
+ "kalenjin": "Kongoi",
4
+ "swahili": "Asanteni",
5
+ "wav": "audio/featured_268bc1833e83.wav"
6
+ },
7
+ "Good morning": {
8
+ "kalenjin": "Amunee karon",
9
+ "swahili": "Habari za asubuhi",
10
+ "wav": "audio/featured_90a90a48e23d.wav"
11
+ },
12
+ "How are you?": {
13
+ "kalenjin": "Inye itestai ano?",
14
+ "swahili": "Wewe unaendeleaje?",
15
+ "wav": "audio/featured_df287dfc1406.wav"
16
+ },
17
+ "I love you": {
18
+ "kalenjin": "achamin",
19
+ "swahili": "Nakupenda",
20
+ "wav": "audio/featured_c33084feaa65.wav"
21
+ },
22
+ "Water": {
23
+ "kalenjin": "Peek",
24
+ "swahili": "Maji",
25
+ "wav": "audio/featured_7ca7dea90680.wav"
26
+ },
27
+ "Milk": {
28
+ "kalenjin": "Chego",
29
+ "swahili": "Maziwa",
30
+ "wav": "audio/featured_8189b8409b18.wav"
31
+ },
32
+ "My child": {
33
+ "kalenjin": "Lakwenyu",
34
+ "swahili": "Mtoto wangu",
35
+ "wav": "audio/featured_ea4c0f6bd91d.wav"
36
+ },
37
+ "Come and eat": {
38
+ "kalenjin": "Nyon any amisye",
39
+ "swahili": "Njoo na kula",
40
+ "wav": "audio/featured_46c861c190e2.wav"
41
+ }
42
+ }
prerender.py ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Pre-render the featured phrases (English -> Kalenjin -> Cheps WAV) and write
2
+ a phrasebook.json + audio/featured_*.wav bundle so they play back INSTANTLY in
3
+ the app (no live Modal call on the hot demo path). Run locally before deploy:
4
+
5
+ INFERENCE_TOKEN=... python prerender.py
6
+ """
7
+
8
+ import hashlib
9
+ import json
10
+ import os
11
+
12
+ import requests
13
+
14
+ TRANSLATE_URL = "https://tonykipkemboi--kalenjin-cascade-api-textcascade-translate.modal.run"
15
+ TTS_URL = "https://tonykipkemboi--kalenjin-tts-serve-kalenjintts-fastapi-app.modal.run/synthesize"
16
+ TOKEN = os.environ["INFERENCE_TOKEN"]
17
+
18
+ FEATURED = [
19
+ "Thank you", "Good morning", "How are you?", "I love you",
20
+ "Water", "Milk", "My child", "Come and eat",
21
+ ]
22
+
23
+ HERE = os.path.dirname(os.path.abspath(__file__))
24
+ AUDIO = os.path.join(HERE, "audio")
25
+ os.makedirs(AUDIO, exist_ok=True)
26
+
27
+ book = {}
28
+ for phrase in FEATURED:
29
+ t = requests.post(TRANSLATE_URL, json={"text": phrase}, timeout=70).json()
30
+ kal, sw = t.get("kalenjin", "").strip(), t.get("swahili", "").strip()
31
+ wav = requests.post(
32
+ TTS_URL, json={"text": kal},
33
+ headers={"Authorization": f"Bearer {TOKEN}"}, timeout=150,
34
+ ).content
35
+ key = hashlib.sha256(phrase.encode()).hexdigest()[:12]
36
+ fname = f"featured_{key}.wav"
37
+ with open(os.path.join(AUDIO, fname), "wb") as f:
38
+ f.write(wav)
39
+ book[phrase] = {"kalenjin": kal, "swahili": sw, "wav": f"audio/{fname}"}
40
+ print(f" {phrase!r:22} -> {kal!r:28} ({len(wav)} bytes)")
41
+
42
+ with open(os.path.join(HERE, "phrasebook.json"), "w") as f:
43
+ json.dump(book, f, ensure_ascii=False, indent=2)
44
+ print(f"wrote phrasebook.json ({len(book)} phrases)")
requirements.txt ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ requests>=2.31
2
+ faster-whisper>=1.1.0