Hugging Face's logo

Audio8
/
audio8-TTS-0.1B-ONNX-INT8

Text-to-Speech
ONNX
onnxruntime
arktts
int8
audio
tts
voice-cloning
zero-shot
multilingual
custom_code
Model card Files
xet
 Community
3

0.1B INT8 的 AR token 生成链路异常

#2
by Yulin2 - opened
Discussion
Yulin2
 
INFO:     127.0.0.1:55774 - "POST /api/tts HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 416, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 63, in __call__
    return await self.app(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1163, in __call__
    await super().__call__(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/applications.py", line 96, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 2734, in app
    await route.handle(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 1281, in handle
    await super().handle(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/routing.py", line 280, in handle
    await self.app(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 158, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 144, in app
    response = await f(request)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 706, in app
    raw_response = await run_endpoint_function(
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 354, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/starlette/concurrency.py", line 34, in run_in_threadpool
    return await anyio.to_thread.run_sync(func)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/anyio/to_thread.py", line 65, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2641, in run_sync_in_worker_thread
    return await future
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1033, in run
    result = context.run(func, *args)
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/onnx_runtime_0_1b_int8/arktts_runtime/service.py", line 202, in tts
    audio, _ = obj.synthesize(
  File "/home/yyl/RapidSpeech.cpp/Audio8_TTS/onnx_runtime_0_1b_int8/arktts_runtime/runtime.py", line 436, in synthesize
    raise RuntimeError("model produced no codec frames")
RuntimeError: model produced no codec frames

onnx 0.1b int8生成tts失败，0.6b的可以成功，如果根据readme来，注册的default音色可以顺利推理，但是音频是乱码，不可用。
chua
 Audio8 org
•
edited 1 day ago

感谢反馈。我们已定位并修复 /api/tts 返回 500 的问题。

原因

0.1B Slow AR 输出 4097 维 compact logits，其中最后一项是 EOS。原运行时在首个生成步采样到 EOS 时会直接结束，未生成任何 codec frame，最终触发：

RuntimeError: model produced no codec frames

INT8 量化和不同 ORT CPU kernel 可能导致首步 EOS 在不同环境下偶发出现。

请使用最新的 GitHub master 和该 HF 模型版本测试。如仍有问题，请附上请求 JSON、操作系统、CPU、onnxruntime 版本、seed，以及生成的 .npy 或 WAV 信息。

------------ English Version --------------

Thanks for reporting this. We tracked down and fixed the /api/tts HTTP 500 issue.

The 0.1B Slow AR model outputs 4,097 compact logits: 4,096 semantic logits plus one EOS logit. With the old runtime, if EOS was sampled on the very first generation step, decoding stopped immediately without producing any codec frames. That eventually caused "RuntimeError: model produced no codec frames".

Because of INT8 quantization and small differences between ORT CPU kernels, this could happen intermittently depending on the environment and random seed. Please try the latest GitHub master branch together with this HF model revision. If the problem still occurs, please share the request JSON, OS, CPU, onnxruntime version, seed, and basic information about the generated .npy or WAV file.

Sign up or log in to comment