Self-host audio.cpp
v0.9.5High-performance C++ audio inference for local AI models
- Stars
- 4 309
- Port
- 8080 (HTTP server and WebUI)
ghcr.io/0xshug0/audio.cppWhat is audio.cpp?
This C++ inference framework builds on ggml to run modern audio AI models locally without Python. It covers over 110 model families across TTS, STT, voice cloning, conversion, music generation, separation, VAD, diarization, alignment, and codec tasks. You get a CLI, an HTTP server with OpenAI-compatible endpoints, and an embedded WebUI — all from a single binary. It works on Windows, Linux, and macOS using CUDA, AMD ROCm, Vulkan, Metal, or CPU backends. Daily Docker images are on ghcr.io. Licensed Apache-2.0 with no external service dependencies; models load from local GGUF or safetensors files. The server defaults to port 8080 and supports lazy loading, residency limits, and streaming where models allow.
audio.cpp alternatives
voice-chat-aiSelf-hosted open-source voice chat app for talking to AI characters in a web UI or terminal, mixing Ollama and cloud LLMs with Spark-TTS, Kokoro or ElevenLabs voices.Comparevoice-chat-ai with audio.cpp
koboldcppSelf-hosted, open-source (AGPL-3.0) single-file GGUF LLM server with a bundled KoboldAI Lite web UI and OpenAI-, KoboldAI- and Ollama-compatible API endpoints.Comparekoboldcpp with audio.cpp
SpeachesSelf-hosted OpenAI Audio API server for transcription, translation, and speech synthesis using faster-whisper, piper, and Kokoro models.CompareSpeaches with audio.cpp
openai-edge-ttsA self-hosted, open-source text-to-speech API server that mimics OpenAI's /v1/audio/speech endpoint and synthesizes audio for free via Microsoft Edge's TTS voices.Compareopenai-edge-tts with audio.cpp
transcriberSelf-hosted, open-source web app for transcribing audio/video with Whisper. Supports YouTube, local files, real-time progress, dark mode.Comparetranscriber with audio.cpp
ebook2audiobook---LegacySelf-hosted, open-source converter that turns ebooks and PDFs into audiobooks using multiple TTS engines, voice cloning, OCR and 1158 languages.Compareebook2audiobook---Legacy with audio.cpp
whisper-transcriber-botSelf-hosted Telegram bot that converts voice messages and audio files into text using a local AI model, running on CPU without needing a GPU or cloud APIs.Comparewhisper-transcriber-bot with audio.cpp
whisper-asr-webserviceSelf-hosted, open-source speech-to-text webservice wrapping OpenAI Whisper, Faster Whisper and WhisperX behind a REST API with Swagger docs.Comparewhisper-asr-webservice with audio.cpp
Should you self-host audio.cpp?
audio.cpp scores 75/100 on selfhost.directory (how scoring works) and ranks #14 of 123 in AI. It has 4 309 GitHub stars, +474 in the last 30 days.
It shipped 11 stable releases in the past year. The latest, v0.9.5, came out on 14 November 2026. New versions land roughly every 7 days.
Setup is advanced: plan for several services and some ongoing maintenance. An official Docker image is available. It is Apache-2.0-licensed and written mostly in C++.
Its closest alternative on selfhost.directory is openclaw (93/100, 394 614 stars). It scores higher overall, so compare the two before you pick. See audio.cpp vs openclaw side by side.
Features
- GGUF Model Format
- JSON PipelinesExperimental
- Multi-backend Support
- Music Generation
- Source Separation
- Speech-to-Text (ASR)
- Streaming Inference
- Text-to-Speech (TTS)
- Voice Cloning
- Voice Conversion
audio.cpp README
audio.cpp is a high-performance C++ audio inference framework built on top of ggml, designed to make modern local audio models practical, portable, and fast.
Tired of juggling a dozen Conda environments, hundreds of Python packages, and dependency conflicts just to try a few audio models? audio.cpp gives those paths a shared native runtime instead. Runs on Windows, Linux, and macOS, with support for NVIDIA, AMD, Apple Silicon, and CPU-only machines.
How to install audio.cpp
Install audio.cpp with Docker
docker run --rm --gpus all -v ":/models:ro" ghcr.io/0xshug0/audio.cpp:full-cuda12 <cli|server> --model /models/ <...>
CUDA variant requires NVIDIA container toolkit. Also available: full-cuda13, full-vulkan, full-cpu, full-rockchip (arm64).
Install audio.cpp with Homebrew
brew tap 0xShug0/audio-cpp brew trust 0xShug0/audio-cpp brew install audio-cpp
Installs audio-cpp package from custom tap
Install audio.cpp with Build from Source
cmake -S . -B build -DENGINE_ENABLE_CUDA=ON cmake --build build --target audiocpp_cli audiocpp_server
Requires GCC 13+, CMake, and backend toolchain (CUDA Toolkit, Vulkan SDK, ROCm, Xcode). Helper scripts available: scripts/build_linux.sh, scripts/build_windows.ps1, scripts/build_metal.sh
How to configure audio.cpp
host=127.0.0.1Server bind addressport=8080Server listen portbackend=cpuInference backendvalues: cuda, cpu, hip, rocm, vulkan, metal
device=0Backend device indexthreads=4Backend/OpenMP worker threads
audio.cpp releases & version history
What's Changed
- Feature/prefix attention return full kv by @apt-install-coffee in #1205
- Add segment streaming and text controls to IndexTTS by @david-bowiegxw in #1206
- Sort saved voices and preserve playback while editing by @0xShug0 in #1207
- Refactor LFM3 Audio around shared LFM runtimes and weight helpers by @0xShug0 in #1210
- kitten_tts2: apply repetition penalty to the silence token by @Twistarkle in #1211
- Refactor Nemotron models with shared RoPE encoder and host CTC decoding by @0xShug0 in #1213
- omnivoice: fix CPU voice cloning (non-contiguous residual into ggml_sub in the audio tokenizer) by @ahmednabilfawzy89-cmyk in #1212
- Reuse shared host CTC decoding across ASR models by @0xShug0 in #1214
- Reduce Nemotron Transcribe transfers and unused prefill work by @0xShug0 in #1215
- Reduce streaming delay by @0xShug0 in #1216
- Reduce voice chat UI playback delay by @0xShug0 in #1219
- docs(drama3): document V2 checkpoint and comparison by @JarodMica in #1217
- Refactor HeartMuLa runtime and reduce memory usage by @0xShug0 in #1224
- package_manager: native Hugging Face Xet downloads by @drphero in #1222
- Add MuLaCover cover-song generation by @0xShug0 in #1226
- OmniVoice: native batching and faster generation by @kawshikbuet17 in #1208
- Allow configuring the WebUI models folder at server startup by @0xShug0 in #1227
- docs: add LocalText2Voice to the Projects list by @estebanstifli in #1228
- sanotts: return word timestamps with the NEON decoder by @tarelerulz in #1221
- pocket_tts: honour capitalize_first_letter from config.yaml by @michsethowusu in #1229
- Add MOSS-SoundEffect v2 sound-effect generation by @0xShug0 in #1232
- Refactor Drama3 and Nemotron Transcribe with shared MoE execution by @0xShug0 in #1234
- Refactor Ming3 Audio with shared MoE execution by @0xShug0 in #1238
- sense_asr: return word timestamps from CTC emission frames by @jiabochao in #1236
- feat: expose C API logging callback (audiocpp_log_set) by @kikaitachi in #1235
- Remove logging-only tensor readbacks and complete session timing by @0xShug0 in #1240
- Add SYCL backend support by @newjordan in #1241
- ggml-sycl: kernels and graph replay for audio models by @newjordan in #1242
- docs: add SYCL validation report by @newjordan in #1247
- ggml-sycl: add CONV_2D_DW and BF16/I32 CPY conversions by @newjordan in #1246
- Add opt-in BF16 activation rounding for Qwen3 TTS by @0xShug0 in #1248
- Stop WebUI model status flickering during downloads by @0xShug0 in #1249
- Run YuE2 on the SYCL backend by @newjordan in #1243
- Add session-owned PreparedGraph execution and scheduler support by @0xShug0 in #1250
- Add reusable device denoising and MOSS SoundEffect fusions by @0xShug0 in #1251
- server: accept inline voice_ref in task requests by @magnus-dev-melin in #1252
- 🚨 Add shared DiT runtime and migrate denoiser models by @0xShug0 in #1254
- server: return 400 for an inline voice_ref that is not a WAV by @magnus-dev-melin in #1255
- chatterbox: cache prepared voice-conversion targets by @magnus-dev-melin in #1253
- Refactor model variant selection into explicit routing tables by @0xShug0 in #1256
- fish_drama3: increase default text chunk size to 4000 by @JarodMica in #1257
- Revert "fish_drama3: increase default text chunk size to 4000" by @0xShug0 in #1258
- Add Edda ASR with DFlash2 speculative decoding (edda-v0.2-large-hurtig) by @lkarlslund in #1223
- docs(drama3): explain chunk size tuning for repeated speech by @JarodMica in #1260
- omnivoice: RKNN generator step-graph sidecar (RK3588) by @wOvAN in #1259
- docs: OmniVoice concurrency benchmark report (RTX 4090, C++ vs Python, f16/bf16) by @kawshikbuet17 in #1262
- omnivoice: front-shard RKNN ladder so guidance works on the NPU by @wOvAN in #1261
- docs: record the vendor zero-op matmul single-core ceiling by @wOvAN in #1264
- Fix WebUI ID generation over plain HTTP by @0xShug0 in #1265
- docs(rockchip-npu): price the vendor zero-op matmul path, record the QuaRot int8 negative by @wOvAN in #1266
- Add opt-in RMSNorm lowering and audio runtime memory controls by @0xShug0 in #1267
- Use PreparedGraph build contexts in Nemotron Transcribe and Ming3 Audio by @0xShug0 in #1269
- Add experimental MoE hybrid CPU and GPU expert execution by @0xShug0 in #1270
- Add Kroko XL offline ASR and improve Zipformer inference by @0xShug0 in #1273
- pocket_tts: support flow_lm.z_proj when z_dim differs from transformer d_model by @michsethowusu in #1239
- Promote Qwen3 audio tokenizer into reusable framework codecs by @0xShug0 in #1276
- feat: add reference-audio voice cloning to KugelAudio by @mirek190 in #1275
- Add MossFormer2 speech enhancement and super-resolution by @0xShug0 in #1277
- New Community Model: ZONOS3-Mini (Zyphra) by @Ryu1845 in #1271
- 🚨 Unify MoE expert offloading through shared framework resources by @0xShug0 in #1280
- Add published models to the Web UI by @0xShug0 in #1279
- New Community Model: ZONOS2 (Zyphra) by @Ryu1845 in #1274
- v0.9.5 cleanup: documentation, model manager and test gating by @0xShug0 in #1281
- Refactor Zonos3 execution and preserve chunked voice consistency by @0xShug0 in #1282
- Add Zonos models to the WebUI and default Audio8 TTS to Realistic by @0xShug0 in #1283
New Contributors
- @david-bowiegxw made their first contribution in #1206
- @Twistarkle made their first contribution in #1211
- @ahmednabilfawzy89-cmyk made their first contribution in #1212
- @drphero made their first contribution in #1222
- @michsethowusu made their first contribution in #1229
- @jiabochao made their first contribution in #1236
- @kikaitachi made their first contribution in #1235
- @newjordan made their first contribution in #1241
- @magnus-dev-melin made their first contribution in #1252
- @Ryu1845 made their first contribution in #1271
Full Changelog: v0.9.4...v0.9.5
What's Changed
- UI: add recipe-based cross-language dubbing and fix Studio/Arena state by @0xShug0 in #1106
- Echo-TTS: Sway time scheduling by @lojack5 in #1105
- Add opt-in VoxCPM2 ultimate cloning by @ersensari in #982
- Add Music Flamingo and SpotSound LoRA support by @0xShug0 in #1107
- feat(webui): allow multiple model package downloads, Fix #1075 by @ZeroDot1 in #1082
- omnivoice: size automatic long-form chunks in code points, not bytes by @linyqh in #1108
- OmniVoice: match Python duration weights for Bengali marks, danda and digits by @kawshikbuet17 in #1110
- Speed up Earshot VAD with per-ISA SIMD kernels by @Etherll in #1102
- OmniVoice: apply the tokenizer's NFC normalizer to input text by @kawshikbuet17 in #1116
- Fix Parakeet offline chunking for stereo and non-16-kHz audio by @0xShug0 in #1117
- Add
--list-devices --jsonwith PCI ids and device memory by @Etherll in #1118 - Fix Vulkan RMSNorm dispatch overflow for long voice references by @0xShug0 in #1120
- Add Magenta RealTime 2 Small and Base with streaming generation by @0xShug0 in #1121
- feat: add lfm3_audio for LFM3-Audio-2B ASR, TTS and S2S by @ykhrustalev in #1119
- fix: return 400 for malformed option values by @ykhrustalev in #1123
- build(nix): use stdenv.hostPlatform for platform checks by @luizribeiro in #1125
- Restore ordered model loader selection by @0xShug0 in #1126
- Move package loading rules into model-owned resolvers by @0xShug0 in #1127
- Fix Arena package selection with restored Studio settings by @0xShug0 in #1128
- new asr model: parakeet_tdt: add Underdog Bark 3.0 Pro English ASR by @mirek190 in #1130
- Add GGML implementation selection and gfx1151_v1 HIP kernels by @0xShug0 in #1134
- fix: bound the resample kernel cache by memory with LRU eviction by @ykhrustalev in #1124
- new asr model: parakeet_tdt: add Underdog Bark 3.0 Plus English ASR by @mirek190 in #1131
- fix: keep lfm2_audio and lfm3_audio fixed ASR chunks within the cap by @ykhrustalev in #1135
- OmniVoice: make the audio tokenizer codebook weight type configurable by @kawshikbuet17 in #1129
- perf: reuse lfm3_audio backbone head and prefill block graphs by @ykhrustalev in #1136
- echo-tts: minor docs cleanup (#1105 followup) by @lojack5 in #1140
- Expose backend implementation selection (rollout is deferred) by @0xShug0 in #1138
- new asr model: parakeet_tdt: add standard Underdog Bark 3.0 English ASR by @mirek190 in #1132
- Preserve backend implementation selection in Qwen3-TTS by @0xShug0 in #1141
- Allow gfx1151_v1 kernel selection on other RDNA GPUs by @0xShug0 in #1142
- fix: fetch BoringSSL with CMake older than 3.24 by @ykhrustalev in #1144
- Load an installed package by id through POST /v1/models/load by @HamSanders in #1098
- Fix OmniVoice instruct splitting for UTF-8 separators by @supomarlo in #1146
- cuda: gate Flash Attention zero-fill workaround to NVCC 13.4 by @askmyteapot in #1143
- Extend HIP batched F32 matmul and IM2COL kernels by @0xShug0 in #1150
- Use Flash Attention for RoFormer separation on Vulkan by @ruihe774 in #1152
- fix: scale Qwen3 forced aligner timestamps to the input sample rate (#1147) by @m-abs in #1148
- community: add Nemotron 3.5 Transcribe 30B-A4B by @JarodMica in #1137
- Add Nemotron Transcribe CPU expert offload and reduce resident memory by @0xShug0 in #1154
- Minor Parakeet graph and resource lifecycle cleanup by @0xShug0 in #1155
- Expose Qwen3-ASR performance controls and cuda_perf preset by @0xShug0 in #1156
- Optimize Parakeet subsampling on Metal by @0xShug0 in #1157
- Magenta RealTime 2: accept the legacy duration_seconds field and add the missing WebUI catalog entry by @noctrex in #1158
- Update WebGPU backend and add audio operator support by @0xShug0 in #1159
- Add native WebGPU backend integration by @0xShug0 in #1160
- Rename Whistle decoder exec_plan implementation to native by @0xShug0 in #1161
- Add browser WebGPU support by @0xShug0 in #1162
- feat(irodori_tts): support Irodori-TTS v4 Large (T5Gemma 2 text encoder) by @KKTTSPJ in #1163
- Extend video VAE runtimes and opt-in CUDA lowerings by @0xShug0 in #1165
- fix(irodori_tts): normalize input text like Python normalize_text by @KKTTSPJ in #1164
- Add opt-in artifact output to generic task streams by @0xShug0 in #1166
- Add SoulX-FlashHead Lite streaming portrait generation by @0xShug0 in #1167
- perf(irodori_tts): read the RF context K/V cache in place by @KKTTSPJ in #1168
- perf(irodori_tts): leave caption tokens out of the RF context without a caption by @KKTTSPJ in #1170
- fix(metal): correct SSM scan token-block reduction indexing by @0xShug0 in #1171
- ggml: add Metal IM2COL_3D support by @0xShug0 in #1169
- ggml: add WebGPU GROUP_NORM support by @0xShug0 in #1172
- perf(metal): optimize SSM scan and convolution by @0xShug0 in #1173
- fix(torch_bin): read checkpoints that wrap the state dict and hold lists by @KKTTSPJ in #1174
- feat(ui): add streaming voice chat with LFM3 and SoulX avatar by @0xShug0 in #1176
- Improve Metal SSM performance by @0xShug0 in #1179
- Synchronize shared storage in Mamba-1 scan by @0xShug0 in #1181
- Separate model runtime tests from framework unit tests by @0xShug0 in #1182
- Correct Vulkan pipeline selection for oversized buffer bindings by @0xShug0 in #1183
- Clean up first-party compiler warnings (WIP) by @0xShug0 in #1184
- Separate Parakeet, Phonon-2, and Bark ASR family routing by @0xShug0 in #1189
- Find hipcub and rocprim in the HIP build by @apt-install-coffee in #1186
- Add Haas Japanese ASR using the shared Parakeet runtime by @0xShug0 in #1191
- Add Real-time RE-USE with incremental audio streaming by @0xShug0 in #1192
- nemotron_transcribe: normalize regional language hints by @JarodMica in #1190
- Add MuScriptor Medium and Large F32 variants by @0xShug0 in #1193
- Add PocketTTS Small with streaming and acoustic batching by @0xShug0 in #1194
- Add WebUI voice library export, import, and sorting by @0xShug0 in #1195
- Keep HIP release pip cache on temporary scratch storage by @0xShug0 in #1196
- Add an opt-in repetition guard for Nemotron Transcribe by @0xShug0 in #1198
- Preserve Drama3 history anchors and fit generation to context by @0xShug0 in #1197
- Add D1-Omni audio decisions and reusable LFM runtimes by @0xShug0 in #1199
- Add opt-in skipping of empty Drama3 chunks by @0xShug0 in #1200
- Refresh model usage guides and consolidate validation reports by @0xShug0 in #1201
- Prepare WebUI 0.9.4 with missing models and app readiness by @0xShug0 in #1202
- Add WebGPU support for 3D im2col by @0xShug0 in #1203
- v0.9.4 cleanup by @0xShug0 in #1204
New Contributors
- @linyqh made their first contribution in #1108
- @luizribeiro made their first contribution in #1125
- @supomarlo made their first contribution in #1146
- @askmyteapot made their first contribution in #1143
- @ruihe774 made their first contribution in #1152
- @m-abs made their first contribution in #1148
Full Changelog: v0.9.3...v0.9.4
What's Changed
- Run the Piper TTS duration path on the host for Vulkan as well by @wOvAN in #913
- Add an opt-in RKNN decoder sidecar for inflect_v2 by @wOvAN in #915
- Modify .gitmodules to work with https and ssh checkouts while maintaining compatibility with repo forks by @simcop2387 in #923
- New Community Model: Fish Audio Drama 3 by @JarodMica in #916
- Reduce Drama 3 VRAM with opt-in disk-backed expert offload by @0xShug0 in #927
- Add Drama 3 voice cloning and multi-speaker WebUI by @0xShug0 in #928
- Fix Drama 3 Vulkan voice conditioning with opt-in F32 encoder accumulation by @0xShug0 in #929
- Fix Drama 3 offload prefill noise and repeat consistency by @0xShug0 in #930
- Optimize Drama 3 output head for speech token candidates by @0xShug0 in #931
- Gate family unit tests on the model set whose symbols they link by @wOvAN in #925
- Keep CPU f5_tts passes inside the graph pool ceiling, and free the evicted graphs by @wOvAN in #924
- Strip the ONNX module prefix in the Inflect v2 converter by @wOvAN in #933
- Fix CPU Fish DAC reference encoding for voice cloning by @0xShug0 in #936
- Align CUDA reference-encoder precision for Fish Audio and Drama 3 by @0xShug0 in #937
- Fix C++17 lambda capture breaking HIP server builds by @0xShug0 in #939
- fix(build): compile model unit tests' own family sources into the test by @wOvAN in #932
- fix(build): split compiled-in model specs into 16 KB literals by @wOvAN in #934
- webui: list the server's voices with UI management enabled by @KKTTSPJ in #941
- docs(piper_tts): the non-finite duration abort is the plain Vulkan path, not the sidecar by @wOvAN in #946
- Split the RKNN quality numbers by what varies, in docs/build/rockchip-npu.md by @wOvAN in #945
- webui: preview server voices in config mode and label the list by content by @KKTTSPJ in #943
- Document the RK3588 IOVA ceiling for engine size by @wOvAN in #949
- perf(qwen3_tts): opt in to voicing chunked text in one decoder pass by @wOvAN in #910
- server: add default_session_options for configured and UI-loaded models by @KKTTSPJ in #944
- Support updated Kitten TTS 2 weights with BF16, Q8 and Q4 GGUFs by @dignome in #947
- perf(voxcpm2): size the step graphs by the request, not by max_length by @wOvAN in #950
- fix(f5_tts): free the evicted graphs' staged CUDA constants by @wOvAN in #953
- feat(cosyvoice3): run the flow DiT estimator on the Rockchip NPU by @wOvAN in #951
- refactor(f5_tts): drop the unused raw DiT weight store by @wOvAN in #956
- Reduce shared WavLM memory usage for MioCodec, MioTTS, and MeanVC2 by @0xShug0 in #954
- server: let voice presets carry request options by @KKTTSPJ in #942
- refactor(rknn): build every engine through one converter helper by @wOvAN in #961
- refactor(rknn): split the aperture accounting from the aperture measurements by @wOvAN in #957
- Fix Drama3 seeded repeatability with sorted expert routing by @0xShug0 in #955
- feat(tensors): add i-quant (IQ) storage types for GGUF tensor loading by @Th-Underscore in #958
- Add opt-in NEON library and native model components by @0xShug0 in #962
- Add Plapre 3 Danish TTS and voice cloning by @0xShug0 in #965
- Align instruction fields across CLI and server by @0xShug0 in #894
- Declare shared model object-library dependencies and document Plapre TTS by @0xShug0 in #966
- Add OpenWakeWord Alexa GGUF package and documentation by @0xShug0 in #968
- Restore Whistle host decoder and add opt-in NEON acceleration by @0xShug0 in #967
- Add opt-in NEON network execution for microWakeWord by @0xShug0 in #969
- feat(fish_drama3): support per-layer expert counts and speech-only head by @Th-Underscore in #959
- fix(server): count exec_plan engines in the free-memory guard by @wOvAN in #964
- server: keep config model settings on WebUI loads, and offer demo voices with --ui by @KKTTSPJ in #971
- fix(rknn): find an exec_plan engine named next to the model by @wOvAN in #963
- webui: treat configured models the same with and without UI management by @KKTTSPJ in #972
- perf(rknn): price a ladder engine by what the driver reports by @wOvAN in #970
- feat(server): support per-model backend device selection by @railep in #973
- perf(fish_dac): chunked reference encoding to cut VRAM peak by @Th-Underscore in #960
- feat(webui): add Vietnamese interface translations by @kinhatula in #839
- Drama3 base/small release cleanup by @0xShug0 in #976
- text chunking: keep closing quotes with their sentence in tag-aware mode by @5uck1ess in #977
- fish_drama3: default top_k to 4097 (no cap), matching Fish's Space by @5uck1ess in #978
- Refactor LoRA adapters and add ComfyUI/LoKr support for YuE2, VibeVoice, and ACE-Step by @0xShug0 in #974
- Polish Studio layout and unify Studio/Arena voice selection by @0xShug0 in #980
- Fix server build with Clang in C++17 mode by @0xShug0 in #986
- refactor(rknn): read the sliced graph through one shared helper by @wOvAN in #983
- Add Linux Clang C++17 CI build by @0xShug0 in #987
- perf(voxcpm2): cheaper AudioVAE decode (exact-size graph, one-pass CPU Snake, optional CPU decode overlap) by @ersensari in #981
- refactor(tools/rknn): one report line and one bootstrap per converter by @wOvAN in #990
- Preserve Higgs Audio voice across reference-free text chunks by @0xShug0 in #985
- fix(rknn): name the driver in a sidecar failure by @wOvAN in #989
- fix(server): price a plan's NPU engines by driver residency by @wOvAN in #975
- feat(model_spec): ship prebuilt RK3588 engines as rknn packages by @wOvAN in #991
- Extend framework modules and ggml with opt-in CUDA FP16 paths by @0xShug0 in #988
- Extend model LoRA support and add GGUF adapter merge tools by @0xShug0 in #993
- tests: fake-engine coverage for the RKNN sidecar loader paths by @wOvAN in #995
- feat(qwen3_tts): stream the RKNN decoder with window semantics by @wOvAN in #992
- piper_tts: RKNN duration-encoder ladder by @wOvAN in #994
- Add built-in recipes for Any-TTS voice cloning and cross-language dubbing by @0xShug0 in #997
- fix(audio): discover Homebrew libsoxr on macOS by @0xShug0 in #998
- Add explicit SheetSage2 encoder attention tiling by @0xShug0 in #999
- feat(docker): Rockchip NPU image (arm64, pinned librknnrt) by @wOvAN in #1002
- Allow opt-in incomplete Higgs TTS generation by @0xShug0 in #1009
- Qwen3-TTS: add optimized streaming mode by @0xShug0 in #1011
- ACE-Step: reduce tokenizer graph peak VRAM by @0xShug0 in #1012
- Reduce OmniVoice generation copies and add streaming timings by @0xShug0 in #1013
- Add Kokoro voice blending by @0xShug0 in #1014
- feat: xTTS v2 support by @whoreson in #876
- (WIP) Add Ming 3.0 TTS 0.5B with voice cloning and voice design by @0xShug0 in #1015
- Restore Qwen3-TTS KV-cache alignment default to 1 by @0xShug0 in #1018
- Rename Ming 3.0 TTS family to ming3_tts by @0xShug0 in #1019
- Add LatentSync 1.6 (experimental) by @0xShug0 in #1001
- text chunking: include parentheses in is_tag_{open,close} by @apt-install-coffee in #1000
- Ming 3.0 TTS: chunk CJK text at 50 codepoints by default by @hiroki-abe-58 in #1020
- Ming 3.0 TTS: use the request's language when the language option is unset by @hiroki-abe-58 in #1021
- Kokoro: chunk Chinese and Japanese text at 100 codepoints by default by @hiroki-abe-58 in #1022
- Add opt-in NEON execution plans for Kokoro and Tone Color VC by @0xShug0 in #1024
- Chunk CJK text for MOSS-TTS-Nano at 50 codepoints by default by @hiroki-abe-58 in #1025
- Chunk Irodori-TTS text at 100 characters by default by @hiroki-abe-58 in #1027
- Fix ACE-Step planner attention output dimensions by @ZeroDot1 in #1026
- Add Ming3 Audio Tiny with expert offload and shared Ming speech runtimes by @0xShug0 in #1023
- Publish nightly Rockchip NPU Docker image by @0xShug0 in #1029
- Fix Ming3 Audio voice selection and Metal offload defaults by @0xShug0 in #1030
- Chunk CJK text for FireRedTTS3 at 80 codepoints by default by @hiroki-abe-58 in #1039
- Split long Ming 3.0 TTS voice design and freeform text like the official runtime by @hiroki-abe-58 in #1034
- Chunk Chinese and Korean text for MagpieTTS at 60 codepoints by default by @hiroki-abe-58 in #1043
- Use 100 codepoints for MagpieTTS Chinese and Korean chunks by @hiroki-abe-58 in #1044
- Add installable VocalRender-Pro VoxCPM2 package by @ZeroDot1 in #1037
- Fix MOSS-Nano CJK chunking for supplementary ideographs by @ZeroDot1 in #1036
- Preserve Ming3 audio tails shorter than one trim window by @ZeroDot1 in #1041
- Chunk CJK text for MioTTS at 120 codepoints by default by @hiroki-abe-58 in #1046
- Add opt-in VoxCPM2 AudioVAE tiling and packed projections by @0xShug0 in #1047
- Refresh Kitten TTS 2 checkpoints, Q4/Q8 packages, and CPU graph reuse by @0xShug0 in #1049
- Add opt-in TQ2_1 ternary CPU kernels and Kitten TTS 2 export by @0xShug0 in #1050
- Kitten TTS 2: add opt-in NEON HiFT vocoder by @0xShug0 in #1052
- Zerodot1/higgs887 preserve token order by @ZeroDot1 in #1038
- Reduce packed-weight loading RAM with opt-in deferred uploads by @0xShug0 in #1048
- Ignore non-finite logits in Higgs greedy sampling by @ZeroDot1 in #1032
- Cut unpunctuated CJK runs at the text chunk budget by @hiroki-abe-58 in #1045
- Support the multilingual Granite TurboCTC checkpoint by @bradmurray-dt in #1035
- Expose Qwen3 TTS optimization controls and extend cuda_perf preset by @0xShug0 in #1055
- Expose YuE2 performance controls and cuda_perf preset by @0xShug0 in #1056
- Zerodot1/ming3 parallel disk reads by @ZeroDot1 in #1042
- 🚨 Metal: accelerate 1D IM2COL with cooperative tiling by @0xShug0 in #1057
- Zerodot1/arch linux amd installer by @ZeroDot1 in #1058
- Add opt-in plain RMS norm for timestep embeddings by @davencyw in #1061
- Add Gradium Phonon TTS on the PocketTTS runtime by @davencyw in #1062
- Optimize Drama3 sampling and keep tower decode on-device by @0xShug0 in #1063
- Enable parallel expert reads on Apple without OpenMP by @0xShug0 in #1064
- UI v0.9.3: add missing models, LatentSync panel, and models-folder history by @0xShug0 in #1066
- Batch Drama3 streaming codec frames and overlap decoding with AR generation by @0xShug0 in #1065
- Precompute Drama3 Gemma scales and remove redundant CUDA decode views by @0xShug0 in #1067
- Expose Drama3 decode performance controls and default to F16 KV by @0xShug0 in #1068
- Reduce Ming3 Audio Thinker latency and KV memory by @0xShug0 in #1069
- 🚨 Expose ACE-Step performance controls and cuda_perf preset by @0xShug0 in #1073
- Fix HIP VMM access updates for growing pools by @ZeroDot1 in #1059
- gguf: read numeric array metadata in one bulk fread (Kitten-TTS 2 cold load 20 s -> 7 s) by @JoseConseco in #1072
- Echo-TTS: Implement optional Joint-CFG by @lojack5 in #773
- Expose MiniMax Music3 performance controls and compact output head by @0xShug0 in #1077
- Enable validated HIP VMM defaults in Arch AMD installer by @ZeroDot1 in #1080
- Fix large-row HIP argsort for device sampling by @0xShug0 in #1076
- Add Irodori-TTS MeanFlow support by @0xShug0 in #1083
- fix(assets): preserve direct uploads in weight norm tensor views by @ZeroDot1 in #1081
- Stage HIP device weight uploads through a bounded host buffer by @Ramon-Balaguer in #1070
- Treat empty and null speech voice values as omitted by @0xShug0 in #1086
- Avoid redundant VoxCPM2 KV cache staging and resets by @0xShug0 in #1085
- Make Drama3 device sampling opt-in outside cuda_perf by @0xShug0 in #1087
- Expose LoRA controls in Studio and unify adapter groups by @0xShug0 in #1088
- Align Arena controls with Studio styling by @0xShug0 in #1089
- Add Earshot VAD (native C++ port) by @Etherll in #1095
- Let ASR VAD chunking load any built-in VAD family by @Etherll in #1096
- Report build version and git commit in /health by @HamSanders in #1097
- BreezeTTS: opt-in fused depth graph for CUDA (
perf_mode=cuda_perf) by @Fishballllllll in #1090 - CUDA: opt-in fusion of bf16-reference rounding into rms_norm, add and SwiGLU by @Fishballllllll in #1091
- BreezeTTS: opt-in batch-2 CFG backbone decode on CUDA (
cfg_batch) by @Fishballllllll in #1092 - ci: add best-effort HIP/ROCm test prebuilts by @0xShug0 in #1100
- Add TTS workspace with reusable speakers by @0xShug0 in #1099
- Ming3 TTS: add streaming, consistent long-form voices, and embedding export by @0xShug0 in #1101
- 🚨 Extend BreezeTTS performance controls to Vulkan and Metal by @0xShug0 in #1104
New Contributors
- @simcop2387 made their first contribution in #923
- @JarodMica made their first contribution in #916
- @railep made their first contribution in #973
- @kinhatula made their first contribution in #839
- @ersensari made their first contribution in #981
- @whoreson made their first contribution in #876
- @apt-install-coffee made their first contribution in #1000
- @hiroki-abe-58 made their first contribution in #1020
- @ZeroDot1 made their first contribution in #1026
- @davencyw made their first contribution in #1061
- @JoseConseco made their first contribution in #1072
- @Ramon-Balaguer made their first contribution in #1070
- @Etherll made their first contribution in #1095
- @HamSanders made their first contribution in #1097
- @Fishballllllll made their first contribution in #1090
Full Changelog: v0.9.2...v0.9.3
What's Changed
- Run the Piper TTS duration path on the host for Vulkan as well by @wOvAN in #913
- Add an opt-in RKNN decoder sidecar for inflect_v2 by @wOvAN in #915
- Modify .gitmodules to work with https and ssh checkouts while maintaining compatibility with repo forks by @simcop2387 in #923
- New Community Model: Fish Audio Drama 3 by @JarodMica in #916
- Reduce Drama 3 VRAM with opt-in disk-backed expert offload by @0xShug0 in #927
- Add Drama 3 voice cloning and multi-speaker WebUI by @0xShug0 in #928
- Fix Drama 3 Vulkan voice conditioning with opt-in F32 encoder accumulation by @0xShug0 in #929
- Fix Drama 3 offload prefill noise and repeat consistency by @0xShug0 in #930
- Optimize Drama 3 output head for speech token candidates by @0xShug0 in #931
- Gate family unit tests on the model set whose symbols they link by @wOvAN in #925
- Keep CPU f5_tts passes inside the graph pool ceiling, and free the evicted graphs by @wOvAN in #924
- Strip the ONNX module prefix in the Inflect v2 converter by @wOvAN in #933
- Fix CPU Fish DAC reference encoding for voice cloning by @0xShug0 in #936
- Align CUDA reference-encoder precision for Fish Audio and Drama 3 by @0xShug0 in #937
- Fix C++17 lambda capture breaking HIP server builds by @0xShug0 in #939
- fix(build): compile model unit tests' own family sources into the test by @wOvAN in #932
- fix(build): split compiled-in model specs into 16 KB literals by @wOvAN in #934
- webui: list the server's voices with UI management enabled by @KKTTSPJ in #941
- docs(piper_tts): the non-finite duration abort is the plain Vulkan path, not the sidecar by @wOvAN in #946
- Split the RKNN quality numbers by what varies, in docs/build/rockchip-npu.md by @wOvAN in #945
- webui: preview server voices in config mode and label the list by content by @KKTTSPJ in #943
- Document the RK3588 IOVA ceiling for engine size by @wOvAN in #949
- perf(qwen3_tts): opt in to voicing chunked text in one decoder pass by @wOvAN in #910
- server: add default_session_options for configured and UI-loaded models by @KKTTSPJ in #944
- Support updated Kitten TTS 2 weights with BF16, Q8 and Q4 GGUFs by @dignome in #947
- perf(voxcpm2): size the step graphs by the request, not by max_length by @wOvAN in #950
- fix(f5_tts): free the evicted graphs' staged CUDA constants by @wOvAN in #953
- feat(cosyvoice3): run the flow DiT estimator on the Rockchip NPU by @wOvAN in #951
- refactor(f5_tts): drop the unused raw DiT weight store by @wOvAN in #956
- Reduce shared WavLM memory usage for MioCodec, MioTTS, and MeanVC2 by @0xShug0 in #954
- server: let voice presets carry request options by @KKTTSPJ in #942
- refactor(rknn): build every engine through one converter helper by @wOvAN in #961
- refactor(rknn): split the aperture accounting from the aperture measurements by @wOvAN in #957
- Fix Drama3 seeded repeatability with sorted expert routing by @0xShug0 in #955
- feat(tensors): add i-quant (IQ) storage types for GGUF tensor loading by @Th-Underscore in #958
- Add opt-in NEON library and native model components by @0xShug0 in #962
- Add Plapre 3 Danish TTS and voice cloning by @0xShug0 in #965
- Align instruction fields across CLI and server by @0xShug0 in #894
- Declare shared model object-library dependencies and document Plapre TTS by @0xShug0 in #966
- Add OpenWakeWord Alexa GGUF package and documentation by @0xShug0 in #968
- Restore Whistle host decoder and add opt-in NEON acceleration by @0xShug0 in #967
- Add opt-in NEON network execution for microWakeWord by @0xShug0 in #969
- feat(fish_drama3): support per-layer expert counts and speech-only head by @Th-Underscore in #959
- fix(server): count exec_plan engines in the free-memory guard by @wOvAN in #964
- server: keep config model settings on WebUI loads, and offer demo voices with --ui by @KKTTSPJ in #971
- fix(rknn): find an exec_plan engine named next to the model by @wOvAN in #963
- webui: treat configured models the same with and without UI management by @KKTTSPJ in #972
- perf(rknn): price a ladder engine by what the driver reports by @wOvAN in #970
- feat(server): support per-model backend device selection by @railep in #973
- perf(fish_dac): chunked reference encoding to cut VRAM peak by @Th-Underscore in #960
- feat(webui): add Vietnamese interface translations by @kinhatula in #839
- Drama3 base/small release cleanup by @0xShug0 in #976
- text chunking: keep closing quotes with their sentence in tag-aware mode by @5uck1ess in #977
- fish_drama3: default top_k to 4097 (no cap), matching Fish's Space by @5uck1ess in #978
New Contributors
- @simcop2387 made their first contribution in #923
- @JarodMica made their first contribution in #916
- @railep made their first contribution in #973
- @kinhatula made their first contribution in #839
Full Changelog: v0.9.2...v0.9.2-drama3-base-small
What's Changed
- feat(asr): add experimental Whistle community model by @BumpyClock in #824
- irodori_tts: accept Speaker Inversion embeddings as the speaker condition by @KKTTSPJ in #829
- irodori_tts: load checkpoints without a duration predictor by @KKTTSPJ in #832
- Add Hviske v6 Danish ASR support by @0xShug0 in #833
- Add TF-GridNet and MossFormer2 speech separation by @0xShug0 in #830
- irodori_tts: opt-in chunked codec decode and reference encode to bound peak memory by @KKTTSPJ in #835
- cuda: skip noncontributing transpose-convolution taps by @mirek190 in #801
- perf(lfm2_audio): use ggml's repacked CPU kernels for quantized weights by @ykhrustalev in #834
- feat(lfm2_audio): multi-turn speech-to-speech with client-carried history by @ykhrustalev in #838
- feat(whistle_asr): run the decoder as a GGML step graph by @BumpyClock in #843
- feat: input artifacts in server and CLI task requests by @ykhrustalev in #844
- perf(lfm2_audio): reuse the turn before's prefill and encoder output by @ykhrustalev in #845
- Add FireRed VAD and streaming support by @0xShug0 in #846
- 🚨 fix(ggml-cuda): backport the block_reduce shared-memory race fix (llama.cpp #26385) by @ykhrustalev in #837
- 🚨fix(metal): bound padded rows in scalar copy kernel by @0xShug0 in #831
- server: stream /v1/tasks/stream events as SSE on request by @ykhrustalev in #849
- Add native wake word and audio classification models by @0xShug0 in #851
- perf(lfm2_audio): keep several conversations in an S2S session by @ykhrustalev in #853
- feat(irodori_tts): cap reference audio length like Python (max_ref_sec) by @KKTTSPJ in #854
- Promote Qwen3 audio encoder, S3Gen, PReLU, ESPnet subsampling, and probability segmentation to the framework by @0xShug0 in #855
- Publish C API libraries and add Windows Vulkan portable releases by @0xShug0 in #823
- feat(webui): LFM2.5-Audio conversation panel for speech-to-speech by @ykhrustalev in #857
- Improve Qwen3 and Breeze TTS streaming and Breeze longform voice consistency by @0xShug0 in #858
- Add Higgs Audio TTS streaming with reusable graphs (WIP) by @0xShug0 in #859
- fix(webui): LFM2.5-Audio conversation panel follow-ups by @ykhrustalev in #864
- feat(runtime): continuous model-load progress in the trace log (llama.cpp-style) by @drzsdrtfg in #840
- docs: add OpenDub to Projects by @estebanstifli in #868
- Improve Kokoro, NeuTTS, and VoxCPM2 streaming by @0xShug0 in #865
- feat(sanotts): opt-in ARM NEON CPU decoder for nano voices (sanotts.cpu_decoder=neon) by @tarelerulz in #860
- parakeet_tdt: support Phonon-2 hotwords, streaming and timestamps by @mirek190 in #863
- Add streaming for MOSS-TTS-Nano, VibeVoice and Fish Audio by @0xShug0 in #870
- Centralize ASR streaming lifecycle and transcript publication by @0xShug0 in #871
- Add Voxtral 4B TTS with embedded voices and streaming by @0xShug0 in #872
- fix(higgs): use exact attention for cached-reference suffix by @K4viar in #874
- 🚨Refactor/model option normalization (1) by @0xShug0 in #875
- Fix model option handling and align specs with runtime behavior by @0xShug0 in #879
- parakeet_tdt: reduce Vulkan decoder transfers and encoder allocation peaks by @mirek190 in #869
- Refactor/ Vevo2 and ACE-Step by @0xShug0 in #882
- [Feat(Irodori-TTS)] add streaming mode by @asfdrwe in #881
- perf(moss_tts_nano): incremental decode with a K/V cache by @SkYContact in #878
- Reorganize model documentation and archive stale reports by @0xShug0 in #883
- 🚨 Refactor shared legacy and parallel server request handling by @0xShug0 in #886
- moss_tts_nano: reuse the encoded speaker reference, single_row prefill for quantized GPU runs, q8/CUDA docs follow-up by @SkYContact in #885
- mention pyVideoTrans integration by @jianchang512 in #889
- Improve model documentation, CLI help and server examples by @0xShug0 in #890
- 🚨 Refactor/Normalize model synthesis route aliases by @0xShug0 in #893
- Fix Vevo2 Whisper overrides and Qwen3 TTS language codes by @0xShug0 in #899
- Extract the shared librknnrt plumbing used by the model-local NPU sidecars by @wOvAN in #900
- Add CTC word timestamps to Granite Speech 5.0 TurboCTC by @bradmurray-dt in #903
- Fix Vulkan ceiling division overflow by @0xShug0 in #905
- miotts: apply repetition_penalty once per distinct token (HF semantics) by @zlatkoviz-png in #906
- WebUI: improve model downloads and add missing models by @0xShug0 in #907
- Add an opt-in RKNN speech decoder sidecar for qwen3_tts by @wOvAN in #896
- Add an opt-in RKNN VITS decoder sidecar for piper_tts by @wOvAN in #897
- Add an opt-in RKNN iSTFTNet generator sidecar for kokoro_tts by @wOvAN in #904
- Package server config examples as a separate release asset by @0xShug0 in #909
- Add a shared Voice Library page to the WebUI by @0xShug0 in #912
- Size the piper_tts graph arenas for long unchunked latents by @wOvAN in #911
- Pre-release cleanup: server documentation and WebUI presentation by @0xShug0 in #919
- Prefer current runtime model contracts with explicit embedded-spec selection by @0xShug0 in #918
- Unify RKNN and NEON component execution plans by @0xShug0 in #922
New Contributors
- @BumpyClock made their first contribution in #824
- @KKTTSPJ made their first contribution in #829
- @estebanstifli made their first contribution in #868
- @tarelerulz made their first contribution in #860
- @K4viar made their first contribution in #874
- @SkYContact made their first contribution in #878
- @jianchang512 made their first contribution in #889
- @wOvAN made their first contribution in #900
- @bradmurray-dt made their first contribution in #903
- @zlatkoviz-png made their first contribution in #906
Full Changelog: v0.9.1...v0.9.2
@ Every self-hosted release, weekly
One Monday email with the week's stable releases across the directory.
More in AI
AudioMuse-AIOpen-source, self-hosted sonic-analysis tool that builds playlists, a 2D music map and semantic search for Navidrome, Jellyfin, Emby, Plex, LMS and Lyrion libraries.
auliteSelf-hosted HTTP proxy that checks AI interactions for EU AI Act compliance, records them in a tamper-proof audit log, and produces legal-grade PDF reports.
Basic MemoryAI conversations that actually remember. Never re-explain your project to your AI again. Join our Discord: https://discord.gg/tyvKNccgqN
beamA self-hostable, open-source platform for running serverless AI workloads with GPU inference, code sandboxes, and background task queues.
beelzebubOpen-source, self-hosted deception runtime that turns SSH, HTTP, TCP, Telnet, MCP decoys into honeypots, configured via YAML rules, with LLM replies and Go plugins.
blindaiBlindAI is an open-source AI privacy solution using Intel SGX enclaves for confidential inference. Self-host models or query hosted ones via Python client.
BlinkoSelf-hosted AI note-taking app using RAG for intelligent search. Supports Markdown, native desktop apps on macOS/Windows/Android/Linux, PostgreSQL storage.
BokeBoxSelf-hosted private AI podcast studio turning videos, links, articles, and meetings into spoken episodes with customizable persona, voice, and MCP/plugin extensibility.
brain-dbSelf-hosted memory database for AI agents with fused semantic, lexical, graph, and temporal retrieval. Single Rust core, binary wire protocol, Apache-2.0.
brain.mdSelf-hosted markdown notes with MCP server for AI agents. Single Bun binary, local vector search via WASM ONNX, per-folder permissions, optional auth. AGPL.
Breeze TTS 2Self-hosted open-weight TTS model with voice cloning, design from descriptions, and voice direction. Streaming REST API and CLI for real-time synthesis on NVIDIA GPUs.
BricksLLMOpen-source Go AI gateway that meters LLM traffic with per-key rate limits, spend caps, and PII masking, covering OpenAI-style APIs, Anthropic, Azure and vLLM.
0 comments