Tencent Open-Sources AuK AI: Word Replacement, Vocal Isolation, and Voice Cloning

The Tencent Hunyuan team has unveiled AuK, an open-source universal model for speech generation, editing, and restoration using text instructions. The development handles tasks that previously required multiple separate plugins: it can clone voices from a short sample, fix slips of the tongue in a mixed audio track, alter speaker emotions, and isolate vocals from music tracks. Weights and code have been published under the permissive MIT license.

Text-Based Audio Editing and Paralinguistics

AuK’s key advantage over standard text-to-speech (TTS) generators lies in its deep editing capabilities for existing audio. As noted on the model’s Hugging Face page, control relies on natural-language text prompts:

• Targeted Inpainting: Users can select an audio segment and replace a word or insert a phrase — the model synthesizes the missing portion while matching the speaker’s original timbre and breath;
• Paralinguistic Edits: Text instructions can modify emotional tone, eliminate regional accents, turn speech into a whisper, or add natural laughter and coughing;
• Acoustic Cleanup: The system suppresses room reverberation, filters out background noise, and separates overlapping speakers into isolated tracks.

The scale of the training dataset is massive: according to the technical report on arXiv, engineers trained the algorithm on 3.03 billion instruction-audio pairs, equivalent to nearly two million hours of live speech.

🚀 AuK is officially here. Nano banana🍌 for audio

An open-source foundation model for unified speech generation and editing.
Natural-language instructions + reference audio. One interface.
Zero-shot TTS. Instruction-controlled generation. Content editing. Whisper-conversion.… pic.twitter.com/AIOOzPYFNL

— Tencent Hy (@TencentHunyuan) September 10, 2026

Hardware Requirements and Fast AuK-Flash Variant

Social media posts often oversimplify the architecture, calling it a lightweight 1.5B parameter model. In practice, the pipeline demands a powerful GPU: the generative module alone takes up 6.12GB, but it loads alongside the Qwen2.5-Omni-3B encoder and a VAE compression module into VRAM.

For PCs with lower memory capacity, the team released a distilled version called AuK-Flash: it generates audio in just four steps and runs 4.5 times faster than the base algorithm while offering nearly identical speech intelligibility.

The code is hosted in Tencent’s official GitHub repository. Out of the box, the developers provided a CLI client, a Gradio web demo, and custom nodes for ComfyUI, turning the release into an accessible free tool for local video voiceovers and podcasts.