Recorded from the app, not sped up. Unmute for the voice.
Talking Avatar
Turn one portrait into a face you can talk to. You speak or type, an LLM answers, a local voice reads the answer, and a diffusion model animates the portrait in step with the voice, streamed to a browser tab as it is generated.
Everything except the LLM runs on your own machine. Built and measured on an RTX 5070 Laptop GPU with 8 GB of memory.
This page is a recording, not a live demo. The app needs a GPU with about
6 GB of memory, so it runs locally rather than in this Space.
Animated characters too
Drawn and 3D-animated characters keep their style. These three were generated for this project and animated by the same app, on the same laptop.
- A clearly drawn mouth lip-syncs well. A mouth that is a thin line, or hidden under a moustache, barely moves.
- Faces soften while speaking, and some characters soften much more than others.
- Flat cartoons often have no face the detector recognises; the picture is then used as it is.
How it works
- Hearfaster-whisper turns your speech into text, on the CPU.
- ThinkAn LLM streams a reply (Meta Model API or Claude).
- SpeakKokoro voices each sentence as it arrives, on the CPU.
- AnimateSoulX-FlashHead Lite draws 24 frames for every 0.96 s of speech.
- StreamFrames and audio travel together and play on one clock, so lips and voice stay locked.
Measured on the laptop
25 fps512 x 512 output
1.3xreal time: about 720 ms to render each 0.96 s chunk
5.7 GBpeak GPU memory
0 to 1 framelip-sync offset between voice and mouth
Speed by configuration
Time to render one 0.96 s chunk, engine alone.
| Denoise steps | Plain | With torch.compile |
|---|---|---|
| 2 (default) | 667 ms (1.44x) | 585 ms (1.64x) |
| 4 | 963 ms (1.00x) | 808 ms (1.19x) |
Limits
- Head and shoulders only, 512 x 512. No body, hands or camera motion.
- Replies start after a second or two, not instantly: each chunk has to be rendered before it can play.
- On a laptop the GPU has to be kept busy between chunks or its clock drops and the video falls behind, which costs about 108 W while the avatar is on screen.
- The FlashHead weights carry no clear licence text, so this is for tinkering. Only animate faces you have the right to use.