- Home
- AI Video Lip Sync Tools
- MakeItTalk
MakeItTalk
What it is
MakeItTalk is free, open-source research software for generating expressive talking-head videos from one facial image and an audio file. Its method separates audio content, which drives lip and nearby facial movement, from speaker information, which affects expressions and other head dynamics. The project describes use with portraits, paintings, sketches, 2D cartoons, manga, and stylized caricatures, including faces not seen during training. The repository includes pretrained models, module training code, and Google Colab demos for natural faces. For human faces and paintings, the README specifies a centered 256×256 JPG portrait and WAV audio. Command-line options can adjust lip motion on the x and y axes and head displacement. Local use requires Python 3.6, ffmpeg, and Python packages; cartoon face warping on Ubuntu also requires winehq-stable. Custom puppets remain a to-do item, and several training sections are unfinished. The CC BY-NC 4.0 license permits noncommercial use under its terms.
For and against
For
- Generates talking heads from image and audio
- Supports portraits, cartoons, manga, and paintings
- Pretrained models are included
- Motion can be adjusted by command line
- Google Colab demos are provided
Against
- Custom puppets remain unfinished
- Several training sections are marked Todo
- Local setup requires Python 3.6 and ffmpeg
- Commercial use is not permitted by the license
The verdict
MakeItTalk provides flexible research code for expressive image-and-audio animation across several visual styles. Its setup requirements, unfinished items, and noncommercial license matter for practical use.
Who it suits
It suits developers and researchers working with local or hosted research code for animated portrait videos. Users must be comfortable with Python, ffmpeg, required packages, and the stated noncommercial license.
Plans and prices
Open-source research code
CC BY-NC 4.0 · noncommercial use only
Read 4 Oct 2026 · github.com
On the record
- Available code
- The repository includes pretrained models, training code for modules, and Google Colab demos for natural faces.github.com · read 4 Oct 2026
- Custom puppets
- The README lists creating custom puppets as a to-do item.github.com · read 4 Oct 2026
- Image and audio input
- For human faces and paintings, the README instructs users to provide a centered 256×256 JPG portrait and WAV audio file.github.com · read 4 Oct 2026
- License
- The repository license is Creative Commons Attribution-NonCommercial 4.0 International, which permits licensed use for noncommercial purposes subject to its terms.github.com · read 4 Oct 2026
- Local requirements
- The README specifies Python 3.6, ffmpeg, and Python packages; cartoon face warping on Ubuntu also requires winehq-stable.github.com · read 4 Oct 2026
- Motion controls
- Optional command-line arguments adjust lip motion along the x and y axes and head-motion displacement.github.com · read 4 Oct 2026
- Purpose
- MakeItTalk generates expressive talking-head videos from a single facial image, using audio as the only input.github.com · read 4 Oct 2026
- Security and privacy
- The repository page does not state security certifications or privacy practices.github.com · read 4 Oct 2026
- Speaker-aware animation
- The method separates audio content, which controls lip and nearby facial motion, from speaker information, which influences expressions and other talking-head dynamics.github.com · read 4 Oct 2026
- Supported imagery
- The project describes use with human portraits, paintings, sketches, 2D cartoons, manga, and stylized caricatures.github.com · read 4 Oct 2026
- Training status
- Voice conversion, speaker-aware branch, and image-to-image translation training sections are marked “Todo...”.github.com · read 4 Oct 2026
- Unseen inputs
- The method is described as generalizing to faces and characters not observed during training.github.com · read 4 Oct 2026
