How to Add Dialogue and Lip Sync to AI Video
Learn how to add dialogue and lip sync to AI video with three routes: native lines, a separate voice or voice-over. Then direct a synthetic voice like an actor.
Chapters (7)
In this lesson you will learn
- The three routes for dialogue in AI video: native, separate voice and voice-over
- How to write a native line in the prompt, with the words, the language and the delivery
- How to bring a voice file into ScreenWeaver with a character's Add Voice button and an audio node
- How to direct a synthetic voice with a written acting note
- Why voice-over is often the easiest and most cinematic route
- The consent rule for cloning a real person's voice
Spoken lines are where AI films break most often: the wrong voice, the wrong timing, lips that don't match. This lesson covers the three routes for dialogue in AI video and when to use each one. Native dialogue is generated with the picture, with the voice, the lip movement and the ambience in the same take, the way Thomas's line in The Photo was made. A separate voice, made first with a text-to-speech model, gives you control over performance and timing, and in ScreenWeaver it goes into an audio node that can be sent to video models that accept audio. Voice-over needs no lip sync at all, an idea Lost Garden uses with knights that have no faces and no mouths. You also learn to direct a synthetic voice with an acting note, keep each character sounding like the same person, and respect the consent rule for voice cloning.
Full transcript
0:04Spoken lines are where AI films break most often – the wrong voice, the wrong timing, lips that don't match. In this lecture you'll learn the three routes for dialogue, when to use each one and how to direct a synthetic voice like an actor.
0:19Route 1 – native dialogue, generated with the picture. Some video models produce the voice, the lip movement and the ambience in the same take. That's how Thomas' line in the photo was made. The words are in the prompt, in quotes, with the language and the delivery, softly. Keep native lines short, one or two sentences, and keep the speaker clearly in frame.
1:02Route 2 – a separate voice, then the picture. You generate the voice first, with a text -to -speech model, so you control the performance and the timing, then the voice drives the video. In Screenweaver, a character's story element has an Add Voice button. It creates an audio node, where you upload the voice file, and that audio can be sent as a reference to the video models that accept audio. Dedicated lip -sync tools exist, too, outside the app, for shots that need frame -perfect sync.
1:31Note that the app doesn't generate voices itself – you bring the audio file. For a short film, a text -to -speech service with voice design is usually enough. For recurring characters, save the voice settings of each one, so every scene sounds like the same person.
1:46Whichever read you choose, direct the voice like an actor. Write an acting note in words – calm, grounded, a little tired, a southern English accent. The narration of this course was generated exactly that way, from a single voice and a single acting note. Keep one voice per character for the whole film. Generate a full scene in one pass, so the time beholds, and listen for mispronounced names before you move on.
2:12For Thomas's line, the note could be quiet, almost a whisper, a man who has rehearsed this sentence for days and still isn't ready to say it. A few words of direction change everything about a line.
2:26Route 3 – voiceover, with no lips to sync at all. An inner monologue, an orator, a letter read over images. It's the easiest route, and often the most cinematic one. Lost Garden uses a version of this idea. Its nights have no faces and no mouths, so there is nothing to sync. Emotion comes from posture, light and voice.
2:54One rule with no exception – never clone a real person's voice without their written consent. And check the rules of the platform where your film will play – some require you to disclose synthetic voices.
3:09Your exercise? Pick one line of dialogue from your script and produce it two ways – native, inside the generated shot, and as a separate voice with an acting note. Compare them in the cut. In the next lecture, we'll build the full soundtrack, edit the film and export it.
The full course
YouTube playlistSection 1 · Welcome to AI Filmmaking
Section 2 · Writing for AI Film
Section 3 · Design Your Film
Section 4 · Shot List & Storyboard
Section 5 · Generate the Film
- 5.1Choosing the Right Video Model for Each Shot5:18
- 5.2Prompting Like a Director: Camera, Action, Performance5:55
- 5.3Reference-to-Video: Keeping Characters and Sets On Model5:13
- 5.4Fixing Bad Shots: Repair, Reframe, Regenerate4:53
- 5.5Workshop, Part 1: Script & Storyboard, Step by Step6:21
- 5.6Workshop, Part 2: Production, Step by Step5:02