레퍼런스 투 비디오: AI 캐릭터와 세트를 일관되게 유지하기
더 긴 프롬프트가 아니라 이미지 레퍼런스로 AI 영상의 캐릭터와 세트를 샷에서 샷으로 일관되게 유지하세요. 그리고 같은 씬으로 ScreenWeaver 파이프라인 두 개를 비교합니다.
이 강의에서 배울 내용
- 텍스트 설명만으로는 생성할 때마다 얼굴을 일관되게 유지할 수 없는 이유
- 네 가지 모드: 텍스트 투 비디오, 이미지 투 비디오, 레퍼런스 투 비디오, 첫 프레임과 마지막 프레임
- 샷에 필요한 레퍼런스와 깔끔한 레퍼런스의 조건
- 프롬프트에서 각 레퍼런스 이미지를 그 역할에 연결하는 법
- ScreenWeaver 파이프라인 Reference to video와 Storyboard panels to video의 작동 방식과 각각을 쓸 때
- 클립 사이의 연속성을 위해 첫 프레임과 마지막 프레임을 쓰는 법
- 샷을 망치는 네 가지 레퍼런스 함정
텍스트만으로는 얼굴을 지킬 수 없습니다. 캐릭터를 완벽하게 자세히 묘사해도 두 번 생성하면 서로 다른 두 사람이 나옵니다. 이 강의는 이미지 레퍼런스로 AI 영상의 캐릭터, 세트, 소품을 일관되게 유지하는 법을 보여 줍니다. 텍스트 투 비디오부터 첫 프레임과 마지막 프레임까지 이미지로 영상 모델을 움직이는 네 가지 방식, 깔끔한 레퍼런스의 조건, 프롬프트에서 각 이미지를 그 역할에 연결하는 법을 배웁니다. 그다음 ScreenWeaver의 두 파이프라인을 봅니다. Reference to video는 프롬프트 디렉터가 샷 액션으로 씬 전체를 위한 브리프 하나를 쓰게 하고, 프로덕션 시트가 자동으로 연결됩니다. Storyboard panels to video는 최대 14개의 패널을 그리드로 모아 모델이 당신의 프레이밍을 따르게 합니다. The Photo의 같은 씬을 두 방식으로 생성해 나란히 봅니다. 마지막으로 연속성을 위한 첫 프레임과 마지막 프레임, 그리고 레퍼런스 기반 샷을 망치는 네 가지 함정을 다룹니다.
전체 스크립트
영어 내레이션의 스크립트입니다.
0:04Text alone can't hold a face. You can describe Amelie in perfect detail and two generations will still give you two different women. References are what turn a description into a specific person, a specific car, a specific photograph. In this lecture, you'll learn the four ways to drive a video model with images and how Screenweaver's two pipelines use them.
0:30Four modes, from loosest to tightest. Text to video. Words only, maximum freedom, minimum consistency. Image to video. One start image and the model animates it. Reference to video. Several images that define who and what appears while the model stages the shot itself. And first and last frame. You give the start and the end and the model finds the way between them. The more images you give, the more you control and the less the model invents.
1:02What should you give as references? For each character in the shot, their production sheet. For the location, one clean image of the set. For key props, a sheet of the object alone. And the art direction image for the overall look. Keep it to what the shot needs. Every extra reference competes for the model's attention.
1:23Quality matters more than quantity. A reference should be sharp, evenly lit and show the subject clearly with nothing that competes. No busy background, no second person, no text. A blurry or cluttered reference gives you a blurry, cluttered character.
1:41Then bind each reference to its job in the prompt. Our real prompt starts with a line of asset bindings. Thomas's car is image one. The photograph is image two. Then the text says, the vehicle interior and ambient lighting of image one and image two governing the prop's physical structure and the print on it. Name it and give it a job.
2:05In Screenweaver, the first pipeline is called reference to video. It takes the action of each shot. A prompt director writes one brief for the whole scene and a scene video node generates one clip. The characters, places and objects you mention in your shot actions are resolved into their production sheets and bound to the prompt automatically. Note what it doesn't use, your storyboard panels. The model stages the scene from your words and your references.
2:35The second pipeline is storyboard panels to video. It assembles your storyboard panels into a single grid image, up to 14 panels and sends it to the scene video node together with your art direction text and image and the grid tells the model the order and the composition of the shots. Never the look. The look comes from the art direction and the sheets.
3:00So which one should you use? Reference to video when you want the model to stage freely from your actions. Storyboard panels to video when your framing has to be respected shot by shot. Here's the photo generated both ways. On the left, reference to video. On the right, storyboard panels to video. Same story, same actors. Look at how the framing follows the previous on the right.
3:29In practice, I often start with reference to video to see how the model stages the scene on its own. If the staging works, I keep it. If the framing drifts away from what I designed, I switch to storyboard panels to video and the grid pulls the camera back to my previous.
3:45First and last frames are your tool for continuity between clips. Take the last frame of one clip and use it as the first frame of the next. The second clip starts exactly where the first one ended. Same pose, same light, same position. It's the cleanest way to continue an action across a cut or to extend a moment that needs more time than one generation allows.
4:09Start frames also let you control exactly where an action begins. Give the model a first frame where Thomas is already turned toward Emily. Then it won't spend two seconds of your clip turning.
4:21Watch out for four traps. A reference that contains a set, like our photograph shot on the back seat, which moved Emily into the back. Two references that contradict each other, like a sheet with a beige coat and a mood board image with a red one. Too many references for one shot and photorealistic faces on models that filter them. When a reference -based shot fails, check the references before you touch the prompt.
4:50Your exercise. For one scene, list the references each shot needs and write the bindings. Which image governs what? Then generate the scene with both pipelines and compare. In the next lecture, we'll deal with the shots that still go wrong and how to fix them without starting over.