What is ‘Training Footage’?
Training is how Lipdub is able to achieve such great results, but a certain amount of footage is required per actor, for training to work well.
See Training for more details on training requirements.
How do I format my ‘Training Footage’?
Lipdub performs at its best when the training footage is consistent. Therefore, we do not recommend simply uploading a whole film or TV show or 1hr podcast in the “Additional Footage for Training” section.
See the section regarding Footage Consistency on the Training page.
Where do I upload ‘Training Footage’?
Check the ‘Uploading Training Footage’ section on the How do I upload to lipdub? page.
When do I provide ‘Training Footage’?
If you do NOT have at least 1 minute of footage for each speaking actor in your footage, getting additional footage for training is recommended.
Where do I get ‘Training Footage’?
If you have the opportunity, please film your own training footage. See below for directions on how to do so.
Below are your options for getting training footages, and their positives and negatives.
(1) Filming the Additional Footage On-Set:
Pros | Cons |
Exact same lighting | Requires more effort on the user to actively film an additional 1min for each camera setup. & may slow production down a little. |
Exact same camera angle | |
Exact same location | |
Exact same actor’s look (same makeup, hairstyle, clothes) |
(2) Outtakes/Dailies:
Pros | Cons |
Exact same lighting | Lack of lip shape diversity.
Often in outtakes they are repeating the same lines over and over. LipDub wants a diverse set of lip shapes per actor per camera pose to train on. |
Exact same camera angle | A lot of useless footage.
Often Dailies is footage that is shot for the entire time the scene runs for. This results in significant amounts of time that the footage is not useful for LipDub. LipDub only wants to see the actor in the same pose, moving their mouths. When the actor is not speaking or facing the camera it is useless data and can actually interfere & might result in poorer results.
We recommend editing the Dailies Footage to where the actor you’d like to lip-sync is visibly moving their mouths in frame. |
Exact same location | |
Exact same actor’s look (same makeup, hairstyle, clothes) |
(3) Shots from other scenes in the Film/Show/Video.
Pros | Cons |
Might be useful Lipdub AI to train on if the previous two options are not possible. | Different lighting |
Different camera angle | |
Different location | |
Different actor look (make up, hairstyle, clothes) |
🎬 Capturing ‘Training Footage’ on Set
What should Actors say in the training footage?
Actors can do one of two things when filming additional footage:
- Regular Conversation: For each camera set-up, record the actor(s) having a conversation for, minimum, 1 minute.
- Read from Script: For each camera set-up, have the actor(s) read a 1 minute script (see below)
Training Footage should be consistent with Lipdub Footage
Training footage should match the footage being lip synced in these three ways:
- Pose: if they are moving their head in the shot, have them move their head the same way.
- Expressiveness: If they are happy and animated, have them perform in a happy and animated way. Conveying large emotions where the mouth is larger than normal such as anger or screaming is more difficult with LipDub if the training data is the actor is speaking normally.
- Camera Movement: If the camera moves in the footage being lip synced, have the camera move the same way in the training footage.
- Shoot another take with the same lighting & camera angle.
- Have the actor repeat the same movement just slower while talking or reading the script for 1 min.
- If the actor finished their movement and it has not been 1 min, then repeat the movement.
Training Script
Please download the script below:
- The birch canoe slid on the smooth planks.
- Glue the sheets to the dark blue background.
- It's easy to tell the depth of a well.
- These days, a chicken leg is a rare dish.
- Rice is often served in round bowls.
- The juice of lemons makes a fine punch.
- The box was thrown beside the parked truck.
- The hogs were fed chopped corn and garbage.
- Four hours of steady work faced us.
- A large size and stockings are hard to sell.
- The boy was there when the sun rose.
- A rod is used to catch pink salmon.
- The source of a huge river is the clear spring.
- Kick the ball straight and follow through.
- Help the woman get back to her feet.
- A pot of tea helps to pass the evening.
- Smokey fires lack flame and heat.
- The soft cushion broke the man's fall.
- The salt breeze came across the sea.
- The girl at the booth sold 50 bonds.
- The small pup gnawed a hole in the sock.
- The fish twisted and turned on the bent hook.
- Press the pants and sew a button on the vest.
- The swan dive was far short of perfect.
- The beauty of the view stunned the young boy.
- Two blue fish swam in the tank.
- Her purse was full of useless trash.
- The colt reared and threw the tall rider.
- It snowed, rained, and hailed the same morning.
- Read verse out loud for pleasure.
- Hoist the load to your left shoulder.
- Take the winding path to reach the lake.
Why does LipDub like this script?
It is a series of sentences that ensures a wide diversity of mouth shapes. However, following this script is absolutely NOT necessary. You can simply have a regular conversation in the same camera-setup & angle & have the actor pose the same way.