The LipDub platform has its strengths, but can also be challenging if you don’t know all its in’s and outs.
Here’s a quick list of common user errors that can cause delayed or even poor results with EASYDUB.
EasyDub is essentially Advanced Workflow but simplified. It combines all the advanced workflow steps into one, making it much easier to use and get a result. This ease of use & automation however may lead to un-intentional errors that cause the output to be poor. See below on the different scenarios where EasyDub may fail.
Scenario 1: Output is not lip-synced for the entire video
- What is happening? - the face in the background is actually being lip-synced, and not the main actress. This is a known face detection issue that can sometimes happen with EasyDub. LipDub face detection is highly sensitive, so its always best to ensure only 1 face is viewable to prevent this from happening.
- Solution -
- Option 1 - Please restart this project using Advanced Workflow. This advanced workflow gives users ability to select face detection tracks of which face you'd like to lip-sync, and which ones you do not want to lip-sync.
- Option 2 - If you really want to use EasyDub you can! But please be sure to upload a different video where only 1 face is visible throughout.
Scenario 2: Output is only partially lip-synced
- What is happening? - There is a cut in the video which makes LipDub think there is 2 unique "identities". EasyDub only dubs one identity. So it only dubs one of the cuts.
- Solution - Please restart this project using Advanced Workflow.
Scenario 3: Output is not lip-synced at all
- What is happening? - the original source video does not have mouth movements. Therefore, the AI model is only trained on when the actor has closed mouth. This will yield to poor and unnatural lip-sync. LipDub requires some mouth movement of the actor before it trains an AI model.
- Solution:
- Option 1 - Create a new EasyDub project. This time upload a video that includes mouth movement for the actor.
- Option 2 - Create a new Advanced Workflow project. This time upload the closed mouth video in the Lipdub clips section & upload additional footage for training where the same actor in same lighting & environment is moving their mouth/talking.