If your video has any of the below examples, we want you to be cautious of the end result. LipDub will produce a result even if your footage contains everything listed below.

If you have any questions join our community discord! Check out our link article to learn how to join. But if you prefer to email we’re happy to help so don’t hesitate to reach out at support@lipdub.ai.

Object Interference is when an object (hand, microphone, etc.) comes into the LipDub mask area.

LipDub can generate a result despite interferences in front of the face, but the end result might create strange visual artifacts.

Interference but the object is stable & consistent:

Interference but the object is unstable & not-consistent:

Side profile shots are more difficult for Lipdub AI to lip sync.

However, we recently made some significant jumps in improvement for results with side pose!

Example:

This only applies to graphics that fall within the face mask region.

Lipdub will have difficulty re-generating the text perfectly. It is recommended you apply any graphics to the video after LipDub has been applied.

LipDub will dub the face, even when there are transition effects. This can create strange looking results, as LipDub will not match the effect perfectly.

Lipdub is quite good at handling small beards! But it should be noted that high-frequency details on the face are always harder to handle, especially during extreme close ups camera positions.

If possible, beards should be avoided.

If possible, try avoiding camera angles like the one’s show below.

These camera angles are fairly uncommon, and as a result LipDub may have a more difficult time perfectly pasting back their mask area of their mouth compared to a straight to camera position.

Camera position: Bottom up

Camera position: Top down

For example: 10, 12, 16, 32-bit depth

When the face is so close to the camera where only the mouth is visible. LipDub will be unable to detect a face, and therefore will not Lip-sync this actor.

E.g. Additional footage for training is color graded blue tint but the clip to Lip-sync at the very top of this screenshot is ungraded.

When there is very little light in a scene, it is challenging for LipDub to identify the face in the dark. This is make it difficult or impossible in some cases to lip-sync the face.

LipDub performs its work on a 1024px box.

When faces are larger than this 1024px box LipDub will have to down-sample and then up-sample back to the larger pixel range.

This may cause artifacts to appear on the final render result and we recommend if possible to keep the actors face within this 1024px box.

Example: This is 4K video with a extreme close up face.