FAQs

❓Q1 — Does the original audio in the source video matter?

• The "Translate with AI" feature requires the original language audio in your source video that you upload. Check out our video guide ⁠Single-Actor Guide to learn how to use this automatic translation feature.

• LipDub has an experimental feature called ⁠Maintain Expression Feature ✨ that uses the original audio to train an AI model based on the audio viseme/phonemes.

• If you are trying to lip-sync an AI avatar that doesn't have any source audio, then simply uploading a video without any audio is also fine

• LipDub is language agnostic. You can upload any language and LipDub will lip-sync the actor to it.

• LipDub is HIGHLY sensitive. If you upload a dub audio with background music or SFX, we will lipsync the actor to instruments or SFX if the actor is on screen. This is why we recommend voice-isolated dialogue stems for the dub audio.

• Audio for Training data does not matter! Training data videos will just used to for training the AI model from a visual standpoint. it will not use the audio in the "Additional Footage for Training" section in the ⁠Multi-Actor Guide

❓Q2 — Does Lipdub support variable frame rate?

No. Lipdub will not generate a result on a video with variable frame rate. Please convert to constant frame rate before uploading to Lipdub. Variable Frame Rate is common in videos captured on phone cameras.

❓Q3 — Does Lipdub have an API?

Yes. Please reach out to us via email to get more details. support@lipdub.ai

❓Q4 — How long does it take to upload/process, train and generate?

1) Uploading/Processing: Takes 15min - 1 hr on average 2) Training: Takes 1-2 hours on average (regardless of amount of footage of the actor) 3) Generate: 10 min/ per minute of video to render on average.

❓Q5 — What influences the processing time?

1. Resolution of the video 2. The size of the face in the frame 3. Number of faces in each frame 4. The duration of video So as a tip, one of the biggest ways you can speed up this step is by editing out any shots which contain a large number of faces (e.g. a crowd scene) if you know you only need to dub one actor.

❓Q6 — At what stage do I add my dub audio?

Dubbed audio is uploaded on the last step —> “4. Upload Audio & Generate”

❓Q7 — How do I request a credit refund if something went wrong?

If a Lipdub process fails (training or generating), a refund will automatically be added back to your account. Please reach out to our team if you do not see the refund or are confused.

❓Q8 — How do I stop my training job?

Training cannot be terminated early at this time.

❓Q9 — How do I know when training is finished?

An email will be sent to you once training the AI model is finished for each actor!

❓Q10 — What do I do If I run into an error at any point during my workflow using Lipdub Platform?

If you run into a system error (ie. a video fails to upload and process, a result fails to generate, etc.) we do not recommend repeating the process more than twice. Please reach out to the Lipdub support team for a remedy to the situation

❓Q11 — What happens if a video is longer than my dub audio file? What happens if its shorter?

Here's a few scenarios to be aware of:

• Uploaded Audio is shorter than video - Video is Cut to match with the audio.

• Uploaded Audio is longer than video - Video extends the last frame as a freeze frame until the audio is complete.

• Uploaded Audio is shorter than video - Video length is respected and mouth is shut closed as the rest of the video is silent.

• Uploaded Audio is longer than video - Video is stops at the end of the source video. It does not extend past the length of the source video.

❓Q12 — What is a LipDub scene?

Check out the article below:

❓Q13 — Supported video formats?

Check out the article below:

❓Q14 — Supported audio formats?

Check out the article below:

❓Q15 — How do I capture additional training footage?

Check out the article below:

❓Q16 — How many languages does LipDub support?

Depends on how you get your audio that you want to lip-sync to. (see below)

• Any sound/language in the world. LipDub is language agnostic. So LipDub will lip-sync to all known languages, Dothraki (Game of Thrones), Klingon (StarTrek), Birds chirping, ANYTHING. I

• note: LipDub is extremely sensitive to any noise so be sure to use clean audio stems for best results!

• Currently there are 29 languages that are supported for automatic translation and lip-sync.

• note: For optimal results to have clean audio stems (i.e. no background music/SFX) in the original language audio.

1. English

2. Japanese

3. Chinese

4. German

5. Hindi

6. French

7. Korean

8. Portuguese

9. Italian

10. Spanish

11. Indonesian

12. Dutch

13. Turkish

14. Filipino

15. Polish

16. Swedish

17. Bulgarian

18. Romanian

19. Arabic

20. Czech

21. Greek

22. Finnish

23. Croatian

24. Malay

25. Slovak

26. Danish

27. Tamil

28. Ukrainian

29. Russian

❓Q17 — Do you support multiple dialects per language?

We currently offer these 29 languages for automatic translation directly in the app.

1. You want to automatically translate from American English to a different English accent (e.g. British, or Scottish, or Mexican English accent)
• Accent control is something we are working towards! Not yet available.

2. You want to automatically translate from Mandarin to Shanghainese
• Adding more dialects per language is something we are working towards!
• Currently Mandarin is offered, but not Shanghainese. Please see the list of supported languages above, we will update them with more dialects as soon as we can support them!

❓Q18 — Can you do Animated characters?

LipDub recognizes human looking faces. So if the animated character does not resemble a human face then LipDub may not be possible because there wasn’t any human face detected.