It is important to understand the compatible video formats for LipDub AI, as they vary widely.
Consistency is Key: Video specifications should be consistent across all footage (including footage used for training). This includes frame rate, color space, grading, resolution, codec, format, etc.
Doing any of the following will increase the likelihood that artifacts will appear in the rendered video:
- Uploading a mix of ungraded footage and color graded footage. (meaning the footage looks very different visually)
- Uploading a mix of different looking footage (i.e. one video that is tinted yellow while another is tinted blue)
- Uploading a 1080p video as training footage when the video you’d like to lip-sync is in 4K
Unsupported Video Files:
- Image sequences such as EXRs
- Interlaced footage
- Anamorphic footage
- Variable Frame Rate footage
Unsupported videos should be converted before uploading to LipDub platform.
FILE FORMATS
Video Format | Comment |
MOV | SUPPORTED |
MP4 | SUPPORTED |
CODEC
Video Codec | Comment |
H.264 | SUPPORTED |
Apple ProRes (422, 422HQ, 4444, 4444 XQ) | SUPPORTED |
FRAME RATE
Video Frame Rate | Comment |
23.976 | SUPPORTED |
24 | SUPPORTED |
25 | SUPPORTED |
29.97 | SUPPORTED |
30 | SUPPORTED |
48 | SUPPORTED |
50 | SUPPORTED |
59.94 | SUPPORTED |
IMPORTANT: ‘Variable Frame Rate’ is not supported by LipDub AI. If you are using footage from a phone, be mindful as they tend to film in a variable frame rate. Convert footage to a ‘Constant Frame Rate’ before uploading to the LipDub platform.
COLORSPACE
Video Color-space | Comment |
sRGB | SUPPORTED |
Rec. 709 | SUPPORTED |
BIT-DEPTH
Video Bit-Depth | Comment |
1 | SUPPORTED |
2 | SUPPORTED |
4 | SUPPORTED |
8 | SUPPORTED |
10 | Lipdub will produce a result, pipeline will convert a face crop to 8 bit and then convert back to 10 bit in final result. |
12 | Lipdub will produce a result, pipeline will convert a face crop to 8 bit and then convert back to 12 bit in final result. |
16 | Lipdub will produce a result, pipeline will convert a face crop to 8 bit and then convert back to 16 bit in final result. |
32 | Lipdub will produce a result, pipeline will convert a face crop to 8 bit and then convert back to 32 bit in final result. |
RESOLUTION
Name | Resolution | Aspect Ratio | Pixel Size | Supported? |
SD (Standard Definition) | 480p | 4:3 | 640 x 480 | SUPPORTED |
HD (High Definition) | 720p | 16:9 | 1280 x 720 | SUPPORTED |
Full HD (FHD) | 1080p | 16:9 | 1920 x 1080 | SUPPORTED |
2K video (Quad HD) | 2k or 1440 | 16:9 | 2560 x 1440 | SUPPORTED |
4K video or Ultra HD (UHD) | 4k or 2160p | 16:9 | 3840 x 2160 | SUPPORTED |
8K video or Full Ultra HD | 8k or 4320p | 16:9 | 7680 x 4320 | SUPPORTED |
PIXEL ASPECT
Pixel Aspect Ratio (PAR) | Comment |
1:1 | SUPPORTED |
2:1 | DOES NOT SUPPORT |