It is important to understand the compatible video formats for LipDub AI, as they vary widely.
Consistency is Key: Video specifications should be consistent across all footage (including footage used for training). This includes frame rate, color space, grading, resolution, codec, format, etc.
Doing any of the following will increase the likelihood that artifacts will appear in the rendered video:
- Uploading a mix of ungraded footage and color graded footage. (meaning the footage looks very different visually)
- Uploading a mix of different looking footage (i.e. one video that is tinted yellow while another is tinted blue)
- Uploading a 1080p video as training footage when the video you’d like to lip-sync is in 4K
Unsupported Video Files:
- Image sequences such as EXRs
- Interlaced footage
- Anamorphic footage
- Variable Frame Rate footage
- Videos with multiple video streams
Note: SDR is supported. but HDR videos will result in color difference from the original.
Unsupported videos should be converted before uploading to LipDub platform.
FILE FORMATS
These are the supported file formats:
Video Format | Comment |
MOV | SUPPORTED |
MP4 | SUPPORTED |
CODEC
These are the supported video codecs:
Video Codec | Comment |
H.264 | SUPPORTED |
Apple ProRes (422, 422HQ, 4444, 4444 XQ) | SUPPORTED |
FRAME RATE (FPS)
These are the supported frame rates (FPS):
Video Frame Rate | Comment |
23.976 | SUPPORTED |
24 | SUPPORTED |
25 | SUPPORTED |
29.97 | SUPPORTED |
30 | SUPPORTED |
IMPORTANT: ‘Variable Frame Rate’ is not supported by LipDub AI. If you are using footage from a phone, be mindful as they tend to film in a variable frame rate. Convert footage to a ‘Constant Frame Rate’ before uploading to the LipDub platform.
COLOR SPACE
These are the supported color spaces:
Video Color-space | Comment |
sRGB | SUPPORTED |
Rec. 709 | SUPPORTED |
BIT-DEPTH
These are the supported bit-depths:
Video Bit-Depth | Comment |
1 | SUPPORTED |
2 | SUPPORTED |
4 | SUPPORTED |
8 | SUPPORTED |
10 | Lipdub will produce a result, pipeline will convert a face crop to 8 bit and then convert back to 10 bit in final result. |
12 | Lipdub will produce a result, pipeline will convert a face crop to 8 bit and then convert back to 12 bit in final result. |
16 | Lipdub will produce a result, pipeline will convert a face crop to 8 bit and then convert back to 16 bit in final result. |
32 | Lipdub will produce a result, pipeline will convert a face crop to 8 bit and then convert back to 32 bit in final result. |
RESOLUTION
These are the supported video resolutions:
Name | Resolution | Aspect Ratio | Pixel Size | Supported? |
SD (Standard Definition) | 480p | 4:3 | 640 x 480 | SUPPORTED |
HD (High Definition) | 720p | 16:9 | 1280 x 720 | SUPPORTED |
Full HD (FHD) | 1080p | 16:9 | 1920 x 1080 | SUPPORTED |
2K video (Quad HD) | 2k or 1440 | 16:9 | 2560 x 1440 | SUPPORTED |
4K video or Ultra HD (UHD) | 4k or 2160p | 16:9 | 3840 x 2160 | SUPPORTED |
PIXEL ASPECT
These are the supported pixel aspect ratios:
Pixel Aspect Ratio (PAR) | Comment |
1:1 | SUPPORTED |
2:1 | DOES NOT SUPPORT |