![[Google VP9] HDR Encoding with ffmpeg](https://thuannguyen.net/wp-content/uploads/2019/11/googel-vp9.jpg)
High Dynamic Range (HDR) video is a maturing technology. There are today a number of gradually-harmonizing standards.
For a maintained picture of HDR video evolution see the WikiPedia article High-dynamic-range video.
FFmpeg can be used along with VP9 to both (1) compress HDR sources to Standard Dynamic Range (SDR) outputs or (2) to repackage an HDR source into HDR web-friendly formats.
Input sources
Currently, when encoding HDR with VP9, any high bit depth (10-bit) codecs supported by FFmpeg can be used as input (for example, VP9 Profile 2 and 3, H.264 (10-bit), DNxHR, HEVC, ProRes, et al.). Also see the YouTube support article Upload High Dynamic Range (HDR) videos.
Encoded output reach:
Currently and in general, target devices supported include any device that can decode VP9 Profile 2 and output to an HDR display.
- Chromecast Ultra + HDR-capable TV
- Chrome 64 Canary on Windows 10 Fall Creators Update, with HDR flag turned on. (This is a good debugging tool).
- Any HDR TV with VP9 Profile 2
- All Samsung 2017 HDR models (flatpanelshd.com list) and 2016 models starting with “KS”
- All LG 2017 HDR models (also some 2016 LG G6 models)
- On mobile, devices that support hardware-accelerated VP9 Profile 2 10-bit HDR decode (Meizu Pro 7)
- UHD Bluray from 2017
FFmpeg command-line settings to enable VP9 Profile 2 and HDR EOTFs
Your FFmpeg will need to have been built with 10-bit (or even 12-bit) support.
In the examples below we use a statically-built ffmpeg with 10-bit support. You may use our build script to make your own, or follow the guidance on the FFmpeg site.
Following is a usable FFmpeg command line for HDR encoding to SMPTE 2084 (PQ EOTF) standards with VP9:
ffmpeg -i strobe_scientist_18Mbps.webm -b:v 18000000 -pass 1 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
-maxrate 26800000 -minrate 8040000 -profile:v 2 -vcodec libvpx-vp9 /dev/null && \
ffmpeg -i strobe_scientist_18Mbps.webm -b:v 18000000 -pass 2 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
-maxrate 26800000 -minrate 8040000 -profile:v 2 -vcodec libvpx-vp9 \
strobe_scientist_18Mbps.webmBreaking out the key parameters introduced here:
| ffmpeg | The static build with 10-bit support |
|---|---|
-pass 1 | HDR requires 2-pass encoding. In this example, the first pass is output to /dev/null and processed in memory during the second pass. |
-pix_fmt yuv420p10le | Sets YUV 4:2:0 10-bit pixel format |
-color_primaries 9 | Sets BT2020 See page 5 and 6 of this document for more detail. |
-color_trc 16 | 16 sets Transfer Characteristics to SMPTE 2084. PQ 18 sets Transfer Characteristics SMPTE 2086 HLG |
-colorspace 9 | colorspace is the “matrix_coefficients.” This should be set consistent with how the content was mastered. (e.g., bt709, bt2020_ncl). In this example, it should be 9 for bt2020_ncl. For further reference see page 12 of this document |
-color_range 1 | Color range (0 = unspecified, 1 = mpeg/studio/tv, 2 = jpeg/full/pc) |
-profile:v 2 | For HDR, a profile or 2 or 3 is required. |
-vcodec libvpx-vp9 | Use the VP9 encoder. |
Example encodes
The following examples all use the source file strobe_scientist.mkv
As a general note on bitrates with HDR encoding, a 25-30% higher bitrate than equivalent 8-bit SDR encoding is recommended. The examples typically target 18Mbps. Good HDR results can be achieved with 12Mbps bitrates with optimal tuning.
The encodes were performed on an Ubuntu Linux system with the following specifications:
- Processor: 4x Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz
- Memory (RAM): 8060MB (1492MB used)
- Graphics: Intel HD Graphics 530 (Skylake GT2)
- OS: Ubuntu 16.04 LTS
Convert to YUV 4:2:0 10-bit PQ
Example A: 6Mbps 4k 2-pass
ffmpeg -y -report -i strobe_scientist.mkv -b:v 6000000 -speed 4 -pass 1 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
-maxrate 8000000 -minrate 4000000 \
-profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
ffmpeg -y -report -i strobe_scientist.mkv -b:v 6000000 -pass 2 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
-maxrate 8000000 -minrate 4000000 \
-profile:v 2 -vcodec libvpx-vp9 \
2pass_HDR_strobe_scientist_6Mbps-static.webm- Output file: 2pass_HDR_strobe_scientist_6Mbps-static.webm
- Encode time: 142m30s
- File size on disc: 51.2MB
Example B1: 18Mbps 4k 2-pass
ffmpeg -y -report -i strobe_scientist.mkv -b:v **18000000** -speed 4 -pass 1 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
-maxrate **20800000** -minrate **15040000** \
-profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -pass 2 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
-maxrate **20800000** -minrate **15040000** \
-profile:v 2 -vcodec libvpx-vp9 \
2pass_HDR_strobe_scientist_18Mbps-static.webm- Output file: 2pass_HDR_strobe_scientist_18Mbps-static.webm
- Encode time: 213m32s
- File size on disc: 161MB
Example B2: 18Mbps 1080p 2-pass
ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -speed 4 -pass 1 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
-maxrate 20800000 -minrate 15040000 \
-profile:v 2 **-vf ****scale=-1:1080**** **-vcodec libvpx-vp9 \
-f webm /dev/null && \
ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -pass 2 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
-maxrate 20800000 -minrate 15040000 \
-profile:v 2 **-vf ****scale=-1:1080**** **-vcodec libvpx-vp9 \
1080_2pass_HDR_strobe_scientist_18Mbps-static.webm- Output file: 1080_2pass_HDR_strobe_scientist_18Mbps-static.webm
- Encode time: 113m9s
- File size on disc: 160MB
Convert HDR to SDR with a look-up table (LUT)
Converting HDR to SDR requires the transcoder to understand how to map brightness levels and colors to the smaller BT.709 color space and non-HDR range of brightness. For best results, an LUT describing how to perform that mapping — usually specific to the content — can be used. Examples C and D show how to use a LUT. We used bt2020_to_bt709_example.cube
Example C: 18Mbps 4k HDR to SDR with LUT
ffmpeg -i strobe_scientist.mkv -y -b:v 18000000 -speed 4 -pass 1 \
-pix_fmt yuv420p \
-color_primaries 1 -color_trc 1 -colorspace 1 -color_range 1 \
-maxrate 26800000 -minrate 8040000 -profile:v 0 \
-vf scale=-1:-1:in_color_matrix=bt2020,format=rgb48,lut3d=bt2020_to_bt709_example.cube,scale=-1:-1:out_color_matrix=bt709 \
-vcodec libvpx-vp9 -f webm /dev/null && \
ffmpeg -i strobe_scientist.mkv -y -b:v 18000000 -pass 2 \
-pix_fmt yuv420p \
-color_primaries 1 -color_trc 1 -colorspace 1 -color_range 1 \
-maxrate 26800000 -minrate 8040000 -profile:v 0 \
-vf scale=-1:-1:in_color_matrix=bt2020,format=rgb48,lut3d=bt2020_to_bt709_example.cube,scale=-1:-1:out_color_matrix=bt709 \
-vcodec libvpx-vp9 -f webm SDR_strobe_scientist_18Mbps-static.webm- Output file: SDR_strobe_scientist_18Mbps-static.webm
- Encode time: 124m23s
- File size on disc: 165MB
Convert HLG to VP9 HLG10 10-bit PQ
Requires an HLG source. We used the input file strobe_scientist_hlg.mkv
Example D: 18Mbps 4k 2-Pass HLG
ffmpeg -y -i strobe_scientist_hlg.mkv -b:v 18000000 -pass 1 -speed 4 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 18 -colorspace 9 -color_range 1 \
-maxrate 26800000 -minrate 8040000 \
-profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
ffmpeg -y -i strobe_scientist_hlg.mkv -b:v 18000000 -pass 2 \
-pix_fmt yuv420p10le \
-color_primaries 9 -color_trc 18 -colorspace 9 -color_range 1 \
-maxrate 26800000 -minrate 8040000 \
-profile:v 2 -vcodec libvpx-vp9 \
HLG_HDR_strobe_scientist_18Mbps-static.webm
- Output file: HLG_HDR_strobe_scientist_18Mbps-static.webm
- Encode time: 186m10s
- File size on disc: 165MB
Summary
The output files are of varying size. The input was 4.3GB.
| Example | Target Output | Size on Disk | Encode Time |
|---|---|---|---|
| A | 6Mbps 4k Unscaled HDR | 51.2MB | 142m |
| B1 | 18Mbps 4k unscaled HDR | 161MB | 213m |
| B2 | 18Mbps 1080 scaled HDR | 160MB | 113m |
| C | 18Mbps HDR to SDR unscaled | 165MB | 124m |
| D | 18Mbps HLG conversion | 165MB | 168m |
Note that 2-pass encoding is recommended for HDR, since 1-pass can vary in certain configurations.
Tools needed to signal HDR in WebM and MP4 output (Matroska command-line tools)
At the time of writing, FFmpeg doesn’t provide a mechanism for specifying SMPTE 2086 static metadata in WebM/MKV, although it will propagate it from an input stream when the stream contains it.
The mkvmerge tool, part of mkvtoolnix, can be used to insert or modify this metadata if your application requires it.
Following is an example of inserting HDR metadata into one of the files created above. It is especially suited to preparing HDR content for upload to YouTube.
mkvmerge \
-o HDR_strobe_scientist_18Mbps.mkv\
--colour-matrix 0:9 \
--colour-range 0:1 \
--colour-transfer-characteristics 0:16 \
--colour-primaries 0:9 \
--max-content-light 0:1000 \
--max-frame-light 0:300 \
--max-luminance 0:1000 \
--min-luminance 0:0.01 \
--chromaticity-coordinates 0:0.68,0.32,0.265,0.690,0.15,0.06 \
--white-colour-coordinates 0:0.3127,0.3290 \
HDR_strobe_scientist_18Mbps.webmThis example produces the file HDR_strobe_scientist_18Mbps.mkv, in 0.6 seconds.
