What Is VVC?

通用视频编码(VVC)是由ITU-T和ISO/IEC专家组成的联合协作小组(JVET)起草的编解码器。, 它是ITU-T视频编码专家组(VCEG)和ISO/IEC运动图像专家组(MPEG)的合作伙伴关系。,” as MPEG explains. The codec is designed to meet upcoming needs in videoconferencing, OTT streaming, mobile telephony, and contribution.

According to the final requirements document (Version 5), 目标压缩性能与HEVC主配置文件相比，具有相同的感知质量，编码复杂度约为HEVC的10倍或更多，比特率降低30%至50%. Early third-party performance results show improvements in the 27% to 33% range. VVC将是一种需要支付版税的编解码器，但版税将如何设定和管理尚不清楚.

VVC should be finalized by the end of 2020.

VVC Overview

Figure 1 shows the development timeline of recent codec standards with VVC in red at the far right. As with previous standards, VVC is jointly promulgated by ISO/IEC via the Moving Pictures Experts Group, or MPEG, and the International Telecommunication Union (ITU). Initial work on VVC was conducted jointly with ISO/IEC JTC1 SC29/WG11 (MPEG) from October 2015 to October 2017 by the Joint Video ExplorationTeam (JVET). In October 2017 a new team was assembled with “the same acronym but called Joint Video Experts Team (JVET).”

Figure 1. Progression of video standards. From Trends and Recent Developments in Video Coding Standardization by Jens-Rainer Ohm and Mathias Wien (via SlideShare).

The initial requirements document called for a technology that could manage the following:

Picture formats: At least from VGA to 8Kx4K.
Color space and color sampling: YCbCr color spaces with 4:2:0 sampling, 10 bits per component, YCbCr/RGB 4:4:4 and YCbCr 4:2:2, bit depths up to 16 bits per component, with High dynamic range and wide-gamut color, along with auxiliary channels for transparency, depth, and more.
Frame rates: Starting with 0Hz and upper limits defined by levels.

这种编解码器被命名为通用视频编码，因为它“意味着非常通用，可以满足从低分辨率和低比特率到高分辨率和高比特率的所有视频需求。, HDR, 360 omnidirectional and so on,” explains Christian Feldmann, a codec engineer with Bitmovin. Figure 2 illustrates that idea, listing the test sequences defined in the call for proposals, which included a range of standard dynamic range 1080p and 4K content, as well as 8 HDR videos and 5 360° videos.

Figure 2. Test sequences in the VVC Call for Proposals Figure by Mathias Wien.

VVC Formation

每个MPEG规范听起来都像是在制作詹姆斯·邦德(James Bond)电影期间组装的. With VVC, the initial Request for Contributions on Future Video Compression Technology was published in MPEG Meeting 111 in Geneva, Switzerland (in February 2015), while the initial Requirements Document was published in Warsaw, Poland, in June 2015.

In Hobart, Australia, MPEG issued a Joint Call for Evidence on Video Compression With Capability beyond HEVC in April 2017. In July 2017, in Torino, Italy, MPEG published the results of the evidence, 发现“在相当多的测试用例中实现了与HEVC相比的显着增益?. Comparable subjective quality (i.e. 在SDR和HDR测试用例中，观察到比特率比HEVC低40%至50%. Moreover, even higher bit rate savings was observed for some test sequences... As a result of the evaluation of submitted evidence, 结论是，有证据表明，在进一步发展和标准化之后，存在可能显著优于HEVC标准压缩能力的技术.”

This lead to a Joint Call for Proposals on Video Compression With Capability Beyond HEVC published in Macau in October 2017, and the start of the formal standardization process. 本文件中提供的时间表显示，该标准将于2020年10月完成.

From there, VVC ran through multiple working drafts through Draft 5, which was published in Geneva in March 2019. The most recent Test Model (6) was published in Gothenburg, Sweden in July 2019.

Technical Components of VVC

“As in most preceding standards, VVC has a block-based hybrid coding architecture, 结合图间和图内预测和熵编码的变换编码”(来自 VVC Test Model 6).

Figure 3. The block diagram of the VVC encoder (from VVC Test Model 6).

There are a range of mostly evolutionary technologies in VVC, including intra prediction, inter prediction, transformation, in-loop filtering, palette coding, block partitioning, affine motion, and decoder side search. These are summarized in Figure 4, from a 2018 presentation entitled 多功能视频编码-视频压缩超越HEVC:编码工具的SDR和360°视频 by Mathias Wien, a lecturer at RWTH Aachen University. Note that I combined two slides into one to create the figure.

Figure 4. New features in VVC.

You can find a more detailed explanation of the new features in an article entitled VVC Video Codec – The Next Generation Codec by Bitmovin’s Christian Feldmann. Feldmann also compared AV1 and VVC in a presentation at Streaming Media East 2019. Download the presentation handout and watch the video.

Early Performance Tests for VVC

As mentioned above, responses 联合呼吁证据表明，SDR和HDR测试的节省幅度在40%至50%之间，某些片段的节省幅度更高, though lower gains were reported in the 360° video sequences. These results convinced MPEG to greenlight VVC. Since then, there have been several comparisons involving primarily VVC, AV1, and HEVC.

Before diving in, let’s clarify the nomenclature. JEM stands for the Joint Exploration Model，这是一个实验性的软件版本，包含了VVC规范中的所有工具. It appears that the JEM has been succeeded by the VVC Test Model (VTM) since the initial set of tests described below. Again, VTM is reference software that contains all the tools in the codec, and I could find no tests using the VTM software.

HM is the HEVC Test Model that also contains all the encoding tools in the spec and is not a commercial encoder. AV1 is an actual shipping version of the AV1 codec. JM is the equivalent for H.264, again not a shipping version of the codec but a reference version with all the available tools in the spec.

虽然这些名称听起来很迟钝，并且使图5和图6所示的比较解释变得复杂, the names identify the actual encoder used to produce the comparison. For this reason, these names are more precise than designating the results VVC, HEVC, or H.264, which can be produced by any compliant codec produced by any encoding vendor.

Back on point, one of the more notable comparisons was produced by the BBC, published in June 2018 and updated in June 2019. 测试涉及高清和超高清片段，“代表典型的广播内容”，并通过客观和主观测试进行测量. As Figure 5 shows, 与HEVC相比，AV1的效率较低，而VVC(用绿色标记JEM)在超高清内容下的效率比HEVC高27%，在高清视频内容下的效率比HEVC高33%.

Figure 5. BBC results comparing VVC (JEM in green), AV1 (yellow), and HEVC (HM in orange).

Together with a company called b<>com, Harmonic produced a codec study comparing H.264, HEVC, VP9, AV1, and VVC. Figure 6 shows the BD-Results, 哪个列出了每个编解码器产生与它所比较的编解码器相同质量所需的比特率节省(负数)或剩余, with PSNR on the left and VMAF on the right.

Focusing on the bottom row, as gauged with PSNR, VVC can produce the same quality as H.264 (JM) at a 63% bitrate savings, the same quality as HEVC (HM) at a 24% bitrate savings, and the same quality as AV1 with a 20% bitrate savings. With VMAF, the numbers are a savings of 65% (H.264), 30% (HEVC), and 18% (AV1).

Figure 6. BD-Rate savings for VVC as compared to H.264, HEVC, and AV1. Image edited to highlight JEM performance (no numbers were changed).

Not to get off track, but in all fairness to AV1, these results differ from what others have published, including Facebook, Bitmovin, and Moscow State University, which compared AV1 to HEVC or VP9, not VVC, using actual shipping codecs like x264 and x265, not reference models. In Streaming Media tests also performed with x264 and x265, AV1 produced the same quality as x265 with a data rate savings of 18%.

Regarding VVC, 这些结果是有希望的，但一旦我们知道最终的工具集将包含在任何特定的VVC许可版本中，就需要重新校准. To explain, take a look at the final section.

VVC Licensing Structure

VVC will be royalty-bearing. As we discuss in our article, Inside MPEG's Ambitious Plan to Launch 3 Video Codecs in 2020, it appears that a third party, the Media Coding Industry Forum, will create profiles and levels based upon licensing terms proposed by each IP owner. It also seems likely that there will be multiple patent pools. 在此模式中，池定价可以确定每个配置文件中包含哪些编码工具.

How this will work has not been made public yet, but presumably, if one pool charges three times the royalty of the other pools, 在该池中表示的工具可能被放置在很少实现者会选择使用的概要文件中. Obviously, this impacts competitive quality; the results above include all existing tools, including those that may not be in the profile selected by any particular user. 这种模式是由MPEG创建的，以避免HEVC出现的专利拖延，在HEVC中，几个专利持有人在编解码器实现后很长一段时间才宣布他们提出的版税结构.

这是第一个可以根据版税率或其他许可条款将专利编码工具降级到不同配置文件的编解码器. With HEVC and H.264, all tools were included in the final licensable codec, 因此，使用包含所有工具的参考模型来评估编解码器性能是公平的. However, with VVC, until we know the final royalty-related profile structure, 我们不知道参考模型的性能如何与特定实现者部署的版本相关联. 底线是，在我们了解VVC的许可结构之前，我们真的无法衡量VVC的质量或性能.

Summing Up

VVC显然是一种cpu密集型编解码器，在广泛部署之前需要硬件支持, which pushes deployment to at least two years after the spec is finalized. If the licensing structure isn’t crystal clear at that time, expect few product developers to start incorporating VVC into their product plans. For the same reason, until the IP situation is clear, 任何质量或其他性能评估可能与任何实现者实际部署的内容几乎没有相似之处.